Go back

Time is Honey

38m 31s

Time is Honey

Sunil Makrani, a computer engineer, noticed the internet crashing during the 9/11 attacks due to a sudden surge in demand, prompting him to explore how systems could better handle such fluctuations. Inspired by the efficient, decentralized behavior of honeybees, he collaborated with systems engineer Craig Tovy to develop a model inspired by bee foraging. Bees allocate themselves to flower patches based on the shortest round-trip time, balancing efficiency without a central leader. Sunil and Craig applied this "honeybee algorithm" to internet server distribution, where websites act like flower patches and servers like bees. When a video or event goes viral, servers "dance" via digital pings to recruit idle ones, dynamically sharing the load. Tests showed this algorithm achieved up to 98% efficiency—far surpassing human-designed or predictive methods—proving that decentralized, present-aware systems outperform centralized, future-guessing ones. The solution was shared openly, leading to real-world adoption in data centers worldwide, improving responsiveness and reducing latency. Beyond the internet, the algorithm has been used in forecasting, electric vehicle design, and medical imaging. The story highlights nature’s elegant problem-solving, showing that sometimes the best solutions emerge not from human foresight, but from simple, adaptive rules that respond to the present moment. This bio-inspired approach demonstrates how evolution has solved complex coordination problems, and how such insights can revolutionize technology.

Transcription

6777 Words, 35003 Characters

English
- Wait, you're listening. - Okay. - Come on. - Okay. - Come on. (clears throat) - Dordless. - Listening. - To radio lab. - Radio lab. - From WNYC. - To C. - C. - Yeah. - Lulu. - Hello. - Hey, should we start? - Let's do it. - Okay. All right, so today we're gonna start with a guy, a very sweet, very tall guy named Sunil Makrani. - Yeah, hi, I'm Sunil Makrani. - Where did you grow up? And were you just a computer kid? Like you just love computers or how did this all start? - So do you wanna start from there? - Yeah, I mean, a little bit. - Yeah, so I was actually born in Kenya. - But he grew up between India and the UK. - Right. - He studied hard. - Bachelor of the Master's degree in electrical engineering. - And in 1989, he lands a job at IBM. Just as the world is encountering this new thing called the internet. - What about this internet thing? Do you know anything about this? - Lots of useful information in here. You can get news, recipes. - So, Sunil is like, okay, why not go and study communication engineering? - He goes to Oxford to get his PhD and one day, near the beginning of the semester, while he is on one of the desktop computers in the computer lab. His whole department, including him, gets an email from one of his professors who's an American guy. And the email just says, - Hey guys, America is under attack. Come down to the common room and then watch, right? - It came out of the clear blue sky on a mild fall morning in Manhattan. - Oh, it's 2001. - Yes, 2001. Sunil's standing there in horror. - Smoke appeared everywhere, as though a mist has suddenly settled on the floor. - I mean, he has a million questions. Like, who did this? Why is America a war? So, he goes online to get any sort of new information. And he just can't. - A lot of the websites were just not responding. - Some websites had just crashed. Others would like just keep loading, but they're never fully low. - There's one point there, it was such overwhelming demand for news that the results were serving only text. - Plane text. - 'Cause there were so many people trying to get to those websites. - 'Cause there was just so many people trying to get access. - So much hunger of people trying to figure out what the hell is going on here. - Correct. - He's like, why is it that at the very moment when I and the world want to access something the most, that's when I can't access it? And this used to happen all the time. - As millions of people flooded the system. - Last week a picture of a dress was posted on Tumblr. - Some website. - Healthcare.gov. - Or picture. - The person broke the internet. - Or video. - Suddenly become popular. - In just 15 minutes. - A Pokemon guy who wants to- - And you get this crash of people. - And you get all the past initial predictions. - This flash flood. - And it breaks the internet. - And this situation. - A demand that is beyond what they plan for. - Sunil became kind of obsessed with it. - And so I ended up looking at websites, how they architect some of these infrastructures. - And he's like, there's gotta be a way to fix this. - Yeah, like what can I apply to solve that problem? - Meanwhile, Sunil's wife is working in Atlanta. - And we were doing back and forth, which in Oxford and Atlanta. - And at a certain point, he's having kind of a hard time with this internet problem. - Because how do you design a system for the future when you don't know what the future is? - And one day, he just thinks to himself, Georgia Tech is down the street. Maybe someone there could help me. - Point me to some direction that I could take. - So he just emailed some people in the engineering school. - I said, oh, I'm a PhD student looking to discuss some ideas. Can I come and see you? And that was basically it. I didn't describe the problem in my email. I didn't really expect anything. Anything substantial, but within 30 minutes, he gets a response from a guy named Craig Tovy. - So he come by my office. - And then a very tall guy knocks on my door and it says, oh, I'm looking for Craig. - This, of course, is Craig Tovy. - Hi, good to meet you. - Craig's a systems engineer. - Operations research. - His job is to make huge operations run smoothly. Factories, shipping routes, that kind of thing. - So anyway, I walked in his office, we sat down, we started talking, I started describing the problem. - So Sunil is like, look, I'm trying to fix the internet. I'm trying to stop it from breaking every time one of these internet flash floods happen. - He didn't say a lot actually at the beginning. He just kept listening. And then 25, 30 minutes later, you know, suddenly Craig Tovy said, oh, oh, oh, wait. - Craig stands up and then went back to his desk and pulled out a paper. - And he plops it down in front of Sunil. It is called the pattern and effectiveness of forager allocation among flower patches by honeybee colonies. - So at the time, I thought, why are we talking about honeybees? - Yeah, why are we talking about honeybees? - Well, Craig had this hunch that bees had something to teach Sunil and all of us really, because turns out bees are sort of a model of how to thrive in an uncertain world. - Hmm. - Okay. - So that's the story I want to talk about today. The story of how a multi-billion dollar tech industry used a trick they learned for millions of years of honeybee evolution to build the internet as we know it. - Okay, giddy up. - Okay, so Craig and his bee study, it really began with his collaborator. - I'd like you to refer to me as Tom. - Dr. Tom Sealy. - You could introduce me as Dr. or professor, but let's quickly switch to Tom. - He's a retired professor of neurobiology at Cornell and one of the world's top experts on honeybees. - I did pretty much pioneer the study of how honeybees live in the wild on their own. - And he started doing that work more than 60 years ago when he was a kid. - A swarm of bees moved into a large black walnut, not far from my parents' house. - And Tom says he would watch those bees flying in and out of this knot hole in the tree. - Like shooting stars, zooming off in all directions. - And he'd see them and he'd be like, where are they? Where are they going? Like he kind of knew where they were going. - There were fields of a dairy farm up on the hillside. - With lots of flowers full of nectar and pollen for them to eat. - And they're probably going up to that. - Like I know where they're going, but how do they know where they're going? Like what is going on here? - They're not, aren't they just like flying? And if they see some pollen or nectar, they're bringing home? - No, no, it's way harder and more complicated than that. - Why? How so? - There's many different reasons. And this is all stuff Tom would eventually learn in college and grad school. Okay, so like first, flower patches are not evenly or clearly distributed. They're not just everywhere. - Okay, fair. - They're also not all blooming at the same time. So there might be flowers that are blooming at certain times the year or even certain times of the day. Also, you need to find the flowers in bloom that still have nectar because you're in competition with all these other pollinators. - Right. - And you have to do all your food gathering before winter comes, 'cause you know, then all the flowers go away. And you have to get something like, for, guess how many flowers, if you're a bee? Guess how many flowers you have to hit to get like a little bottle of honey that you would find in a grocery store? - Oh, oh, okay. Like one of those bare little squeezy bottles. - Yeah, one of those little bare squeezy bottles. - How many flowers went into that? That's such a nice question. Uh, I don't know, like 10,000? - Two million. - Two million? - Two million. And the hive needs like the equivalent of 200 squeezy bottles of honey to survive through the winter. - Wow. All of this is an efficiency game. Like you only have so much time when the sun is shining, when these flowers are blooming that you can hit 'em. - Okay. - Okay, so they have to do all of this. It's really hard. And they have like nobody, they have no boss. Nobody is in charge. - It's queen bee isn't in charge? - So that's the thing. So back in Aristotle's time, that was the idea. - For many centuries, people thought the queen was a ruler, but no, that's not correct. - The queen's whole job is to make babies. - Yeah. - She's not telling anybody else what to do. - Nobody is. - The colony is intelligent in some way that the individual bees are not. - And when Craig found out about this. - It started with NPR. - From a radio story. Tom Silly talking about honeybees. - He was just like, what? - I was just in awe. - Because at the time, Craig was working on robots. He was trying to get a group of them to build a car. - It was an unsolved problem. - And he was kind of stumped. - I have no idea how to get these robots to work together to be as a group like more intelligent than the individual stupid robots. But here were these honeybees. - Doing exactly that. And I'm thinking, wow, let's understand how the bees are doing that and then we can copy that and apply it to robots. - And the first, Craig thought this would be pretty straightforward. - I thought that biologists had figured out everything. - But turns out they hadn't. - That's right. - If somebody asked me how much we know about how a honeybe colony works, I'd say 50%. - And you know, 50%'s not nothing. They knew, for example, about. the waggle dance, which you might have heard of? - Yes, I've heard that bees will waggle, and what is the waggle though? - I mean, it's quite sophisticated. It's a dance that bees do to show other bees where they just came from. - Oh, so it's like a choreographed map. It's like a map dance. - It's like a map dance, and this is like a piece of the puzzle. - Oh yeah, Von Frisch won the Nobel Prize, but with the biologist had not figured out. - Is the bigger picture? What Tom would call. - The wisdom of the hive. - And that is what Tom was working on when Craig called it. - And Tom said, "Well, come help me run these experiments." - And the two of them would end up doing an experiment together that would give us a little peek into that wisdom of the hive, and would eventually become the paper that many years later, Craig would slap down in front of Suniel in his office. - It still vivided my memory. - I mean, in fact, this is one of the best weeks of my life. - It's July 1991. - Okay, now this is in the Adironducts upstate New York. - Like way upstate. - Close to the Canadian border. - Craig had to drive on these dark back roads. - For an hour and a half, two hours. - Looking for a little sign for this research station. - cranberry biological station. - Cranberry biological station, okay. - Cranberry lake, I'm sorry, cranberry lake. - Cranberry lake, yeah. - I always amazed that he pulled it off, that he found this biological station up in the middle of the woods, deep at night. - And the reason Tom studies bees at this place way out in the middle of nowhere is because. - There are no naturally occurring honey bee colonies. - What? Why would you study bees in a place where there are no bees? - Right, 'cause the absence of bees lets you set up kind of controlled experiments without any other bees messing it up. - That's right, it's a BYO-B, bring your own bees set up. - So Tom brought an experiment, you know, a little colony of about 4,000 bees. - Okay. - And, you know, after breakfast, we go out to where the hive is that Tom has set up. - A wooden box. - Two feet high. - Kind of like a beekeeper would have, but with one big difference. - The front is all glass. - Basically a transparent hive. - Yeah. - So you can watch the bees inside. - And each bee had to be individually recognizable. - How do you do that? - So you put a little two-digit number on their thorax and a little dab of paint on their abdomen. - Until they're painted, they're anonymous members of a colony. But once they're painted, you get to know them. Some are nervous Nellies. Others come running. - Really? - Yeah. - So they have little personalities? - Oh, definitely. Definitely. Get up in the morning and get things going. Others hang out until 11 o'clock. - Anyway, when the sun comes out on that first morning, a big wave of painted bees whooshes out. - Except for the slackers who are sleeping in. - I'm just flying. Except for the lazy ones who are sleeping in. But pretty quickly, the ones who have gone to work, they start flying out to these feeders. These fake flower patches that Tom has put out for them. - Imagine a glass petri dish where we have high sugar content water. - It's a little buffet for them. - Yeah. - That's right. - And so the way the experiment works is that there are two fake flower patches. But they're not equal. One is better than the other. And what they want to see is like, how do the bees suss that out? And how do they make it kind of a collective decision to send more bees to one rather than the other? - Oh, okay, oh, so this is it. This is where you see, maybe it's not just random, like they might get to glimpse, have intelligence. - Yeah, because remember, this is the whole thing, right? Like, winters coming, they're on a clock and let's see how they prioritize how they make that decision. - Huh. And then how do they make one like fake flower patch better than the other? - Well, there's lots of ways. - You know, we can put 1.5 molar sugar. - One feeder might have sweeter sugar water than the other. We can change it from one and a half to two and see how the behavior of the bees changes. - Or one might be bigger and one smaller. - If you had a lot of bees coming to the same feeder, some of them would have to wait. - Craig says, for the purposes of explaining this, just imagine this situation. - One feeder which is close. - About five minutes from the hive and another one that's. - Ten minutes to fly out to the flower patch, fill a per stomach with nectar and fly back. - A little further away. - Okay. - Tom is at the hive recording each bee is at least the hive and each bee is at comes back. - Also, have you ever, I mean, probably not, but like, have you ever been stung by a bee? Are you afraid of bees at all? - I am somewhat allergic to bees. - Oh. - I did not know that. - But, so I am scared of them, but ordinarily, honey bees are not all that aggressive. - Anyway, right away, the guy's noted. - There are a lot of bees going to this five minute feeder. - The feeder close to the hive is blowing up, and when they follow those bees back to the hive. - To my eyes, it just looks like chaos, but if you're tongue silly, you'll spot the waggle dance. - The bee, effectively saying, "Hey, go that way." - And pretty soon, they see another bee from that close by feeder come in and dance. - Hey, go that way. - And another? - Hey, go that way. - And another? - When each time those bees dance, they're bringing more bees back to the five minute patch. - Hey, go that way. - Hey, go that way. - Hey, go that way. - And as all that's happening, every now and then, a bee comes in and dances for the 10 minute patch. - Hey, go this way. - But. - Go that way. - Go that way. - Hey, go that way. - There will be more bees going to that five minute patch, and there will be fewer bees going to the 10 minute patch. - Hey, go this way. - Just because the bee's coming back twice as often. - Go that way. - Hey, go that way. - Hey, go that way. - Go that way. - Hey, go that way. - Oh, okay, so the closer flower patch gets more bees saying, "Go that way, that way, that way, and some more bees go that way instead of this way." - Right. So now, here comes one of the beautiful parts of it. - And this is where it gets really interesting. - If there are a lot of bees going to this five minute patch, eventually, Craig says, - There will be more and more depleted flowers. - This patch, it starts to run out of neck. - Hey, go that way. - Go that way. - That means that a honey bee, she's going to take longer to fill her stomach. - Hey, go that way. - Right. - So a five minute patch, if it's crowded, is no longer a five minute patch. - Go that way. - Right. - It might become a seven minute patch. - The patch gets more and more picked to over. - Now, it's an eight minute patch. - Hey, go that way. - And then. - A ten minute patch. - And. - Hey. - At that point. - Go this way. - It's taking the bees the same amount of time. - Go that way. - To go to the close-by patch, as it is to go to the one that's further away. - Go this way. - And because of the dancing, the hive is sort of evening out. The number of bees it's sending to each patch. - Go that way. - Go this way. - Go that way. - Go this way. - Then you're in equilibrium. - Even though the bees don't have stopwatches, they equalize the round trip time. - The hive has taken into account distance and crowdedness and figured out the way to get the most nectar in the least amount of time. And in the real world. - The bees may very well be going to five different patches. - Or it does it. - Not just two. - And dealing with actual nature. - Which means taking into account different types of flowers and weather and predators, but no matter how many other variables the bees have to deal with. - The allocation of bees amongst the flower patches. - Is this astonishingly efficient? It looks like there's an air traffic controller or like the hive is thinking. - Yeah, that's right. I think though when we use the words like thinking, we're thinking in human terms. But if you say thinking is just a matter of taking in information, processing it to make decisions, then I think that definition of thinking applies. - And the way that they were processing, this incredibly complex set of information, was by following one simple rule. - If one flower patch has a smaller round trip time than the others. - Send more bees there. Like whichever patch bees are coming back most quickly from. - That's where you send more bees. - And that's it. - It's gorgeous isn't it? A friend of mine once said it's very zen. - But it's also weirdly, it's very bottom line. Like it's like, I don't care where you're going. I don't care what you're, like it's like, what really matters is when you show up with the goods. Like show up with the goods and then we'll talk. - And that will negotiate. - So this system, Craig and Tom observed at cranberry lake. Craig wrote it up as math. He called it. - The honey bee algorithm. - What does it look like on paper? - Equations I went when I showed a fourth grader. F sub n of x sub n of x sub n divided by x sub n. is equal to F sub M. - And when Craig tried to use this algorithm on his car building robot problem-- - It didn't apply. - It was completely unhelpful. - It was such a different problem. - Which is why it was so exciting to him when Sunil walked into his office more than a decade later with this other problem, the internet problem, which to Craig anyway looked basically the same as the one the bees were facing. - And so after 15 minutes I said to Sunil, let's imitate the bees. - How could imitating the bees help Sunil stop the internet from breaking? - I am desperate to finally understand this, but first we need to take a quick break. - All right, we'll be right back. - Boo-do-do-do-do-do-do-ha. Lulu. - What the? - Radio lab. - Back with bees. - Bees. And you were going to tell us how they're like in our internet or something? - Yeah, basically, pretty much. But to do that I needed an example. So I found something in my own life, which I apologize in advance for how annoying this is going to be. Do you remember the hamster dance? No, I don't think I do. Is this like a website? - Yeah, yeah, yeah, yeah. Okay, I'm kind of surprised you don't remember it. It was a very early web kind of thing. It was, but all it was, it was just a song with a web page and the web page was just a bunch of gifts of cartoon hamsters dancing. It's the most annoying song that you will ever. - Can you still sing it? - I can. Would you like to hear it? - Yes, please. It was, um, D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D- - Right, and so what was happening behind the scenes was, well, okay, so like basic internet 101. - Yeah. - Every website on the internet, like for example, what was at the time, hamsterdance.com, exists geographically somewhere. - Like in the cloud or? - No, no, like a single earthbound computer, the owner of the website paid for it to live there. And that computer, which is called a server, is not in a house or in an office, it is in a special building dedicated to servers, the server farm, right, or a data center. - Okay, so it's at a data center in like Ohio. - Yeah, wherever it is. - Okay. - These server farms are all over the world and connected together, they more or less make up the internet. - Oh. - And so. - When the request comes in. - When, you know, young me types in hamsterdance.com and presses enter, that server. - The actual infrastructure that's holding it, who is just sitting around in Utah or whatever on the server farm, gets a little notification. - Well, someone wants to see that hamsterdance.com website. All right, not too busy with anything else. - So it does a little computation. - Go ahead and get that for you. - Some execution of some Java code or something. - Let me see here, I think it's around here somewhere. - And then push that content back to UI internet. - Here you go. - Where it pops up on my family's desktop computer back in 1999 and then, and enjoy it. (laughing) Then the server just goes back to hanging out on the server farm. But then, oh, another person wants to see that website. All right, let me just go ahead and get that for you. There you go. All right, where was that? Man, I just love this control. Oh. Oh, no. And as it starts to go viral. - Number of people are all combating with the request. - 'Til that one server is like, oh my God. (screaming) What's somebody help? - Can no longer serve up this website in a timely manner. - And so now, people have to wait in a queue, right? - And this would happen all the time because back then, the way servers were allocated was like, you'd get how many servers you paid for. - Right. - Like the owner of the hamsterdance website, probably would have been like, hey, I'll just pay for the one server. - Because I don't expect my website to experience a lot of demand. And how wrong they were, right? - Do you remember what that was like though? - Yes. - Like how slow was it? - I do look like slow loading and like, ah. - If something took longer than five seconds, generally the human psychology was that people give up on the website. - So as soon as you saw it, the problem was relatively simple. There are a bunch of servers. - And sometimes there can be not very busy. - Sometimes they're just sitting around doing nothing, but other times there could be overly busy. - And so how in a system that is changing so quickly, do you get those servers who are doing nothing to help those servers who are doing too much? - Start moving these servers around where they need it. - And this is the problem that Sunil brought to Craig. - Yeah, hi, I'm Sunil and I need to be back within 20 seconds. - That was part of the Gordon. - I saw that the problem was similar to the honey bee problem. - Like a meheli? Like that fast? - Yeah, 'cause to Craig it was like, he'd been holding on to this rusty old key that he was holding for more than a decade. And Sunil showed up with something that looked like it might be the exactly matching lock. - Yeah, yeah. - And like evolution has solved this problem in some way, right? And now you're saying, okay, can you do it in the artificial domain? And so, they got to work. - Right, F sub N of X sub N. - Just it off that old math equation. - Mapping the two structure of the, you know, what the bees are doing. - And sitting there, they started to connect all of these dots. - Like collection of bees make up the beehive, collection of servos make up the internet servo farm. - Yes, exactly. - And wait, so that would mean, oh my God, that stuff on the internet are the power patches. - Oh, bingo. - At that point, they've got really exciting. - Wait, I'm sorry, just, please stop that. - Sorry. - How is what the bees are doing, anything like what the server's doing? Like, lay out the parallel. - Okay, why don't we, okay, why don't we cranberry lake this thing? Okay? - So, instead of a meadow filled with wildflowers, - Yeah. - Okay, picture the entire internet. - Beep, beep, beep, beep, beep, beep, beep, beep. Got it. - It's the beginning of the day. You're just signing on to your computer. And the first thing you decide to do on there. - Ha ha, Charlie, Charlie bit me. - Is watch a video of a baby biting his older brother's finger, naturally, right? - Oh, a little bit, yes. Okay, how I love to stop my day. - Stop making me think of course. Okay, so you watching this video is like a flower opening in the meadow, right? The meadow of the internet. - Why is me watching analogous to a flower opening? - You watching is like the flower filling up with nectar. It's your desire, which they can capitalize on, basically. - Oh, me watching is money for, it's, oh, oh. - It's money, it's money for them. - My eyes on the video. - Your eyeballs and your attention, that's like the server company's nectar. - Oh, okay, I get it, I get it. - We are the flowers. - Yeah, and as more and more people watch and then share this video with their friends who watch it too and share it with their friends who watch it too, that's like more and more flowers opening in the flower patch. And the server hosting that video is like a single bee being like, holy motherland. Bananas of nectar over here, I'm gonna need some backup. - Oh, so now it has to recruit more bees, AKA more servers. - Yeah, so it does like a computer version of a waggle dance, this server to server digital nudge. Okay, let's call it a ping. That means, hey, I need some help. Come get some of this stuff. - Okay. - And so servers buzz over to the Charlie Bittmi video and start servicing all of these flowers, right? Helping all these people to watch this video. And then it kinda keeps going, right? The video popularity grows, flowers keep opening, more bees needed, more servers needed, and very quickly, all these servers are just servicing the Charlie Bittmi video. - Okay. - But then, wait a second, now there's a ping, but it's not for the Charlie Bittmi video. - A giant container ship wedged from bank to bank. - Now everybody wants to see pictures of this huge boat on the BBC website. - Looking one of the world's most important shipping lanes. - Did just got stuck in the Suez Canal. - I remember that, I was a flower. - Right, right, so like you, all these flowers opening up all over the world, all the sudden, the servers that are serving up these boat pictures, now they're the ones who are saying, hey, we need backup. And again, there is this server to server, peer to peer, B2B, waggle dance recruitment, ping, calling out to the idle servers to come help. And those servers come, and they ping out. And those servers who are no longer needed with the Charlie Bittmi thing are now like, oh man, I gotta go help with this Suez Canal thing. - Okay, got it. But then in the actual internet, this is happening times like a gazillion, right? All the time, literally billions of people, billions of flowers, so could this algorithm this little fix actually work? - Well, they had to figure that out. - You know, initially excitement kind of sets in, but then you go to prove what you thought would be a good solution, right? - To test this idea, Sunil and Craig decided to compare it to another algorithm they invented. - The algorithm we named it omniscient. - Okay. - If it could see the future, what would it do? - Like if you knew ahead of time, everything that everybody would wanna see on the internet, how would you organize your servers to meet that demand? - It's basically God mode. - This is the best you can do. You can't do better than that. - So they compared this perfect model to the way humans had been allocating servers already. - Like the guessing. - Right. And they compared it as well to the B's algorithm version. And what they found was that the B's, even without knowing the future, they were coming within like 2015% of the optimal behavior. - Wow. - In a bunch of their tests, the human algorithms didn't even come close. - Whoa. - Yeah, turns out that-- - At least theoretically. - It worked really well. - So based off these results, they publish a little paper. And then Sunil goes back to Oxford to defend his thesis. - I presented the results to the committee. - He tells them about the problem he's seen with the internet and how the B's could help solve it, talking for like-- - 45 minutes to an hour. - And when he's finally done, the first question that he gets is, have you patented this idea or not? - Oh, wow. That's a good question to get. - But he was like, well, I just published it. - You can't patented your own thing if you published. - Oh, no! - And so he did not patented. - Oh, he gave it away. - He gave it away for free to everybody. And over the following months and years, server farms all over the world worked this B algorithm into the internet. - Wow. - You know, the honey B algorithm made it 10-20% more efficient. - Which means we have the B's to think for the internet being this place where-- - US climber, Alex Huddled, stunned. - You can get whatever you want. - Video from outside the capital shows the beginning of the storm. - Right now. - The hurricane pushed Lake Puncher train D. - It's been a bunch of gelatin! - I mean, that is just a moment for nature that that could outperform so many thousands of human brains working on solving all kinds of problems. And just like evolution had figured this out through its own longer timeline of trial and error. - Yeah, and not only that, this honey B algorithm or variations of it have been lifted into so many other industries. I have found people researching how to use it to forecast exchange rates, design electric cars, detect defects in wood before using that wood for construction, even for sharpening MRI images to better detect breast cancer tumors. - Bless that, little bumblebee. I mean, I guess what's so, I think about humans. One of our gifts and our curses is that we can jump to the future, you know, or the past. We can jump out of the present. But we can worry. And that's like entire industry's prediction forecasting, whether it's weather or money or, you know, whatever. Like, but we spend time worrying about the future. But it feels like what this, the elegance, it's like not only to me is it beautiful that it came from bees because I'm a nature lover and I love it when nature outsmarts us or has more elegance in its design. But it's also like one of the insights, if you wipe away all the technicalities, is like, it's throwing away the future. It's not basing, it's decision of where to move based on a guess about the future. It is only responding to the present. - Hmm. Right, right. It's like, it's kind of at this very subtle, like it's like right at the line between responding to the clues of the moment and predicting the future. - It's on the edge of the present. - It's at the edge of the present. And it's like, all we got to do is like, pay attention to what's happening now and then, and like build it into these feedback loops so that we can, yeah, so that we can address just the next moment, like just the next moment and just the next moment after that and just the next moment after that and just the next moment after that. I mean, that's, if anything, that might be a thing I like. Take a round with me. (gentle music) - This episode was reported by me, Lative Nasser, with reporting help from Maria Paz Gutierrez, production by Maria Paz Gutierrez, Annie McHughan, and Pat Walters, edited by Pat Walters and facts checked by Diane Kelly. - We also got a lot of help for this episode from Radio Lab and Terrestrails resident bug correspondent, Sammy Ramsey, and we couldn't have done this one without his help, big thank you to you, Sammy. And if you wanna hear more about these and hear Sammy talking about 'em, we have a Terrestrails episode called the Crystal Ball, Honeybees, who predict the future. You can go listen to him over there. - Other major thank yous to John Bartholdi, John Vandivate, and James Marshall. As well, we wanna thank the folks at Triple AS who administer the one and only Golden Goose Award. If you remember, the award goes to government-funded science that sounds kinda silly or bizarre, but then goes on to change the world. This research won a Golden Goose Award back in 2016, which is how I first heard about it. So thank you to all our friends there, Aaron Heath, Gwendolyn Bogard, Valeria Sabate, Joanne Padrone-Carnie, and Meredith Asbury. - And Bees, we barely scratched the surface of how amazing they are. If you wanna learn more, read any of Tom Ceeley's books. His most recent one is the incredibly titled memoir, Piping Hot Bees and Boys Just Buzz Runners. - I think that should be it, but if you are still here, let me leave you with this bit of tape that has been haunting me. - Danielle, like, dude, over the last 20 years, I've used the internet a lot. If the internet today took the same amount of time as the internet in the '90s, I can easily imagine not just minutes, like, I mean, hours full days of my life, like a second at a time that this could have saved. - Yeah, you're right. But also, what most internet service company wants is stickability for you to keep using the service, right? - Okay. So you're saying the opposite. You have made the internet so enjoyable that you have cost me days and hours, potentially even years of my life. So I should be mad at you. - Yeah. - See, that thing, dude, don't say, but it ain't no. - And that's it. See you next week. - Hi, I'm Gabby. I'm from San Francisco, and here are the staff credits. Radiolab is hosted by Lula Miller and Lethifnossar. Soran Wheeler is our executive editor. Sarah Sandbach is our executive director. Our managing editor is Pat Walters. Dylan Keefe is our director of sound design. Our staff includes Jeremy Bloom, W. Harry Fortuna, David Gabel, Maria Paz Gutierrez, Sindu Nainasambandan, Matt Kielte, Mona McGawker, Annie McEwan, Alex Nason, Sarah Kari, Rebecca Rand, Anise Vista, Ariane Wack, Mollie Webster, and Jessica Young, with help from Gabby Santas. Our fact checkers are Diane Kelly, Emily Krieger, Natalie Middleton, Angelie Mercado and Sophie Semi. Leadership support for Radiolab's science programming is provided by the Simon's Foundation and the John Templeton Foundation. Foundational support for Radiolab was provided by the Alfred P. Sloan Foundation.

Podcast Summary

Key Points:

  1. Sunil Makrani, a computer engineer, observed the internet breaking during the 9/11 attacks due to overwhelming demand, sparking his interest in solving infrastructure inefficiencies.
  2. Inspired by honeybee behavior, Sunil and systems engineer Craig Tovy developed a decentralized algorithm that mimics how bees distribute themselves across flower patches to optimize resource gathering.
  3. The honeybee algorithm, based on a simple rule—send more bees to the patch with the shortest round-trip time—effectively balances load without needing a central controller.
  4. This algorithm was applied to internet server allocation, where "flower patches" represent popular websites and "bees" represent servers responding to user demand.
  5. When tested, the honeybee algorithm outperformed human-designed and omniscient (future-knowledge) models, achieving near-optimal efficiency in load balancing.
  6. Sunil shared the algorithm publicly without patenting it, leading to global adoption in server farms and improving internet performance by 10–20%.
  7. The algorithm demonstrates that optimal solutions can emerge from decentralized, present-focused feedback systems rather than predictions about the future.
  8. The approach has since been applied in fields like finance, healthcare, and AI, proving the value of bio-inspired, adaptive systems in complex problem-solving.

Summary:

Sunil Makrani, a computer engineer, noticed the internet crashing during the 9/11 attacks due to a sudden surge in demand, prompting him to explore how systems could better handle such fluctuations. Inspired by the efficient, decentralized behavior of honeybees, he collaborated with systems engineer Craig Tovy to develop a model inspired by bee foraging. Bees allocate themselves to flower patches based on the shortest round-trip time, balancing efficiency without a central leader.

Sunil and Craig applied this "honeybee algorithm" to internet server distribution, where websites act like flower patches and servers like bees. When a video or event goes viral, servers "dance" via digital pings to recruit idle ones, dynamically sharing the load. Tests showed this algorithm achieved up to 98% efficiency—far surpassing human-designed or predictive methods—proving that decentralized, present-aware systems outperform centralized, future-guessing ones.

The solution was shared openly, leading to real-world adoption in data centers worldwide, improving responsiveness and reducing latency. Beyond the internet, the algorithm has been used in forecasting, electric vehicle design, and medical imaging. The story highlights nature’s elegant problem-solving, showing that sometimes the best solutions emerge not from human foresight, but from simple, adaptive rules that respond to the present moment.

This bio-inspired approach demonstrates how evolution has solved complex coordination problems, and how such insights can revolutionize technology.

FAQs

Sunil noticed that during major events like the 9/11 attacks, websites crashed or became extremely slow due to overwhelming demand, making it impossible for people to access information.

He studied how systems could manage demand efficiently and eventually found inspiration in honeybee behavior, particularly their ability to distribute resources dynamically based on real-time conditions.

The honeybee algorithm is a system where bees move more to flower patches that require less time to reach, effectively balancing load based on distance and crowding, without centralized control.

They discovered that bees naturally balance resource allocation by responding to real-time conditions, leading to a model that could be applied to distribute server loads efficiently across the internet.

The algorithm helps server farms dynamically allocate resources by sending 'pings' to idle servers, allowing them to assist during peak demand, improving efficiency and reducing downtime.

It achieved up to 98% efficiency by responding to real-time demand, while traditional methods based on fixed server counts failed to adapt, leading to crashes and long wait times.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.