Go back

Why everything is broken

28m 4s

Why everything is broken

On Friday the 13th, a flawed industrial router upgrade in Arkansas disrupted the U.S. aviation system, grounding flights nationwide due to data overload in an aging, patchwork air traffic control network. This failure highlights systemic issues: decades of underinvestment, outdated infrastructure, and political delays in critical upgrades. Simultaneously, Microsoft Outlook suffered a massive, prolonged outage caused by a faulty internal update that spread through its cloud infrastructure, showing how software patching risks can cascade across modern tech systems. Both incidents underscore a broader pattern of technological fragility, rooted in poor maintenance, reactive updates, and insufficient investment. The U.S. compares poorly to countries like Canada and Europe, which use public fees and less aviation demand to sustain infrastructure. Meanwhile, cybersecurity threats—including an Iranian cyberattack on supermarket fridges and a $6 billion crypto heist—demonstrate that vulnerabilities are not new but are now amplified by AI tools that accelerate both discovery and exploitation. Experts warn that while such outages are cyclical, the increasing complexity of AI and digital systems, combined with political instability and inadequate funding, makes the current crisis more severe than ever. A long-term, bipartisan solution—funded by sustained investment in both aviation and digital infrastructure—is urgently needed to prevent future collapses.

Transcription

4438 Words, 25181 Characters

English
"It was Friday the 13th. Microsoft Outlook was still struggling, but we had a new problem. The government ordered a complete ground stop of flights. The president was nowhere to be seen, but he sent his Homeland Security Secretary Mark Wayne Mullen out to ease the fears of the nation. "The last time I believe that we've had a national ground stoppage was 9/11." "It's actually the president posted about a golf course renovation project in DC about 13 times before he said anything about the ground stoppage affecting millions of Americans this weekend. Again, not super helpful, but we did eventually find the culprit. An industrial router that was being upgraded, which leads one to ask, "What is going on in this country that a single industrial router could take out our entire aviation system?" And everything else that's broken coming up on today's "Splain From Blocks." I'm Aina Garten, and each week on Happy Hour, I invite new and old friends for a drink and a fun conversation at my kitchen table in New York City. And this week's episode, I'm joined by Rose Byrne. We talk all about her love of mixology, doing a stoop sale in Brooklyn, and her dislike of marshmallows on sweet potatoes. New episodes of Happy Hour are available every Wednesday. You can watch on YouTube or listen wherever you get your podcasts. Cheers! Support for today's episode comes from I Am 8. I Am 8 is the way to simplify your supplement routine once and for all. I Am 8's daily ultimate essentials has 16 supplements and one drink for $2.61 a day. That covers 90 ingredients across nine major organ systems. Go to I Am 8 Health.com/explained right now, or click on the link in the description and use code explained for free Welcome Kit, 5 free travel sachets, plus 10% off your order. That's code explained at I Am 8 Health.com/explained. Code explained at I Am 8 Health.com/explained. These statements have not been evaluated by the Food and Drug Administration. This product is not intended to diagnose, treat, cure or prevent any disease. What a day, right? Today, I explain I'm Sean Ramos from here with Daryl Campbell who writes about aviation at the verge. So let me take you to Friday the 13th. And there's a contractor called L3 Harris and they're doing what they hope is routine maintenance on a dinky little part of the aviation infrastructure backbone for data in the middle nowhere Arkansas. So because they think it's this little tiny place, they do it during the day on a Friday, which is usually not when you want to be messing with this massive data infrastructure because you know, what if something happens. But you know, they think, well, we're going to do it anyway. So they plug in this router for the first time. They're basically pulling out an old telephone switch that is using copper cables. And they're installing one that has these new fiber optics. It's super fast and all this good stuff, except they leave a factory setting on. And the factory setting starts paying the network hundreds of times every second. Now if you're on a normal kind of massive new infrastructure, that's not a problem. So the thing is the FAA's data backbone is this kind of patchwork system of old school technology from quite literally designed in the 50s and 60s to some replacements in the 80s and 90s. So kind of like if you remember cable modems in DSL from the late 90s, early 2000s to some of these brand new fiber optic systems and they all have to play nice together. And so that means you can't force the old parts of the system to work at the same part of the new one. But the problem is this factory setting assumes that you're plugging it into a brand new system. And it quickly starts overwhelming the data flow with all of these requests and kind of causing packet loss in the same sense of like what you would get if you're on a FaceTime call with someone and your Wi-Fi signal isn't very good. So you start losing bits, the audio gets a little bit choppy. Now the difference here again because it's an industrial scale thing is that FaceTime call doesn't really matter if you start to lose data. But if you're tracking hundreds of flights and you need up to the second audio feeds that are perfectly crystal clear with absolutely perfect data, once you start to lose some of that then you can actually cause a plane crash and so that's something that you really really don't want to happen. Now Sean Duffy did partly blame the Biden administration as did President Trump. However, they also had the decency at least Duffy to say this goes back a very long time. Many administrations. And I think on some level there he's right that these are like very old problems. Why is it so hard to fix these problems that it's going to take decades and generations of suffering? Yeah, I mean the weirdest part about it all is even though everybody flies it's you know a critical system. The timeline for doing these kinds of upgrades is really out of sync with the political timeline. So you think you have a major party change, a presidential change every four and eight years. But if you're going to do a large scale system upgrade, I mean you literally have to be digging hundreds of miles of cables, ripping out systems, replacing them without causing the whole system to go down for months on end. So it really is a complex logistical ballet. And that's why it takes 10, 15, 20 years to do these kinds of step change upgrades. And then if you've got these political parties pulling funding one year and then reinstating it another. So I'm calling on Congress as soon as they come back to pass a clean extension of the surface transportation bill along with a clean extension of the FAA bill to give workers and communities across America the confidence that vital construction projects won't come to a halt. Or fighting over it because the government shut down and causing these delays. We're not going to be able to address it overnight but we now have 2,600 air traffic controllers in training. A government shutdown would stop that training. Even a shutdown lasting a few weeks could set us back by months or more. They can easily turn you know a short term project, short term in the sense of you know 3 to 5 years into something that takes 10 or 20 or just never happens. Duffy says he can't guarantee this won't happen again. You will have more of these. You will have more of them. And last, the American people in the Congress tell us to build. Does that mean it's going to happen again because it's pretty bad on Friday? I mean, it was. And if I were Sean Duffy, I would definitely be saying this is probably going to happen again because honestly it is, ironically we've gotten so good at kind of kicking the can down the road and making the best out of these systems for so long that we're now to the point where all of these band aid fixes are just falling apart all at once. And so if we had done a little bit more to cause failures 5 to 10 years ago, then we probably would be in a better place today in a weird sense. But yeah, I mean, it's pretty clear that we've seen the pace of these aviation infrastructure and equipment crises kind of accelerating over the last 3 to 5 years. And it's really because there's nothing else that we can do to stop it. You just have to take the pain at some point and we could have done it earlier with better political will. But now the equipment itself is telling us just can't go on anymore. Daryl, we famously aren't the only country with planes and certainly we can't be the only country with aging infrastructure. How are the other countries fixing their aging infrastructure without having to ground all their planes on a Friday night? Yeah, well, there's a couple of ways. Europe probably has a very similar system in terms of the age of its components, but they do a lot of their, what would be the equivalent of domestic travel, so short haul trips through trains. So the load on the aviation system is substantially less. US airspace is the most crowded in the entire world. And it's because we really just rely on flights to get from point A to point B like outside of the Northeast. There's really no alternative if you want to go more than 200 miles and you don't want to spend a whole day driving. Obviously, if you were redoing this system today, like for example, is happening in China, you would just put in this fancy fiber optic stuff. Now, obviously it's a lot easier when you've got, you know, a single political system that you can, you know, rip up entire 100 mile runs of cable without having to go through all of these permitting process and use M and it domain, et cetera, et cetera. But the main problem is still there. Europe famously across the pond, but what about there I ask Canada? Well, Canada has about eight to 10% of the volume of the United States in terms of air. In fact, Canadian airspace is partially controlled by American air traffic controllers. And the other thing too is that Canada actually requires people to pay fees for the infrastructure. And then the US, we've decided that aviation is essentially a right for people. And so the pressure is to push down costs as much as possible. So in Canada, you're not gonna get, you know, your $70 air fares to. Um, wherever, you know, airfares are a lot more expensive, partially because they're putting some of that money back into the system. So are you saying we could have fixed this with fees? Well, I mean, partially, yeah, I think the, the big infusion of cash required to do these big infrastructural projects would have taken acts of Congress, but like some of the day-to-day maintenance would certainly have been a lot easier if you have, um, a lot of fees just coming in on a regular basis. So famously, um, some radar systems in the United States are held together with duct tape and aluminum foil. Did you just say that our radar systems are held together with duct tape and aluminum foil? Oh, yeah. I again wanted to interrupt you, but I didn't because I was trying to be polite. I'll text you a picture afterwards, but I really do think that the big stuff, like this kind of upgrade, um, is only going to happen with this kind of political congressional funding, these infusion of billions and billions of dollars that you're really never going to get through just simple fee structures. Does the fact that it went so bad on Friday and that people are so mad and it looks so awful for our country, does that help with the politics of all of this? I hope so. And I do think that Sean Duffy is, is smartly deciding to use this as a way to get at least some of his, um, upgrade budget and agenda over the line. I want Congress to give us emergency funding. This is bipartisan, this is American. I want $50 billion, President Trump is a builder. He's going to build this network, um, and deliver for the American people, but the President can't deliver for America unless the Congress delivers the money to make it happen. Regardless of political persuasion, we really do need tens of billions of dollars of investment in our air infrastructure in order to, you know, make things, uh, not collapse spectacularly before. Um, but I really think that, you know, it's, it's a systemic problem that if it doesn't, if it gets solved this year for a year or two, that would be great. But, um, you know, people just tend to forget about these things, especially if you're going into a season where you're going to have winter storms. It's just so complex and so hard for an individual person to understand all the parts that happen. Um, that, you know, we only have a little bit of time to really capitalize on these kinds of things. And like I said, it takes years and years of sustained attention to really solve some of these problems. Okay. But do we think they'll stop doing updates on like Friday nights or like, I don't know, the Wednesday before Thanksgiving or like, I don't know, the week before Christmas? Yeah. I think that was easily avoidable. I mean, it was just a stupid move to do that kind of thing on a Friday, um, like do it like Sunday night. There's nobody flying, but Friday afternoon, it was a bad idea to begin with and then just a lot of bad decisions cascaded on top of that. Okay, that covers the planes, but when today explained his back, we're going to ask about all the other outages, outlook, the banks, finance, everything. Support for Jake, explain comes from Vanta as more companies embrace AI, managing security and compliance is only getting more complicated. New requirements, more audits, more vendors to keep track of and chances are your team isn't growing nearly as fast as you need. Plenty of compliance tools promise automation, but then you still find yourself breed in manual work, tracking down screenshots, validating controls, scrambling to get everything ready for the next audit. Vanta takes a different approach. It's a genetic trust platform is designed to grow with your business, not hold it back with more than 1400 automated tests against 400 plus integrations. Vanta continually collects evidence and monitors your controls throughout the year, helping reduce audit prep time by 82%. And the Vanta agent is available wherever you work, whether your team spends its days in Vanta or uses tools such as cloud or cursor. It's why more than 16,000 companies I'm told, including ramp, writer and Harvey, trust Vanta to save time, free up resources and keep security moving as their businesses grow with Vanta. Learn more at vanta.com/explained, spelled V-A-N-T-A dot com slash explained. Supported today's plane comes from vanguard to all those financial advisors listening today. Let's talk about bonds for a minute. Finding value in fixed income isn't always easy. Vanta markets are enormous, complex, constantly changing, keeping up with all of it is a lot for any team. And that is where vanguard can help vanguard has decades of experience navigating fixed income markets and their scale gives them a distinct advantage with the ability to invest across sectors, maturities and geographies. They can identify and pursue opportunities that might otherwise be overlooked. Some investment managers chase higher returns by taking bigger risks, but that can also mean volatility for your investors. Whether it takes a more measured approach, rather than relying on big concentrated bets they focus on discipline investing, thoughtful risk management and consistent execution, might not be the flashiest strategy, but when it comes to helping clients navigate the bond market, dependability and consistency can go a long way. So if you're looking to give your clients consistent results year in and year out, you can go see the record for yourself at vanguard dot com slash audio. That's vanguard dot com slash audio. Global investing is subject to risk vanguard marketing corporation distributor. Support for the show today comes from granola, but hold the cranberries because we're talking granola AI. Understanding something takes more than simply writing down the facts. It's about absorbing information and creating that light bulb moment. The same is true at work. Your best ideas don't usually come when your nose deep and note taking they come from talking through problems that loud and spit ball with colleagues. Kick your spit to yourself, colleague, granola makes that possible. granola is an AI powered notepad that makes your meeting notes useful. It captures what happens in your meetings and turns it into clean structured notes with decisions and action items pulled out and made easy to find. They say once you use granola in your first meeting, you'll never want to take manual notes. Again, you can try it totally free head to granola dot AI slash explained. That is granola dot AI slash explain to get your time back. Go to granola dot AI slash explain and try it on your next call. See what happens screen granola from the rooftops enjoy. This is today explained. I'm Lily Hey Newman and I'm a senior writer at Wired. Lily we spent the first half the show talking about the issues with our air traffic control systems, but last week of course also brought chaos to a lot of people's inboxes when outlook was not so good. Tell us what happened there. Yeah, Microsoft 365. So all the productivity platforms had a massive outage and it really dragged on for like a full day, which is very noticeable when it's a set of products that are as integrated in everything as 365 is. That means you just like can't work, right? This was like probably billions of dollars of productivity lost. Yeah, this is the cloud version of Microsoft office. So you know, outlook, word, but also excel, but also new sort of generations like co-pilot, that AI integration and teams. If that's your productivity platform, then yeah, you can't work. That's everything. Outlook and teams is down. This is our snow day as an adult. You're so sad Microsoft doesn't working, right? No, they're sad we want to have a devastated and devastated. Microsoft rate is so enmeshed in the tech ecosystem for decades, like from even before all the cloud platforms, because they're the Windows maker, right? So they're in everything. Now we heard earlier in the show that our planes were grounded across the country on Friday because of like essentially a faulty router. What happened with outlook? Yeah, so Microsoft, you know, a few days later on Monday, what they were saying was one was that they had sent out an update. It's a Windows update, but it's only really for their infrastructure. It's not an update in consumer Windows. So it's not something like your laptop would have gotten or something, but it has been out and proliferating through their infrastructure for about three months, but it actually had a flaw in it that wasn't caught by their testing. And then at the same time, they say cloud infrastructure is all about like managing traffic and managing different regions of data centers, the issue with that, you know, three months of proliferation of that update started triggering, and that's why it was such a sort of intractable issue or so difficult for them to get, because at first I saw it, I thought, okay, it'll be a few hours or something and they'll fix it and then waking up Friday morning, everybody's waking up in their mornings and it's still not fixed. And it's like, whoa, so something is really going on, you know? And so that it was the interplay of multiple factors that made it so hard for them to recover, they say. Okay, very different situations here between Air Traffic Control and Microsoft 365. However, both kind of have a similar origin story in that they're about patching systems and upgrading systems. True, yes. We've heard from Sean Duffy, Transportation Secretary, that he cannot guarantee what happened on Friday the 13th. Won't happen again. In the near distant future. Very ominous. Have we heard something more reassuring from Microsoft? Well, Microsoft would definitely say, you know, this isn't going to happen again. We want to give you all assurances, we are so sorry. Sometimes problems arise because of sort of cyber attacks or exploitation of vulnerabilities. And then it's kind of like, well, why weren't all those systems patched? Like there was a patch out. But a lot of times patching is slower than we would like because the ramifications of a bad patch or like an unforeseen consequence of an update can be really severe. And so organizations tend to be very nervous. But it's never the last time there's all, you know, these types of outages and failures happen in all technology systems. And they will continue to, but you know, people try to learn as much as they can from each incident and like make things more and more robust. Maybe a good time to ask you about all of the other breakdowns we've been having in our technology and elsewhere over the last few weeks. We have an interstitial that's like meanwhile. If you were wondering why a third of the internet is burning right now, this is why I'm about to read you the list of 19 incidents that have occurred between October 19th and 22nd. What the hell is going on with black October and why are there agents inside of bank accounts? I mean, people may not have even realized they just lived through something called black October. What is black October beyond like, you know, Halloween? I think in October, there were like a few things going on including the widely used web scripting language or web programming language PHP. And there were these two vulnerabilities in it that researchers were really concerned that, you know, if they were exploited, they could cause like remote access, takeovers of all different websites and all different platforms. And so there was this big industry wide effort to make sure that everybody was patching, speaking of patching. Like we got to patch this right away, don't wait type of thing. So there's black October and then there's the phone apocalypse people have been talking about this for maybe about a year now, just kind of this idea. Like everything's vulnerable. Right for that AI tools. We need a better day for that. Look, is that my thing? I'm just doing air quotes here, you know, heavy, heavy air quotes. But, you know, this idea that agent AI and AI models are becoming more and more powerful in a depth at finding bugs and software and in helping both researchers and potentially attackers figure out how to exploit those vulnerabilities and essentially weaponize them. So there's this sort of fear that, you know, cybersecurity is always this cat and mouse people call it. Again, I'm doing the air quotes cat and mouse game or like an arms race is the other, you know, another phrase that's often used. And so that in itself is not new, but the idea that AI in the mix just really accelerates for both sides and how to manage that dynamic. Okay, so that was one with a quasi happy ending, but there were a couple that weren't so happy, like this Iranian cyber attack on supermarket fridges or finance getting hacked for like, I don't know, I think like the most money I've ever heard about in any kind of hack or highest ever, like $6 billion. Yeah. And crypto, I thought we were meant to believe that crypto was like unhackable in here. Oh, man. I mean, that's like hacked for six billion. Okay. And I just want everyone to know, like you can rest easy things were already really bad before like great. So, you know, the fridge hacks, for example, that the vulnerabilities that are being exploited there are actually kind of go back like numerous years. Yeah, French officials like publicly formally in a whole document have pointed to Iranian actors as having been behind at least some of those this is targeting the food supply chain and same with the finance situation. Like North Korean hackers stealing cryptocurrency, including like large sums of cryptocurrency is in narrative that reporters have been following and you know researchers everybody that has been following for, you know, years and years and years. And then on Monday, we had this bank outage across banks across the United States Bank of America chase Wells Fargo. It just feels like everything's kind of breaking at a time where we're on like the precipice of yeah driverless cars and like AI running everything like is it getting worse or is it just a rough few months here. To me this stuff feels more cyclical like people may remember like eight to ten years ago when there were like a ton of DDoS attacks taking down, you know, huge services that was even like and basically an East Coast internet outage. DDoS attack the disrupted internet was largest of its kind in history experts say users reported sporadic problems reaching several websites including Twitter, Netflix, Spotify, Airbnb, Reddit, Etsy, SoundCloud. So for me, it feels like we're in another round of those I like one of those moments where a lot of stuff starts to pile up and incident responders are really overworked and really, you know, exhausted and then like other things are happening and just like nobody has as much sort of elasticity or bandwidth because there's so many things piling up but once. Especially right now, you know, in this new era of sort of ubiquitous AI and access, you know, more and more access to more and more capable AI systems. Again, it's not to diminish that feeling. It's just maybe helpful to have some context that it's not like truly truly unprecedented right now. Lily, hey, Newman, wired.com earlier, it was Daryl Campbell, author of Fatal Abstraction, why the managerial class loses control of software seems relevant. While we were making the show, CNN reported that a highly classified US national security communications network is down. Good thing the president isn't hanging out with like the leaders of China and Russia this week. What's that? Oh, no. Ariana Spru, produced with Denise Gareff, feel better Denise, David Tatashore, mixed with Bridger Dunn again, who feels fine Gabriel Donatov, fact checked with Kelly Westinger, who is well-rested, and Miranda Kennedy edited for today explained. [BLANK_AUDIO]

Podcast Summary

Key Points:

  1. A faulty industrial router upgrade in Arkansas, left with factory settings, caused a nationwide aviation ground stop by overwhelming the FAA’s aging, patchwork data backbone.
  2. The aviation system’s vulnerability stems from decades of outdated infrastructure and inconsistent upgrades, with political and logistical delays prolonging necessary modernization.
  3. Similar systemic failures occurred with Microsoft Outlook, where a flawed internal update spread through infrastructure and caused a widespread, prolonged productivity outage.
  4. Both aviation and tech outages highlight the risk of poorly tested system patches, especially when applied during high-traffic times like Friday the 13th.
  5. Global comparisons show that countries like Canada and Europe manage infrastructure better through funding models, public fees, and lower aviation demand.
  6. The U.S. faces a systemic crisis due to political fragmentation, lack of sustained investment, and underfunded maintenance, despite clear warnings from experts.
  7. AI-powered tools and cybersecurity threats are now accelerating attacks and discovery of vulnerabilities, creating a new arms race in digital security.
  8. Experts agree that while outages are not unprecedented, the current wave reflects growing complexity and strain on infrastructure, demanding urgent, bipartisan investment.

Summary:

S. aviation system, grounding flights nationwide due to data overload in an aging, patchwork air traffic control network. This failure highlights systemic issues: decades of underinvestment, outdated infrastructure, and political delays in critical upgrades.

Simultaneously, Microsoft Outlook suffered a massive, prolonged outage caused by a faulty internal update that spread through its cloud infrastructure, showing how software patching risks can cascade across modern tech systems. Both incidents underscore a broader pattern of technological fragility, rooted in poor maintenance, reactive updates, and insufficient investment. S.

compares poorly to countries like Canada and Europe, which use public fees and less aviation demand to sustain infrastructure. Meanwhile, cybersecurity threats—including an Iranian cyberattack on supermarket fridges and a $6 billion crypto heist—demonstrate that vulnerabilities are not new but are now amplified by AI tools that accelerate both discovery and exploitation. Experts warn that while such outages are cyclical, the increasing complexity of AI and digital systems, combined with political instability and inadequate funding, makes the current crisis more severe than ever.

A long-term, bipartisan solution—funded by sustained investment in both aviation and digital infrastructure—is urgently needed to prevent future collapses.

FAQs

A faulty industrial router installed during a maintenance update in Arkansas triggered a data overload in the FAA's aging aviation infrastructure, causing widespread flight groundings.

The router was left in a factory setting that sent hundreds of data requests per second, overwhelming the outdated, mixed-age network and causing packet loss critical to flight tracking and safety.

The U.S. has the world's most crowded airspace and relies heavily on flights for travel, while countries like Europe and Canada use more trains for short trips and have better-funded, more modern infrastructure with fees that support maintenance.

Lack of sustained, bipartisan funding over decades has delayed large-scale upgrades, and political shifts have caused inconsistent investment, making systemic fixes difficult and slow to implement.

A flawed Windows infrastructure update, deployed over three months without detection, caused a widespread and prolonged outage affecting Outlook, Teams, and other Microsoft 365 services.

Yes, such outages are recurring due to patching risks and system complexity. Experts note that while organizations learn from each incident, the underlying risks remain, and future failures are likely unless systemic improvements are made.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.