Go back

Why Everything Is Breaking All at Once

1h 16m 33s

Why Everything Is Breaking All at Once

A recent surge in critical infrastructure outages—such as FAA ground stops, banking disruptions, and NSA system failures—has been primarily driven by the defensive response to AI-powered vulnerability detection, not direct cyberattacks. As AI models rapidly identify flaws in legacy software, companies and government agencies are rushing to patch systems, often without adequate testing, leading to cascading failures. This highlights a deep "digital deficit" where decades of unpatched, poorly maintained code have finally come due. While AI is also being used offensively in autonomous attacks through tools like RTEX, the immediate crisis stems from the speed of patching and the fragility of outdated systems. Government technology, particularly in public services like air traffic control and unemployment systems, suffers from a lack of internal expertise, rigid procurement models, and outsourced management that stifles innovation and resilience. The current wave of failures is not a sign of an imminent AI apocalypse, but a symptom of systemic neglect in software quality and governance. Media coverage amplifies the crisis, creating a perception of chaos that may not reflect the underlying reality. Experts argue that lasting solutions require a fundamental rethinking of how technology is developed and managed—shifting from costly, bureaucratic contracts to agile, expert-led, and resilient designs. In the short term, the pain from outages will persist, but the long-term path forward lies in addressing the root causes of digital fragility, ensuring that critical systems are not just patched, but fundamentally restructured.

Transcription

11789 Words, 66359 Characters

English
Today, why everything's breaking. So last week I was giving a talk in Boston, and on the auspicious date of Friday the 13th, I boarded a plane to fly back to Washington, DC. I get on the plane, I put away my luggage, I settle into my little seat, and literally 10 minutes before we were scheduled to taxi, the flight attendant announces the entire plane, that I'm sorry we're going to have to disembark the flights delayed for maybe no more than 45 minutes an hour, apparently some wires got crossed at FAA, and so a few planes will have to remain grounded, it'll just be an hour again. So we all get off the plane, and I pull out my phone, and within minutes the story of a ground stoppage is at the top of the New York Times, CNN, ABC, every flight in the United States has been halted the third ground stop of the century, and to yada yada yada to the rest of the story, I ended up driving home from Boston to DC. As everybody knows, by the time you're listening to this emergency episode, the FAA outage was just the beginning. On Monday, United Airlines announced its own tech outage. On Tuesday, several banks, including JP Morgan, Bank of America, said its customers couldn't withdraw funds due to a software bug. On the same day, the NSA announced that its communication systems were hit with another outage, then Apple announced its own difficulties. On the same day, Google separately warned that quote, the era of autonomous attacks has arrived, and the FBI issued a flash advisory warning that cyber criminals and state aligned hackers were using AI to attack businesses and organizations around the world. Suddenly it seemed like the most important question in news was, why is everything breaking all at once? So as I see it, there's several possibilities here. Possibly number one is a coordinated systemic AI hack, the cyber apocalypse that we've all been warned about. Number two, maybe these outages are related to AI, not because of its offensive attacking capabilities, but because of its defensive capabilities. As AI discovers vulnerabilities in existing software, engineers are rushing to fix those problems, patch those problems, and that rush might be causing its own cascade of failures. But number three, maybe the number of outages isn't statistically abnormal at all, but we've been trained to expect a cascade of hacks from advanced AI, and so the news media is more sensitive to outages than it used to be, creating a false impression when the underlying reality is that internet infrastructure is always breaking, whether it's private sector or public sector infrastructure. But only now are we associating its breaking with this larger story of AI is drawing the world. It reminds me, for example, of the summer of 2001, when if you remember this, there was a huge increase in coverage of shark attacks. This became known as the summer of sharks. But only years later did statisticians look back at the number of shark deaths in the summer of 2001 versus other years and realize that the only thing unusual about the summer of sharks was the news coverage. Shark fatalities were basically running at a historical average. It's just that the news suddenly increased about shark-seating people. So is the everything outage that we've been experiencing? A, the result of AI offensive attacks, be the result of AI defensive patching, or see just some kind of frequency illusion where we're all paying so much attention to the story because we've been trained by everyone talking about the AI cyber apocalypse that something like this was coming. To understand what's going on and what we can do about it, today we have two guests. Nicole Pearl Roth is a cybersecurity expert who spent years as a reporter before starting a company that studies and tries to fix cyber vulnerabilities. And second, we've got Jennifer Polka, an author and former deputy chief technology officer of the U.S. government, who has written deeply on why government technology sucks and what it would actually take to improve it. I'm Derek Thompson. This is Plane English. This episode is brought to you by JP Morgan Chase. In today's unpredictable world, geopolitics isn't just a backdrop. It's shaping markets, business strategy and risk. The JP Morgan Chase Center for Geopolitics brings together former senior policy makers and seasoned practitioners to translate fast-moving global events into clear decision-ready insights from conflict and great power competition to energy and supply chain disruption. The center helps clients anticipate what comes next. Read the latest insights at jpmorganchase.com/centerforgeopolitics Nicole Pearl Roth, welcome to the show. Thanks for having me, Derek. Great to be here. Tell me who you are and what you do. Yeah, it's a longer answer these days than it used to be. I used to lead cyber investigations at the New York Times. I left in 2022 to join the cyber defense agencies advisory board. And then since then, I've been a venture partner at ballistic ventures, which invests in cyber security companies. Before we get to your analysis of why what happened in the last week happened, I would just like to hear from your expert perspective, how would you even describe what's transpired in the last week? I think the best way to understand it is something we don't talk enough about, which is that we have this great digital deficit in this country. We've talked a lot about the fiscal deficit. I always find it very interesting that politicians actually run on the deficit and fiscal responsibility, but we never talk about the digital deficit. And the digital deficit is just that we have been rolling out flawed code. We've been leaving that flawed code in systems. We haven't been updating it for years. And we've also been baking code into every nook and cranny of our economy, of critical infrastructure, our homes, our cars, our bodies, in some cases. And no one ever stopped to think that we were just adding and adding and adding to the digital deficit that would one day come due. And with AI, it is now coming due in a big way. And what I mean by that is AI models have proven that they have tremendous capabilities to find vulnerabilities in code and exploit it to break into systems on the other side. So all of these errors that we rolled out in this code and never addressed are now being found by these powerful models and stitched together in such a way to attack systems. And we are finding 3X, the number of vulnerabilities using AI this year, then we were in the same period last year. Attackers are exploiting those vulnerabilities using AI at a much quicker rate than we've ever seen. By the time a company learns it has a flaw in its code, rolls out a patch, it turns out that on average, attackers have already been exploiting that flaw for seven days now. What this means is that well, we used to be able to live on this patch Tuesday cycle, where companies would roll out fixes that were discovered for vulnerabilities in their code. Companies would have time to update their systems and we were all on the software manufacturer's schedule. Now we're on the attacker's schedule. So companies, government agencies, they're all finding it very urgent to install these patches and software updates, much more so than they have in previous years or decades. The problem is, it's really easy to find a vulnerability now and exploit it. It's still really hard and time consuming to properly install a patch. And when you install a patch, you do it in these bespoke environments. No environment is the same. You can't just push a big red button and everything's fixed. Usually it's likely that something will break on the other end. And what we have seen over this past week, of outage, after outage, after outage, is just that. Companies, government agencies are installing these patches and software updates to fix vulnerabilities that were discovered with AI and they are breaking things on the other end. So these are not cyber attacks, but the impact is the same. Me two really important points, so I want to make sure I understand. Advanced AI has offensive capabilities to discover cyber vulnerabilities and attack, but AI also has defensive capabilities to identify those failures in code and then patch them before the attack happens. You're saying that what we've been seeing over the last few days is not so much evidence that AI hackers are breaking into systems like United, the FAA, but really Rather that sophisticated AI tools are fine day vulnerabilities in all this critical software, and the patches themselves are coming in so fast and messy that they're causing their own breakages, kind of a little bit like an own goal situation for soccer fans out there. Is that right that the attempt to be properly defensive in the face of likely cyber attacks? Is what has really been causing these cascading failures the last few days? So almost, I think there's a few caveats there. One is defensive AI. AI for cyber security for defense is not quite where we need it to be yet. Again, it's much easier to find an exploitive vulnerability using AI than it is to fix that vulnerability using AI. We're not quite there yet. There's still an asymmetric advantage for offense. Number two to say that attackers aren't using AI yet, that's actually not true. Separate from all these outages that we're seeing, we are seeing a huge uptick in autonomous attacks from all kinds of attackers that are using AI and a big red blinking light right now, is that we don't have enough people to help companies that have been hit with these AI attacks recover. There is more than a six month backlog at Mandient, which is, you know, the first 911 call you usually make when you are breached. Crowdstrike, they are totally out of bandwidth. They are turning breached victims away. They actually restructured Crowdstrike, they announced it yesterday so that people at the very top of the organization who haven't triaged an incident in years are being asked and pulled into incident responses because there's just so many breaches that are happening right now. So two things are true at the same time. AI is absolutely increasing the number of incidents and the velocity of these incidents. It's much harder to triage these incidents because when you're hit with an agent, they don't follow the same workflow as a human attacker. You know, we had a pretty good understanding of what the kill chain looks like. You break in, whether it's through fishing or a zero day, then you have some lateral movement as they figure out where are the systems of interest. Then you figure out how to exxful trade or hold hostage data that would be particularly useful. Sometimes you would catch it in the lateral movement phase. Ideally, you catch it at the point of breach. But we had a pretty good sense of how these things worked with agents. When they break in, they are doing things that befuddle our puny human mind. So it's much harder to triage these incidents than it has been. So that is a whole category of things that is absolutely happening right now and is going to get worse and worse and worse. What happened with these outages that are almost simultaneous, right? FAA, the bank issues that happened or the bank disruptions, I guess we should call them, the issue on Sunday or Monday with United Airlines Monday, the NSA net outage yesterday. Those are companies or organizations that are trying to get ahead of this huge tsunami of AI-powered attacks by installing software updates as quickly as they come. Those software updates have not gone through the proper channels, quality control, testing that they might have in an earlier pre-AI era and they are breaking things on the other end, which is to be expected. I saw some people ask the question of whether there are more outages now than there have been in previous weeks or months or whether another thing that's happening is that we're paying closer attention to outages because we are sensitive to the imminence of a hackmageddon cyber apocalypse, whatever headline word you want to put here. Does anyone keep a baseline for how often critical systems fail such that we could see, for example, "Oh, these kind of outages are up 15% over normal." Do we have an answer to that question? Yeah, I saw some people use AI to conduct a timeline analysis of whether we've had these outage clusters before and we absolutely have. I think that one was around 2013 time frame. It's not the first time we've had simultaneous outages or near simultaneous outages. I do think you're right that a lot of this is just baked into the attention that is on AI right now. I don't know that we've had outages in these very public systems, critical infrastructure that would capture our attention in the same way. I don't know if that was not the most articulate explanation, but I do think for years I have been warning about two critical infrastructure systems in particular that kept me up at night. When you cover cyber security for a living, or you write a book called, "This is how they tell me the world ends." You always get the same question at the end of the panel, which is, "What keeps you up at night?" I've always highlighted two potential scenarios. One, an attack on our water treatment facilities that would contaminate the water or lead to a water shut off, and two, an attack on air traffic control. I think I am particularly heightened to the latter there, and I was actually flying to my brother-in-law's wedding on Friday when this went down. I thought, "Everyone just needs to get in the car. I've been thinking about this scenario for years. We are not going to get out of this anytime soon. We need to just start driving." I think there is something very visceral about an FAA ground stop, a national ground stop, and the fact that we haven't had it since 9/11. Obviously, it's going to capture the national imagination, but then it's sort of add in what happened without look, and the day is leading up to it, what happened to the banks, et cetera, et cetera. Before we return to the news, you mentioned water systems and air traffic control. In a previous conversation about cyber vulnerabilities with Alex Stamos, he said that water systems were his greatest fear. They were the thing that he kept them up at night. Can you remind me of all the things to worry about, why water systems and air traffic control? Is there something about those systems that is particularly attractive to attackers vulnerable because of the size or sophistication of their cyber defenses? What is it about water and air? A few years ago, there was a hack, and we still don't even know whether it was a hack, but it was on a water treatment facility in a town in Florida, just outside where the Super Bowl was that year, in the weeks leading up to the Super Bowl. The way the story was initially relieved to us, an engineer was watching his screen and this ghost mouse moved across his screen and moved over to the chemical controls and increased the amount of L-Y-E in the water and could have potentially poisoned everyone downstream. When we started digging into this, it was pretty terrifying to get a sense of the state of cybersecurity around our water treatment facilities. I think that there is, unfortunately, and I learned more and more about it the more I've fallen down the cyber rabbit hole, but there is, unfortunately, a very big story of the haves and have nots in cybersecurity. Companies like JP Morgan, Goldman Sachs, the banks, certain critical infrastructure providers have the budgets to be able to buy every bell and whistle and cyber security and to technically be covered there, but the little guys don't, and water treatment facilities are ones that are notoriously underfunded, understaffed, and often have to rely on outside contractors who remote in to manage things like the chemical levels and the water, and that naturally creates an opening for hackers. And often the people who set these systems up, we're not thinking about the fact that one day an Iranian hacker might want to contaminate the water supply in the United States. So often, they do it with all the poorest cyber hygiene, like password for password, no two-factor authentication, et cetera, et cetera. And so we learned that this is a particularly ripe attack surface for a terrorist or a nation state that wants to do us harm. And when you think about it, I always say, like, great, you know, roach, they can afford all these bells and whistles so that, you know, critical medications or vaccines aren't paralyzed in a cyber attack, but what happens when they can't get clean water? It's still going to impact them. And so we actually can't have a system of halves and have knots in cyber security, it doesn't make sense. We're an ecosystem. And if part of that ecosystem, the most critical part of that ecosystem breaks, it's gonna break the big guys too. So there are these linch pens that we have in society, and I think water is the one that is probably the most susceptible to an attack, even though it's the most critical. If defensive AI means that we're gonna be patching faster than ever in the next few weeks to get ahead of potential cyber attacks in the next few months, would you advise people to expect more outages like this in the near future, or do you feel like maybe we're beyond a little bump in this spate of outages that happened to cluster around a single week? - No, like I said yesterday, this is definitely the opening chapter. We haven't seen until really the last couple of months, in nation states and cyber criminals start leveraging autonomous offensive AI attack tools to come for organizations, but they are now. And we are also seeing attack frameworks like RTEX dropped in their lap. These are tools that can basically leverage different models to search for, find, exploit vulnerabilities at machine speed and a couple of these, we're part of an FBI advisory just this week. And it adds pressure on companies to fix these vulnerabilities and run these software updates as much as possible. So we are in actually a very compressed window right now, not just because of the added pressure from these autonomous attack tools being leveraged and coming to market, but also because we're just ahead of Thanksgiving and the holidays. And so a lot of organizations want to run these updates just before the holidays. So they're protected going into that cycle. So we are in a uniquely compressed timeframe, but this is just the early innings of these models. These are, we always say, the least sophisticated of these capabilities we will ever see. And as I wrote in my book, there are as many vulnerabilities out there as there are stars in the sky. It might not be infinite, but we have truly failed to take quality control of software seriously for decades. Even if magically we were able to tomorrow, use AI, leverage AI to write secure code. There are a lot of systems out there running legacy code that can't update their systems, even if they want to, that can't even get a patch if they want to. I mean, this really is the digital deficit coming due. I think once we have enough of these very public critical outages, we will see the political will summon the budgets to finally rip and replace these old legacy systems. But even that will cause a lot of pain in the process because naturally it will lead to things like misconfigurations and things will break and there will be cascading failures. But it needs to happen, right? Because the idea we're just going to all sit around twiddling our thumbs while AI attackers are able to leverage every vulnerability, chain them together for successful attacks and critical systems. It's a hell, we don't want to walk towards. So we are now looking around and there will be a lot of pain and flickering and outages in the near term. - When I think about the warnings about an existential coding risk, a digital deficit coming due, my mind turns to Y2K. And once again, 26 years later, AI is identifying flaws in programming languages, PHP, Java. You have the risk of cyber attacks, finding vulnerabilities that AI is identifying in corporations, in the federal government. How does the Y2K metaphor serve us and disservice? How is this moment like and unlike Y2K? - Well, yeah, I mean Y2K is like the other extreme, right? You don't want to scare people and then all of a sudden you look around and everything's fine. But in this case, Y2K was baked around one timing issue, but what we're talking about now is near infinite vulnerabilities in all kinds of code. That AI is finding it's able to stitch together vulnerabilities that humans couldn't before, again, with our puny little brains and exploit them at machine speed. And so it's very different because it was like, it's almost like with Y2K, we're talking about one vulnerability. In essence, now we're talking about many, many different potential vulnerabilities that AI is turning up in these systems and across many, many different hardware software configurations and because we baked that code into so much more critical infrastructure than we ever had in our Y2K moment, it's going to impact our livelihoods in a much more real way. - I want you to help me understand a couple warnings that came over the transom yesterday, one from the FBI and the other from Google's Threat Intelligence Group. So the FBI issued what they said was a flash advisory warning that cyber criminals and state-aligned actors are using AI to attack businesses and organizations around the world using something called RTEX, ARTEX. At the same time, Google's Threat Intelligence Group also said that the era of autonomous attacks has arrived, that's a quote, the era of autonomous attacks has arrived. And while they haven't yet seen any fully autonomous start to finish attack in the wild, bring down some enormous infrastructure system, it's suspected that we're going to start to see something like this in the next few months. Can you help me understand what RTEX is and what makes it so dangerous? - RTEX is essentially a tool that makes a lot of different kinds of autonomous attacks as easy to execute as pushing a button, right? It automates the process of leveraging these models to scan for exploit vulnerabilities and systems on the other end. And so someone who has no technical expertise whatsoever can now leverage this tool to attack systems, to find vulnerable systems and attack them by pressing the easy button. And what that means is that attack capabilities that once were the providence of an advanced nation state or cyber criminal group now potentially belong to a much, much broader pool of people. So third tier nation states that never really had the capability to execute an attack using by chaining different vulnerabilities together can now use RTEX to do so. And there are a lot of entities like that that wish to do the United States arm. We've seen Iran probing our water treatment facilities North Korea is much more financially motivated. And while we've seen them use some zero days in their attacks, they weren't, they didn't specialize in finding and explaining zero days. But now these AI models make it much, much easier for them to do so. Cyber criminal groups, you know, when you think about a ransomware attack, a lot of it was manual, you know, you had to go, you had to find vulnerable companies to attack, you needed to go through a list of those companies to find out how many of these are actually worth hacking, how many are likely to pay, how many actually have cyber insurance policies. That's something that they would actively look for. And they were much more likely to hit those companies. Then they had to find the systems of interest or the data to encrypt. Then they had to manage the payment negotiation. All of that has now been fully automated using these tools. You know, you can tell that the AI, you know, you can use the prompt, find me systems of interest that are likely to pay a $10 million ransom, find the data of interest to encrypt and manage the payment negotiation for me for maximum psychological pressure done. And so those tools have now fallen into the laps of cyber criminals, nation states, you know, what I call digital arsonists, people who just have some sick fixation with watching the world burn. We just threw AI attack offensive autonomous tools into their laps like our texts. And that dynamic is going to lead to far more incidents. And again, we are running out of bandwidth on the incident response side. When you call 911 now and breach response, the most likely answer you're going to get on the other end of the line is, we have to turn you away, sorry, we're busy. And so for organizations, that means they have to get ahead of this thing. And that, again, means running software updates as quickly as possible, even if It might break something on the other end. So this is just going to keep happening. As far as Google Thread Intelligence Group's warning, yes, that is not new news. But it means more coming from Google Thread Intelligence Group. But we've seen these attacks get some automation coming over the last two years. It's just that now we're getting closer and closer to full automation. I have a question about the timeline that you're looking at here. Because as you said, at the top of the show, the outages from the past week were mostly on the defensive side. That was defensive patching leading to problems. But separate from what we saw in the last week, you, the FBI, Google, you're also saying that an era of offensive attacks are coming. And to your point, we have seen inklings of this, whether it's Iran targeting America's water systems or that incident where refrigerators in Europe shut down in another incident that I believe was linked to Iran. But if we're just looking at headlines in American news of offensive attacks shutting down American infrastructure systems, it doesn't seem like we've seen that yet. The water systems aren't yet poisoned. ATMs are still working. Flights are taking off at least when there isn't a problem with uploading some communication software in Arkansas. Where do you think we are in terms of the cyber offense targeting timeline? When would you expect the headlines in the New York Times and Wall Street Journal to be about systems breaking because of cyber attacks rather than systems breaking because efforts to prevent cyber attacks led to some own goals by coders? Well, I will be very frank here and say that I don't think the cyber security reporting out there is actually capturing how many attacks are playing out right now. If you talk to any system that's a chief information security officer at a major company right now, they will tell you we are under constant attack from autonomous attackers or AI-enabled attackers. It is constant, it's hitting us thousands of times a day. So that is absolutely happening. The companies that are having major issues are critical to the supply chain, my understanding, and having talked to some of them. Critical in backend payment infrastructure, critical in staffing, and hiring contractors. Those companies were the target of a significant extortion attack from an AI-enabled attack just in the past two weeks. I was partly hosting a bunch of chief information security officers at a conference just a week and a half ago. And there were many attendees who could not show up because they were all being targeted simultaneously with this state of extortion attacks. So there is a lot that is not getting reported right now. And I will come back to this point. People have not paid enough attention to cybersecurity. Covering cybersecurity at the New York Times was a very big uphill battle. Trying to get editors to care about cybersecurity was a huge uphill battle for me. And eventually, I just decided I'm going to stop writing about this problem. And I'm going to go join the guys who are trying to solve it because trying to get people to care about this issue is a huge difficult feat. And I can do it for 13 years. But I don't know if I can do it for one day longer. There are attacks happening all the time. And the signal that you need to look at again is that we are out of bandwidth on incident response. Again, CrowdStrike just completely reorganized itself to the fact that people who are at the top of the company-- I heard someone in the president role at CrowdStrike who hasn't triaged an incident or led an incident response in years. They are a very senior role at a publicly traded American company, multi-billion dollar American company, is getting dragged in to lead incident response right now. Law firms that are paid on retainer that specialize in helping their clients manage incident response can't fulfill their obligations right now. So that is the blinking red light. And I think that we need to do a lot better job trying to figure out where the breaches are happening and how it might break things in the supply chain. It's interesting that AI is already very unpopular. I mean, this came out. This message was resonated loudly in the midterms. And that's before any obvious hit to jobs or to offensive cybersecurity. Whether or not the last week was a spade of AI attacks or major AI assisted attacks in critical pieces of infrastructure as opposed to something that was more on the defense side. It seems to me like those attacks are coming, where some major AI assisted attack on a critical piece of infrastructure, or as you said, some part of a critical supply chain is likely going to happen. What's going to happen to the AI discussion when an attack is confirmed? You see what I'm saying? The country is already, so many people in the country are already incredibly frustrated with this technology before the technology breaks something. What happens after a critical attack is caused by state or non-state actors using AI to break some piece of American infrastructure? - It's really interesting because for years, cybersecurity experts have told me, we're going to have a cyber 9/11. We're going to have a cyber pearl harbor, but it never happened. And I would always ask them, so when is this event going to happen? And almost comically, they would always say, 18 to 24 months, Nicole, 18 to 24 months. It's like just short enough to add urgency, but just long enough that I might not hold them to it if it didn't come true. I think actually, we are in that critical 18 month period where we are going to see systems that we have taken for granted stop working. And as far as how that's going to shift the AI conversation, I welcome that moment in some ways as someone who's been warning about this for years. It was very frustrating when we had our mythos moment. When Dario and Anthropa came out and said, we're going to withhold this model from public consumption because we see it has very powerful capabilities, cyber capabilities. And it was very frustrating to see what the reaction was from the administration, even from people in the cyber security industry who said, this is all marketing. Our text is proof of what Anthropa was trying to warn everyone out about with mythos, which is that these models have very, very real offensive cyber capabilities. And they are going to be thrown in everyone's lap and come back to bite us. My hope is that the conversation will be a nuanced one. My hope is that we can finally have this conversation around, well, let me back up and say, there were two things that collided over the decade that I covered cyber security for the New York Times. One was move fast and break things. Just get your app, get your whistle to market as quickly as possible. You can worry about the security problems later. Mark Zuckerberg. And the other was Mark Andrews in software eats world. Let's get software into every nook and cranny of our physical infrastructure. And it will just add to our efficiencies. It will make our lives better. It might save costs in some cases. It'll allow us to remotely monitor an issue on an oil rig, hundreds of miles offshore. No one stopped to say, wait a minute, should we be putting code into everything? Should we be digitally connecting everything? What happens when this becomes a tax surface? And now we're about to see what happens when all of that code presents as an attack surface that can be leveraged and exploited against us at machine speed. And I think we're in for a very big wake up call. And my hope is that we will have much more nuanced conversations around should we be digitizing everything? Are there some systems that are so precious to us that we should keep them analog, that we should air gap them, that we should absolutely make it mission critical that no one can access this system from the internet? And we just haven't had that conversation yet. The story that you've been telling is a story about AI moving faster than we can handle. But to be perfectly honest, it's not the story that I expected to tell on this show in November of 2026. When I was talking to cyber experts after Mythos came out, and Dario, as you said, withheld the initial technology from the point of view. the publics that he could share it with Project Glasswing with the handful of companies to fortify their systems. What I heard was what's coming is going to be a cascade of offensive cyber attacks. Though that's what the first news headline is going to be. Systems breaking because they are hacked. I did not hear many cyber experts say the first big headline you might hear is that AI would find so many bugs that fixing them quickly would be its own source of breakage and outages. That's a non-obvious result to me as a non-expert in the subject. I wonder if you can help me think through what the next non-obvious chapter of this story is. Is there another ironic crisis that could unfold from AI's mounting capabilities in the cyberspace that's similar to some of the first breakages who won't be hackers breaking systems but actually defenders rushing the fixes for bugs that accidentally break their own systems? >> No, it's a really good question. What surprises await us particularly of the ironic kind? I think unfortunately my answer's pretty boring. I think it's going to be more of the same. It's when I cover this for the New York Times and when you cover these things as a journalist, it's very easy to fall into the black and white territory of you have a bug fix it. There's a major bug in your iOS, iPhone, software that the government finds. They should just fix it instead of hoard it and use it for their espionage or surveillance operations. But it turns out that actually the moment you disclose a vulnerability or release a patch is often when we see the highest amount of exploitation because the first things people will do is scan to see who's still vulnerable, who hasn't rolled out the software update. They will reverse engineer the patch to see if there's another way that they can exploit the underlying bug. And so it just creates more vulnerability in that short-term window. It's like a short-term window of pain. And that's what we're in right now is the short-term window of pain. And the big question is, how long will this last? The optimistic, the most optimistic take on this I've heard is the SISO at Anthropic, who says it will be 18 months. I am much more pessimistic because now that I'm not technically a journalist, now that I've fallen down the cyber rabbit hole, I've seen how hard it is to actually install a patch across different environments, how often things can break on the other end, how these cycles work. In a lot of cases, no patches wherever run and things in critical systems like water, like some pipeline networks, like the power grid, because the impact of trying to fix the underlying issue was no different than the scenario you're trying to buttress yourself against by rolling out the software update. And so the answer was, let's just not run them. And now, with AI, all of those vulnerabilities will be found and exploited at machine speed. So it's not just one attacker knocking on your door, moving around, seeing where they could potentially shut something off. It's every potential attacker in the world attacking you at machine speed all at once. And so the pressure to fix these outages before it can be used by a third party is so high that people are gonna decide it's much better to own myself and buttress myself than it is to let others own me. So we're just, we're in for this period of pain, but we're also in for some pain now that these attacks are going autonomous. So it is truly what we've always called that everything everywhere, all at once, cyber scenario, we used to use that phrase in the context of China because we are very much seeing Volt Typhoon, China's PLA hackers embedding themselves in American infrastructure, that they're not hacking us just yet. They're just laying low, almost like sleeper cells, waiting for future instructions. And so we've talked about it. We qualified it as the everything everywhere all at once cyber attack where China might decide when it makes a move on Taiwan, why not flicker the lights on and off in the US at the same time that we shut off access to gas, shut off access to ATMs to air traffic control so that the United States might think twice before supporting an island 7,000 miles away. And so we contemplated it in that context, but now what we're seeing, and again, some of it self-inflicted with the software updates and faulty updates, but a lot of it's gonna be at the hands of attackers, the true everything everywhere, all at once cyber scenario, where things are going to flicker on and off. And I mentioned this on Twitter the other day, but I think software Beirut is really what we're looking at, where systems that we took for granted, that they would just work all the time. And we are the generation of instant gratification, so we're gonna have some trouble with this. We're gonna start waking up and finding that systems that we counted on are flickering on and off, whether it's air traffic control or not getting access to zeal or not being able to get money out that day. It's gonna start to happen. And again, the bigger question is, how long will that period last? I think eventually the hope is that AI will be able to help us roll out much more secure code and avoid misconfigurations and buffer overflows that have been existence in existence for decades. And so we will get there. We'll have to get there. It's just that right now the bill has come due and it's going to be painful. - Very last question. Within this window of pain, at the next 18, 24, 36 months, a lot of people listening are going to be about to fly home or drive to be with people for Thanksgiving. What should they do? They tell their family to do that they can actually control 'cause they can't control their traffic control. They can't control the quality of code at J.B. Morgan or Microsoft, but they can control what they can control. What should they do? - I think the thing that you need to do and I am not a prepper, but I probably should be because I wrote this book called, this is how they tell me the world ends. Print out your bank statements. Print out critical documents. Make sure that you are as hardened as possible, basic cyber hygiene, backup things that you wouldn't want to lose. Live your life, but also live your life in such a way that your worst day is not going to be the last day. Make sure that the advice I always give people and I also give it to companies is identify the thing that is most precious to you and harden it as much as possible. So if you are a company and you sell, if you're Coca-Cola and you're most precious thing as your IP or for the formula for Coke or your mergers and acquisition strategy or your factory real estate strategy, don't leave that open for anyone at the company to find, add several layers of hardening and multifactor authentication around it. You'll continually ask yourself, does this person actually need access to this? Have just in time access requirements, et cetera, et cetera. If you're a person like me, it's going to look very different. It's going to be something like if I was, in my journalism days, it was my sources. So I just took those conversations completely offline in some cases. I got a car that's not a smart car so that I could drive to some of these source meetings. I wouldn't bring a device, I would never take an Uber, I would bring pen and paper, we'd never communicate, we'd never linked on LinkedIn, we'd never emailed, we'd never texted, we met at the same place on the same day, every quarter, and sometimes one of us wouldn't show up, but those were the very paranoid links I would go to to protect the thing that was professionally most precious to me. And I think all of us need to basically ask ourselves that question and take that thing offline or as do as much as we possibly can to protect it. Nicole, ProRuth, thank you very much. Thank you so much, Derek, great to speak with you. This episode is brought to you by Gira by Atlassian. Your work is scattered. A thread here, a DM there, five different docs related to the same project, and now it's not just your team hunting for context, it's your AI agents too. Gira taps into Atlassian's teamwork graph to bring that context together across Gira, Confluence, Slack, GitHub and more, and makes it easy to share with the agents you already use, like Claude, Cursor, and GitHub, Copilot. Assign them work directly or connect your agents through Atlassian MCP. The result less token waste, more accuracy, and more time moving forward. Learn more at gira.com. That's j-i-r-a.com. Many thanks to Nicole Pearl Roth. So one takeaway from that conversation is that there's this cat and mouse game between offensive and defensive AI, and that while ironically the last week was the result of defensive patches in your future. Good feature. more chaos on the offensive side. Nicole's focus in our conversation was mostly on the private sector, on state and local actors potentially attacking critical pieces of infrastructure in the private sector world. But as our next guest is going to talk about, it's a very different story in the public sector. As Jennifer Palca explains, there have been issues with the way that the government runs its technologies, procures its technologies, modernizes its technologies that has been broken, not just since the advent of Chatch B.T. in 2022, but really for the last 50 years, arguably the last century. And so her interest is how do we take this moment and change the way that we update technology at a moment when there might be more AI risk toward taking critical public infrastructure down? That's the subject of this next conversation. Jennifer Palca, welcome to the show. Great to be here. I want to focus our time on the government technology failures the last few days, which have drawn a lot of attention and a lot of conspiracy theories. First to help us understand the what here, what do you think is the story to focus on when we look at the FAA ground halt and the NSA communication knowledge? Well, you know, mistakes are going to happen. I tend not to think that this is the sign of an apocalypse. We have a lot of systems in the public and the private sector that have gotten very complex and that makes them fragile. And we're going to have things like this because of the world that we have built that we live in. And we're so dependent on software. But to me, the real story of what's different with say the FAA versus the chase outage is that government doesn't own these systems. We outsource all this stuff. And that makes it really hard for government to learn and get better because in the end of the day, if you're going to have outages like this, it's not about necessarily not having them happen. It's about being more resilient from them. And if you look at say the chase outage, like, you know, I'm not sure I know of one that happened at chase recently. But if you look in the history of the FAA, there's actually quite a few that have gone down over the past, well, a couple decades and then the past couple of years. And to me, that's a sign of an organization that's outsourced not just its network, right? Like, we don't own that we, meaning the public doesn't own that network. The vendor does. But it's also outsourced sort of the expertise and the knowledge and the ability to learn and adapt. And that doesn't work. Let's slow down on the FAA ground stop. This is just the third time this century we've had an FAA ground stop. But it's the second time as I think you just pointed out in the last few years. Is it useful to see this as a pattern? And if so, what is it a pattern of? Well, I think one thing we're seeing is a pattern of politicians saying when something goes wrong, give us more money. And we have been giving them more money and it's not making things go better. And the reason for that, in my view, is that things get better when you're able to have the internal expertise and competence that says this is how we're going to do this. We treat, in government, we treat problems like let's keep all the planes in the air or let's make a website that everyone can use to sign up for their health insurance or let's help kids apply for financial aid as problems of contracting. And so the people who get applied to the problem are people who understand not just like how to hire a vendor to do a job, but also all the like really obscure and complicated ways in which government does that, like it's way harder in this section. There's this whole area of expertise. And these people are incredibly smart and know an incredible number of things about how you contract in government. But that's the expertise and talent that we put on the problem, not the expertise of what would a robust and scalable network that stayed up would look like. And the people who make these contracts are not necessarily the people who know how to say manage that vendor because what they do is contract management, not technology management. And I'm not saying there's not those people, but there's a real imbalance of power in government between the people who are doing contracts and compliance and procedure and the people who actually substantively know what it is we're trying to achieve here and how that should best be achieved and how the how the vendor should operate. And in some cases where we're saying sort of the opposite pattern where things are getting better, like on the FAFSA, I mentioned, you know, kids applying for financial aid, big disaster, you know, you know, really terrible. A lot of kids couldn't apply. I think that was 23, maybe that was 24. Now not only is it better, like they stabilized it, but it's actually a really good application form. People like using it has something like a 90% approval rate from students and their parents. And that's because we brought a lot of that knowledge in house. We hired product managers and engineers and user researchers and people who know how to use modern technology. We didn't fire all the vendors there, but we are managing them in a fundamentally different way. They are setting the priorities. They know what good looks like and it's people with the correct expertise fundamentally in charge of the process. Two ideas here that I think are really important that I want to tease apart. The first idea that I'm hearing is that government does a lot of stuff. It does student loans. It does air traffic control. It does social security. It does the census. Some of these things are done internally with internal expertise. Some of these services we contract out other companies, other tech companies, other vendors are really the ones providing the service and the US government is just paying them to do that service. And you're saying that this leads to a situation where expertise is outsourced. And so when something breaks, the people in government have no idea what broke. They have to ask these people outside of government, hey, do you have any ideas how to fix this system even though you're collaborating with a bunch of other vendors on a system. So you only understand half of what has broken. So that's a problem here. There's another problem which is the money. Can I try to interrupt you on that one, though? Please, yes. Okay, so there's a spectrum, though. It's not binary, right? Like there are places like census where they have enormous amounts of fantastic technical talent. And even there, they'll sometimes hedge their bets. Dana Boyd has a fantastic new book called Data Are Made Not Found that talks about this where you have great people internally and they actually did a system, but they also outsourced it because they figured, well, what actually happened is they outsourced it. And the internal people said, we don't trust that the vendor is going to deliver. We're going to do our own internal version. The vendor didn't deliver. And so it was important that they had done an internal version. But even at a place with a lot of good technical talent, you're still using vendors. And then even in a place where you're outsourcing, there's degrees of sort of control and direction. At the far end of the spectrum, the sin that government commits is just abdication of responsibility to vendors. There's a line that you will hear in government contracting circles all the time. This, we need one throat to choke. This basically means we know it will fail, but we will blame them. That does not work because the American public still blames the government, understandably. So there's this other piece that I think is going to really surprise people that you're saying, which is that more money won't solve the problem. I think a lot of people, when they look at technical problems, think, well, if only we invested more money to solve this problem, the problem itself would be solved. This is what you're hearing transportation secretary Sean Duffy demanding at news conferences every day, every hour, sometimes it feels. Congress needs to pony up another $50 billion to upgrade the air traffic control system. And then just if we just spend another $50 billion, we will have perfect technology that solves the problem of air traffic control. It's specifically as possible. Can you explain why giving Sean Duffy and the Department of Transportation another $50 billion does not solve the problem? Well, first of all, you can tell from the kind of escalation of the requests from like 10 to 30 to 50 that these are not engineering driven numbers. This is someone really pulling pulling it out of his butt, so to speak. And just saying, you know, more money, it just comes from belief the more money is better. We haven't, they have not spent a lot of that money that they've already been allocated. But of course, you know, what he's fundamentally what he's saying is, we need to upgrade and then we want to have outages. In fact, it is the upgrading that is creating the outages. So there's a logical problem there. But you want to be specific about what happens inside of your accuracy. So the bigger the number that you are given to contract out, the more contracting and procurement and compliance people get involved, the more complex the contracts become, the more you have to sort of specify all of these different things. And the more you lock us in to what, you know, what's available right now and ways of working now. So it's and a lot of these contracts take for a really long time. So but the time that the contractor is working on them. They're working off requirements that were often written four, 10, 15 years ago, in some cases. Well, guess what? The technology world has moved on, and that's actually not the way to solve the problem anymore. But I think the biggest reason that these things go south when you give them more money is this balance of power between the kinds of expertise that you want to apply to the problem. You're basically giving more power to folks that are going to work, and these aren't bad people. They're good people who are trying very hard to do the right thing. But you're basically saying, let's put more contracting and compliance expertise on this problem, and we will drown out the voices of the people with a technical understanding to know what should have happened that day in Arkansas that didn't happen that caused the outage. Here's Michael Boyd, an aviation consultant writing today in the New York Times, quote, "Just this month, the FAA proudly announced that air traffic control will be free of floppy disks by 2028, memo to the agency. It's not the floppies that are the problem. It's the system they are plugged into. Over the past 40 years, I estimate tens of billions have already been spent on the same organizational and functional approach that has given us the rickety system that collapsed last week. The agency has a track record similar to that of a used car lot. There's been a parade of grand air traffic control upgrade schemes over the decades with flashy names like fans and stars. The latest FAA bromide is something called B-NATCS or brand new air traffic control system. We saw the fruits of these changes when, according to officials, a simple router upgrade somewhere in Arkansas cascaded into a national ground stop." End quote. First, it seems like he is preaching from a similar hymnol as you. And I would like some reflection on the degree to which this is essentially the point that you're making. There's another question that I think this raises, which is, is the problem that government technology is bad? Or is the problem that government technology expertise exists outside of government? So when it has to be upgraded, as technology always has to be upgraded, the patch is always rickety, because the expertise never lives inside of government itself. Yes, he's right. Yes, that is correct. Let me add to Michael's analysis here with another example, I worked on unemployment insurance backlogs during the pandemic. Everybody blamed the cobalt system. That's the equivalent of his upset about ever in blaming the floppies. It's not the cobalt, right? It's in the case of unemployment insurance. It's the 7,119 pages of regulations that govern it, that make it a very, very hard system to get right. It's just always going to be incredibly complex and therefore pretty fragile. These things don't scale well. But the thing you should know about unemployment insurance is that about half of the states had, quote, unquote, modernized before COVID hit. On average, those states did no better than the states that hadn't. So the problem-- everyone thinks that what we need to do is modernize government technology. The way that I put it, which I think is resonant with Michael's statement, is we need to modernize how we modernize. We keep trying to throw more money into a system that does not deliver results. It's not a machine for our current way of building and buying technology and government is not a machine for delivering results. It's a machine for spending a lot of money and creating a lot of mess that basically often sets us back, because then people have less faith in it. There's a lot to that. I've spent 15 years doing this. My focus for 10 years was on the technology side of it. And I kind of came to the conclusion that you can't solve government technology at the level of the technology. It's the operating model that government runs on that dictates why we spend our money this way. It's a very heavy waterfall system where you do a lot of front planning. And then after that planning, the goal is to stay incredibly faithful to the plan and execute it exactly as it was laid out. That's a system that worked really well when we built this sort of new administrative state coming out of the Great Depression and to Fight World War II. It is not a system that gets us results in this day and age. And so the way to get the right technology and government actually is to go upstream and fix what's going on in government more broadly and shift it to an operating system that fundamentally sort of tests and learns its way to success. It's all kind of fractal. If you do that at the level of staffing and budgeting and oversight and all of these things, that will cascade down so that you'll have a test and learn way of making technology that works. We do have to work upstream from where we've been. If the people in charge of unemployment insurance technology or air traffic control systems were first and foremost technical domain experts in government with support from some procurement staff potentially, what would be different? Why would that be the biggest part of the fix here? Well, you can see this. I mean, we have natural experiments, right? Like when you bring people in house on things like the FAFSA, it gets better. And by the way, it also gets cheaper. That's not necessarily the reason to do it. And the reason it gets better is because you've got people who you can just walk over to in the office and say, let's try this thing. OK, we can learn that it doesn't work or it doesn't work. It doesn't work. We roll it back. If it does work, we scale it out. And that's how things get better. And everything you use today got better because people did that, essentially. When you contract everything out-- and I'm not saying we can't contract stuff out-- but when you contract stuff out through this very heavy, waterfall process where you've defined everything up front, you will not learn that something didn't work until potentially five to 10 years after it was spec'd out. That just means that stuff is going to not work. Can I break? The role of AI is interesting here, because I think AI can be a tool to help patch broken systems. But it can also attack and break systems. And then on top of that, it can sometimes create narratives about why systems break. Like when the events the last week happened, everywhere I looked, people were saying, oh, this is because AI. This is because you're being attacked. This is the cyber apocalypse. As someone who's obsessed with getting government technology to actually do its job, how do you think about the role that AI can play here? Well, if you mean specifically in something like the FAA, the ground stop, I don't know it well enough, but I will say that a lot of what's wrong with government technology is that we don't test it very well. The waterfall method puts all the testing at the end, and that's true for software, but it's also true for hardware. And AI can actually be-- I'm guessing that AI could have been quite helpful in helping to run tests and simulations to see what might have broken and to help people. I am not a networking engineer. I'm not going to tell you what exactly they could have done differently. But a genetic coding to me is an incredibly exciting development. And this is why it speeds stuff up, right? Like it used to be that you had to go to a big contractor and do this whole process, and you're stuck in that way that just takes forever to get any working software. It's very hard to test, et cetera. You're waiting-- you basically wait for a really long time for a vendor to deliver you something and then you try to test it before it goes live instead of testing it all the way through. You look at a place like the Department of Education that is brought in-- I think it's 200 new technologists. They are still using their vendor. And with a genetic coding, they can just do stuff way faster. There are states that are doing this, too. They're doing it in the IRS. So you can try something out at a speed that is completely unprecedented. And that can really break the dysfunctional vendor relationship that we've been stuck in. Give you an example in another area. HR1, one big beautiful bill act. One of its requirements was that states now check on work requirements before they give people benefits, like Medicare and SNAP benefits, Medicaid and SNAP benefits. Prior to the advent of agenteic coding, kind of your only option, because of the way that we have set these things up, is every state has to go to their vendor. And it's been approximately $50 million per state to these vendors to add that module so that the states can comply with that requirement in HR1. Now, really, if you were just like a middle manager in a state government and their human services, you could, in theory, spin up, quad codes, codex or something and run it and say, "How should I create this new module? What would be the cheapest and easiest way to do it? What would be the best way to do it that would put the least burden on the people that are going to have to do this, proving that they're looking for work?" It really changes the power dynamic between government and the vendors in a way that I think could lead to a fundamentally different ecosystem that gets a lot better value for the public and also helps government meet its policy goals. When I put all this together, it seems like a big story here is that the government has bureaucrats who don't deeply understand technology. They spend billions and billions, gazillions of dollars procuring technology to provide essential services. Over time, those systems age, they break, they don't work perfectly, they have to be upgraded modernized, so to speak. But because we've procured out all of the expertise that people in government don't know how exactly to run that operation, to run that modernization process, to fix the bugs that have accumulated, the flaws that have accumulated. It seems like a part of the problem here. If you go all the way upstream, is that government doesn't have the right kind of people in the first place. How do we change that? How do we create a government where the people in charge of processes like air traffic control or unemployment insurance or student loan programs have technical expertise and are empowered to build technology that is owned by government, rather than overseeing by short-term bureaucrats whose predecessors 15 years ago procured something that the people currently empower, barely understand. Yeah, I do want to start by saying we do have a lot of great tech talent in government. Very often, it's sort of a power struggle between them and others. I talk about the six-person tech teams surrounded by the 60-person compliance team. That is changing. Your question was how do we change it? I think it's going to be hard. It is actually the key focus of my work now at an organization called Recoding America, where we think that government needs to change its operating model fundamentally. The first piece of that is we need the right people in the right roles. Often, people think that's about things like the hiring processes, which are not great. I agree. But it is fundamentally the question that you asked, like, who do we hire and what kind of skills and expertise do we put on a problem that government needs to solve? I would point to two things. One, fundamentally, we actually are going to have to completely rewrite our civil service statutes. At the federal level, this is all rolled up into something called Title 5. The last real revision of it was 1947, I believe. People will talk about the civil service act of 79, but it really just nibbled around the edges. We created the civil service system that runs our federal government very much in the industrial era. It is not fit for the purpose that we need it to serve. And we need a powerful bipartisan political coalition pushing for fundamental reforms. This is not been on anyone's dance card. It is getting to be on the dance card. The organizations that I work with are pushing it onto the dance card. And there are two senators that are interested in this. There's people, members of the house interested in it. And at the state level, we've already had one state go do it. That was North Carolina has already rewritten its civil service statute. It passed unanimously. A red legislature signed into law by a blue governor, not a single member of the legislature voted against it. So there's real momentum here for that. And that I think is how you get out of the classification system and the sort of rut that we're in of these are the kinds of jobs that we apply to the problems the government has and into a world of, hey, we live in a very fast paced AI world. What is the job we need to hire for that will consistently get us the outcomes that the American public is asking for? And then the second piece of it is just this like belief that came around sort of in the 80s, some started before then, of government can't do anything. So let's outsource everything. It has not worked out well. I think we've sort of covered that here. And I think we have a lot to do to sort of just bust open the myth that the private sector does it all better. I still want to see the private sector involved. This is not about not ever having vendors to government. It's about a real collaboration between vendors and government that is focused on outcomes that we need to deliver not on dreams and dreams and dreams of paperwork that we know how to develop and check those boxes. But Jen, this is why I think it's your moment because it's just interesting that you have things like the FAA ground stop or the NSA news and people are saying, you know, this is AI. This is advanced AI. This is super intelligence for under attack. This is the cyber apocalypse that's been long predicted. And what you're saying is like, no, this is actually about civil service reform. This is about the government's century old approach to building an upgrading technology. And what broke in Arkansas or whatever broke with the NSA's global communication system. This is of a piece with something I, Jen, have been saying for 15 years, which is the government's approach to technology, is not fit for the 21st century. And maybe it's even more ill fit for a world of AI. But the problems here are much older, 100 years older than large language models. So I think it's a really important message to end on. Jen for Polka, thank you. Thank you very much.

Podcast Summary

Key Points:

  1. A wave of recent outages across critical systems—FAA, banks, NSA, airlines—was largely driven by defensive AI patching rather than direct AI attacks.
  2. AI tools are rapidly identifying and exploiting software vulnerabilities, exposing a long-standing "digital deficit" in legacy, unpatched systems.
  3. The rush to deploy patches in response to AI-discovered flaws often breaks systems due to poor testing, complex environments, and lack of quality control.
  4. Offensive AI attacks are already active, with tools like RTEX enabling autonomous, widespread exploitation by cybercriminals and state actors.
  5. Incident response infrastructure is overwhelmed, with firms like CrowdStrike and law firms facing backlogs due to the volume and speed of attacks.
  6. Public sector systems, especially government-run ones, suffer from outsourced expertise, poor governance, and outdated procurement models that fail to adapt to modern technology.
  7. The perception of a "cyber apocalypse" is amplified by media attention, creating a false impression of unprecedented risk when outages are actually a symptom of systemic flaws.
  8. Long-term solutions require not just more money, but a fundamental shift in how government manages technology, prioritizing internal expertise and resilient design over costly, rigid contracts.

Summary:

A recent surge in critical infrastructure outages—such as FAA ground stops, banking disruptions, and NSA system failures—has been primarily driven by the defensive response to AI-powered vulnerability detection, not direct cyberattacks. As AI models rapidly identify flaws in legacy software, companies and government agencies are rushing to patch systems, often without adequate testing, leading to cascading failures. This highlights a deep "digital deficit" where decades of unpatched, poorly maintained code have finally come due.

While AI is also being used offensively in autonomous attacks through tools like RTEX, the immediate crisis stems from the speed of patching and the fragility of outdated systems. Government technology, particularly in public services like air traffic control and unemployment systems, suffers from a lack of internal expertise, rigid procurement models, and outsourced management that stifles innovation and resilience. The current wave of failures is not a sign of an imminent AI apocalypse, but a symptom of systemic neglect in software quality and governance.

Media coverage amplifies the crisis, creating a perception of chaos that may not reflect the underlying reality. Experts argue that lasting solutions require a fundamental rethinking of how technology is developed and managed—shifting from costly, bureaucratic contracts to agile, expert-led, and resilient designs. In the short term, the pain from outages will persist, but the long-term path forward lies in addressing the root causes of digital fragility, ensuring that critical systems are not just patched, but fundamentally restructured.

FAQs

The recent wave of outages is largely due to rapid defensive patching driven by AI's ability to detect and expose software vulnerabilities. As companies rush to fix these issues, the patches often fail due to poor testing and system incompatibilities, causing cascading failures—similar to an 'own goal' in soccer.

While AI is being used by attackers to find and exploit vulnerabilities, the most recent outages are primarily a result of defensive actions. AI tools are rapidly identifying flaws, prompting urgent updates that break systems due to inadequate testing and outdated infrastructure.

RTEX is an AI-powered tool that automates the discovery and exploitation of software vulnerabilities. It allows even non-technical users to launch attacks at machine speed, lowering the barrier to entry for nation-states and cybercriminals, significantly increasing the volume and speed of attacks.

Yes, the perception of a spike in outages may be amplified by heightened media focus on AI and cyber threats. Historically, similar events like the 'summer of sharks' saw increased coverage without a real increase in incidents—suggesting that the current situation may reflect attention bias rather than a true rise in system failures.

These systems are vulnerable due to poor cybersecurity hygiene, outdated infrastructure, and reliance on underfunded, remote contractors. They are often poorly monitored and lack robust defenses, making them prime targets for attackers seeking to cause widespread disruption.

Government systems are often outsourced and managed by contractors, leading to a lack of internal expertise and accountability. This creates systemic fragility, as technical decisions are made by non-technical procurement teams, not by those with deep technical knowledge of system design and resilience.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.