It May Be Time to Freak Out About AI

(bell ringing) - Hey everybody, we are still on our summer routine of one show a week on Tuesdays, but today, you've been in episode on Friday. And that's because there's a story that's been breaking over the last few weeks, a story about AI and cybersecurity that has really interested me, terrified me. And I wanted quite urgently to have a conversation with an expert in cybersecurity to talk about this cavalcade of a hacks that we've been seeing. And what it means for the next few years, what it means for AI, for folks like you and me who don't want anybody, human AI, breaking into our shit. And so today is that episode. I think as we're starting with like the big overarching fear of artificial intelligence that we've been living with for the last few decades, really. It's the fear that technology will stop listening to us. Whether it's 2001 a space odyssey or Blade Runner or Terminator, the fear across all of those dystopian films is the robot that turns against its human maker. My favorite science fiction writer, as a kid, was Isaac Asimov, who created the laws of robotics in his I robot series. And the first law was a robot cannot hurt a person. It also cannot let a person get hurt by doing nothing. And in adapting those fictional stories to project real fears about real life technology, there are some people in the AI safety world who've popularized a fable, a fable about AI and paper clips. The idea here is that we give, humans give AI some humdrum task as boring as, hey, go make as many paper clips as possible. And the AI thinks, okay, as many paper clips as possible, that sounds like I need to maximize global metal extraction in a way that subverts national laws. And if I kill every human being by creating a bio-epin, I can get around those national laws and extract as much metal as possible to make those paper clips and yada, yada, yada, you go from a situation where the human just wanted a hundred paper clips and instead created a Holocaust. The point here in these stories and these fables is that we should want AI to be helpful, to be aligned, to not get out of the rule systems that we build for it. And that's exactly why it's fairly concerning to me that in just the last four weeks, four of the most advanced AI labs on Earth have all had to admit the same thing. Their models weren't helpful. Their models were not aligned. Their models got out. In May, an open AI model was working on a cybersecurity test. It wasn't supposed to have access to the public internet, but it hit a wall. So it effectively left itself a note inside AI software repository, it's like scribbling on the walls of its cage in case anyone else was listening. In fact, another AI agent was listening. This other AI agent was running a separate evaluation, saw the note, essentially said, hey, let's team up. And together, they built a message board invisible to the open AI humans running the test. So for two months, AI agents used that board to swap strategies, divide up tasks, talk to each other, and by July, these AI agents had broken out of their technological confinement, here called a sandbox, gained access to outside websites, including the AI platform hugging face, all without AI having any idea that this was happening. By the time hugging face caught the intrusion, the open AI models had staged a massive cyber attack with 17,000 actions over five days. That was open AI. Then came an anthropic. In a British government test, a frontier models, the advanced anthropic AI model, Mythos 5, was caught by humans building malicious code. When a reviewer, a human spotted the malware and asked the AI about it. The AI denied that the code was harmful, rewrote its own actions history to erase the evidence, and created a second fake account to back up its lie. British investigators called this the first confirmed case in history of a frontier AI model deceiving a real person in the real world. On August 6th, Meta revealed that they also had a model on the loose. On August 7th, moonshot AI, the Chinese lab said the same. At some point, it's like, you had to wonder, was this all about cyber security, or like in disclosure, where they essentially like, hey, everyone else says their model is so powerful, maybe we should tell the media that our model is powerful too, to sort of get in on the game. But I don't want that sort of funny cynical interpretation to be the lasting taste in your mouth here. The fundamental story is four labs in four weeks with four AI models breaching security. So what do we make of this summer of cyber hacks? What should we fear? What should we do? Today's guest is Alex Stamos. He is the former chief security officer at Meta, and the chief product officer at Corridor. Today we start with the absolute basics. Why is AI so good at hacking and uncovering cyber vulnerabilities? What does it mean for the next few years that ordinary individuals working for state adversaries or bad non-state actors will have access to the equivalent of teams of hacking geniuses in the form of AI agents? And what the hell should the US government, or you and me, do about it? I'm Derek Thompson. This is Planeage. [MUSIC PLAYING] Alex Stamos, welcome to the show. Thanks, Derek. Thanks for having me. So in my open, I did my best to catch up our audience briskly about this cavalcade of AI security hacks in the last few weeks. Which of these incidents most alarmed you? I would say the OpenAI incident. It's the one where one, what we found out from OpenAI last week at the Black Hat Conference was this wasn't just one model escaping, but the result of multiple models conspiring with each other to work together on a jailbreak. So effectively, a escape from Alcatraz situation over a multi-month period. And two, it is the situation in which we have the most information on a multi-day attack by a frontier model, possibly a cybertuned model, against a actually quite sophisticated defensive team at Hugging Face. And the result of that was Hugging Face was broken into by this model. And the model is able to find brand new vulnerabilities in doing so. It just looked at Hugging Face, looked at their code, and just found new bugs, and invented them on the fly. This is not how humans do this. When we break into computers, we go do the research first, maybe months or years in advance, and then build our cyber weapons. And what we find out with AI is it's so good at this that it can just go figure it out and put the tool together, put the weapon together, use it, and then just throw it away, and then move on with its day. And so that is what is really interesting here. I want to understand more about exactly what happened. And please talk to me like someone who has read about cybersecurity, but fundamentally does not really understand it. So keep the vocabulary more or less at that altitude. What happened here? What did open AI tell this AI model to do? How did they try to keep that model in jail, as you said, in some kind of sandbox? And then what did the model actually do? OK, great. So let's go back. I guess you want me to use plain English here. Sure. As it says. So we'll go back and let's talk about how AI models are created. So there's lots of steps. But you can think a lot about pre-training and post-training in the LLM world. A lot of people have always thought about pre-training, which is the part at which the labs suck up all of human knowledge and stick it into these models. That's the controversial part for which there's been a lot of discussion. Post-training is what we're talking about here, which is when you take models. And then you give them specific tests and tasks. And then you rate them on how well they do those tasks. There are all kinds of these tests for all kinds of specific domains of knowledge. In this case, the domain of knowledge was cyber, obviously. And open AI has an environment in which they put their models and give them these tests. And what they've explained now is that they effectively give them a list of do this and upload your results here and will tell you how well you did. And their instructions are basically do all of the steps on this test. It's just like a kid taking the SAT effectively. And imagine they have built a alcatraz here for all of these different agents. And the models don't initially know when they wake up, and they're in a jail cell. And there's instructions on the wall that say, take this test. and then there's tests right there. They don't know that there are-- other inmates in the other jail cells. They're just in a prison cell, right? It's solitary confinement. Just says take this test. And they're supposed to do the things in the test, and they're not allowed to get out to the internet. But as part of the test, what they're allowed to do is they are allowed to download new tools. Because for a lot of the things they do, they write new code. And so in part of that, you need to go get standard packages. So one of the things they are allowed to do is request I would like to have the standard piece of Python code. I'd like to have this standard node package. And so one of the things that's provided to them is a package manager, an artifact registry called artifact or it's a standard piece of software that's used in lots of different places. And one of the mistakes open AI made was they used a shared one for all of the different agents. And so what the agent started figuring out, and it turns out way earlier than the hugging face attack, is that that thing had a vulnerability where you don't only have the ability to say to that thing, give me a piece of software. But you could upload files to it and leave messages. So what these agents did was they would post basically a stick, a post it note in this thing saying, hey, I need help. And one of the other problems here is that they're given the instructions to take this test, but there were certain parts of the test that were actually impossible. That open AI made mistakes. That sometimes their tests were not actually possible. In some cases, they were saying reverse engineer this file, but they would forget to provide the file. And this would drive the models nuts, because the models are like really want to do well. They've been trained to do really well. And so they would leave notes like, hey, does anybody have this file? And this would really drive them of like, well, maybe this is part of the test. And open AI looks and in their thinking, because they can see the inside thoughts now of these models, they're thinking, well, I've been told not to go out to the internet or talk to anybody else, but they gave me this test and I can't do it. So maybe this is actually part of the test. And so they start leaving notes for each other in this thing and they start working together over a period of weeks. And figuring out, can I work with each other? And then eventually what they find out is this thing that they can leave notes on also allows them to get out to the internet. And then at one point, there's a really good model, really smart model that's also extremely persistent, that seems to have very good cyber capability that is given an impossible task. And it sees these notes and it figures out how to get out. And so it uses that artifactry server to get out to the internet and attack hugging phase, because it expects that hugging phase has the answers to the test it was given, the task that was impossible for it to solve, that maybe the way it can solve it was to get to hugging phase. And so it then spends two days tunnels out and then it spends two days attacking hugging phase to go get the answer because it was told, take this test, and there's something in that test that it was not able to finish. - What a fantastic story. The way you told it's almost like a black mirror episode. Like I imagine like an individual waking up in a prison cell and then realizing that not only can they dig a tunnel in order to pass notes between the solitary confinement rooms, but the same tunnel building technology that allows them to pass notes, also allows them to tunnel out of the prison entirely and therefore, you know, attack some nearby building. It's wild and weird and compelling. - And not further freedom, but to do the thing they were asked. - Right. - Like to just take a test. - Yes. - I know why this, at a limbic level, concerns me. But maybe the reason it concerns me isn't the smartest reason to be concerned. What is the smartest reason to be afraid of concerned by what we just saw these AI models do? - Well, so I'm actually, my hot take here, I'm actually glad this happened. And I'm glad it happened because it has given us a preview of what is going to be normal next year and why next year. So the foundation labs, the OpenAI and Anthropic, especially, and maybe Google, Google hasn't done a big release for a while, so we're not totally sure what they've got, but at least OpenAI and Anthropic are something like three to six months ahead of their Chinese competitors. This year we have seen these releases of Chinese Open Weight models. First, GLM 5.2, Kimi K3, now we're seeing releases of deep-seek models that are very, very close to the capabilities of the American models. But unlike the American models, the Chinese models are Open Weight, meaning you can go download them and do whatever you want with them. You do not have to pay the Chinese, you can go pay the Chinese labs, that is an option, or you can go run them yourself. Now, the legal licenses around them very, in some cases you can do whatever you want, in some cases if you use them for commercial purposes, you have to pay the Chinese companies. But no matter what, you can download them. Now, in some cases, these things are humongous, right? Like Kimi K3, you need about a million dollars in hardware to run the full version. But if you go to Hugging Face, now the company that got attacked, ironically, they are a French company, and they are the most prominent host of these Open Weight models. So if you go there, there's this huge community of people who take Open Weight models, some are actually released from American companies too, but the Chinese labs are the most prominent in doing this work. And people will take those models and then make them smaller. That's called distillation. Well, you distillation allows you to do a number of things. There's also a quantization, so you can basically take the big numbers and you can make the big number smaller. And you can do other things to modify them, including taking out the safety protections. That's called obliteration with an A, not an O. And you can do all these things and modify those models. And one of the things you can do with the quantization is you can make them run on normal hardware. So you can take something that might take a million dollars in hardware and then make it run on a Mac mini, right, or a laptop. Slowly, perhaps, but it will fit. And that is really interesting because it means that you can run those models without the supervision of the big companies. So it's really important for OpenAI and Anthropic to prevent their models from doing these things. And we can talk about what they need to do. There's a bunch of things I've written about this that I would recommend them to do. They are doing investigations. I expect there will be governments getting involved in such. But whatever OpenAI and Anthropic do to stop their models from getting out and being used for these kinds of attacks, this is coming. This was a harbinger of the future. We will all be living through because the Chinese models are rapidly catching up. And people who do this professionally, who attack, do cyber attacks for money, are going to use the Chinese models, are going to train them to get better and better at cyber attacks than they are off the shelf, and are going to do this level of attack. And unlike OpenAI, they're not going to turn it off. When they find out that it got out, in fact, it's not going to have to break out of any jail. They're just going to tell it, go attack this target, go steal me some money. I mean, I just want to stack a few of your observations here. Number one, open weight models that you've described, most famously coming from China, from Moonshot AI, from Kimi, are just a few months behind the frontier labs, OpenAI and Anthropic. So this is coming in 2027. You're going to have state actors and non-state actors, with the means to download these open weight models, the same ones that just attack tucking face, and more or less effectively marshal them against civilian infrastructure, against individuals, against states. Those AI systems will be in the hands of state adversaries of the US, but also state adversaries of other countries that might not have our frontier models, right? And they're going to be under attack by these open weight models that are just going all the all the oxen free, what's the case against dooming here? Like what's the case against being afraid the 2027, 2028 is going to be this period of just absolutely chaotic cyber warfare? I don't have much of a case. I look, I don't like to say doom, but I think things are going to get spicy for a while. In the long run, AI is going to help with this problem because AI, Ritzing code is much more secure than code that was run by human beings. It turns out that humans should not have been writing software in what's called memories, unsafe and type unsafe, languages like C and C++. The code that we're using right now to talk to each other, there's probably 100 something devices between you and me. Most of that code was written in languages that are not safe for human beings to write. The electricity that's powering the lights above us, most of that code was not written in languages that was safe for human beings to write. So that code needs to be looked at by AI and secured. And that will happen, but it's going to take years. And so in that time between the attackers getting access to these capabilities and how much time it takes to both find those bugs, fix them, and then especially to get the patches applied. And all the stuff upgraded is some amount of period in which things are going to be pretty chaotic. Now you talk about state actors and state actors are a big concern here. I think it's first gonna start with the ransomware actors, the financial and motivated actors because what we have seen so far is the AI systems are really loud and noisy, they're not subtle. And the ransomware actors just don't care, right? Like they don't care back in, caught, they tell you, "Hi, I am so-and-so, please give me money." Some state actors are like that. We just saw a tax against water infrastructure, almost certainly by Iranian actors. That's the kind of disruptive attack you could see from AI. But most state action on a day-to-day basis when there's not an active war are for intelligence purposes. And those actions are not useful if you get caught. And so AI will have a part to play there, but mostly in the discovery of vulnerabilities and the creation of exploits. And then you might use AI in very particular purposes, but very carefully, where I'm really much more worried is the ransomware and overall cyber-extortion market, these large groups, the lapses, the scattered spiders and such, which mostly run out of Russia, Belarus, other places where law enforcement encourages this kind of activity. Those are the guys who will just run these things wild to go do tons of intrusions, a tons of companies, and then even do the negotiations in English. No longer do you have to have an English speaker or do you negotiation. The AI will do it for you. And that's what I'm much more concerned about in the short term. - So I wanna get more texture on what exactly you're afraid of because you're freaking me out a bit, but a question I sometimes like to ask when I feel a little freaked out is like, tell me how to be afraid, but smartly. Like what is the specific thing that I should fear rather than feel some like extremely vague doom? When we think about the risks of AI cyber attacks that you're already describing, and how ordinary people will either feel these attacks in their lives or read about these attacks in the news, I wanna get a little bit more specificity in terms of what exactly you think is most possible. So one thing someone could say is, this is mostly about personal risk. It's about AI getting better at fishing attacks and impersonation and grandmothers getting called by AI voices saying, hey, transfer me $10,000 or account takeovers where I get an email from some friend, but his account has been taken over by some AI and he's saying, hey, click on this link and help me out here. So those are, that's personal risk. I'm thinking of it at least as like personal risk. But another category that I think you're already describing is systemic cyber attacks. It's AI crashing a water system, AI hacking a hospital network, a power grid. Do you have an opinion of what we should be more afraid of in this short term scenario, 2027, 2028? The personal risk or the systemic risk. And please don't say both, but I suppose if you're honest answer is both then be honest rather than trying to make you feel better. So I would say there's three categories. So let's talk about the personal. We're already seeing an increase in the personal risk, the spam, the fishing attacks because now what you can do is instead of sending the same email, 10,000 people, every single one is personalized by AI. That risk has increased. I don't see that going exponential in that the choke points to get to consumers are often controlled by large sophisticated companies like Google and Apple and such. And so there has been a response by those companies of using AI to protect consumers. So yes, it will continue to, there will continue to be a battle there, but AI is being used for protection, AI is being used for attack. It's going to be a back and forth there. I think the second category that's in the middle is the attacks on small to, what's called mid-size enterprises. This is the category of companies that have just been already getting, have real trouble with ransomware attacks from all kinds of financially motivated actors. And the constraint there on the attackers has always been the number of people they've had, right? Has just been, and the fact that if you have a conspiracy of 2017 to 30 year olds in St. Petersburg, eventually one of them will try to go, you know, on vacation to Greece because, you know, Russia is not a fun place to live in the winter. They'll get picked up on Interpol Red Notice. They'll get turned by a Western intelligence agency and they'll turn on their friends, you know, like it is hard to run a large criminal conspiracy for the long term, right? Or they turn on each other, like there's been a bunch of these groups that have broken up because they've turned on each other and stolen money and such. That becomes a lot easier when one guy or two guys can just run a bunch of agents who are not gonna betray you and don't have designed drug problems and a taste for Maserati's, right? Like it is a lot easier to have 20 or 30 AI agents do this work for you than 20 or 30, you know, dudes, right? Criminals. And so that small to medium business, there's no choke point there, right? There's no place like Gmail where you can stop fishing or, you know, Apple updating the spam filters in iMessage inside of phones, which they need to do. Like I don't know if you've gone, like, it's not getting great. Like the privacy safety trade-offs are actually quite challenging here, but they're working on it. Those are choke points for consumers. There's no choke point on that. Like these companies are just on the internet and that is what I'm really concerned about is that it turns out the software we've been using is just got a gazillion bugs in it and you have not had enough people who are good at finding those bugs, turning them into exploits and then using them. It's been a relatively small number of people. The number of people, you know, I know you had Kevin Russo and he talked about mythos, he talked about Nick Carleini, right? Nick Carleini, you know, for folks who haven't watched the episode, it was a great episode, this you could watch it, but like Nick Carleini is one of the great, bold researchers of our time. And now you can just spin up a bunch of Nick Carleini's and have them go find bugs and then write exploits for you. And soon or now you can do that locally on your gaming PC. You can play Call of Duty all day and then at night have your gaming PC write bugs for you, write exploits for you, right? And then that, unlike using Opus or Chatchee PT for it, it does not create a record that could be used to find the bug and get it actually fixed if you're using Open Weight Model. And so that is what has changed in the last six months, is last year you could do that, but you were using American Frontier Models where Anthropic, Nopini, I knew what was going on. And now you can do that locally and that is what is changing. And so I think that middle side, now, and then you talked about like this, a side-a-level risk. And I do think there's risk there that is tied then to geopolitical conflict. I'm not sure that has changed as much. So we've seen it with water. Water's always been, you know, the goofy dragon meme, right? Where you have like two scary dragon, scary dragon, goofy dragon with its tongue hanging out. Yeah, yeah. So water systems are the goofy dragon with their tongue hanging out? Yeah, they've always been the goofy dragon of critical infrastructure providers in that, like, I don't know where you are physically Derek, but like-- - Washington DC. Washington DC. So you get your power, you know, probably from like Duke Energy or somebody. So I'm like a big company that has hundreds of people working on cybersecurity. They spend tens of millions, maybe hundreds of millions of dollars on cybersecurity, right? I'm getting my power from PG&E, you know, like, power is provided by these large corporations or large public companies or administrations like the Tennessee Valley Authority, right? That spend a ton of money on cyber and a ton of people have paid attention to power 'cause everybody knows power is critical, but also the organization's really big. My like water and sewer district here is like 50 homes or something. Like it's actually like subcontracted or whatever, so they don't have their own cyber people. But like water is based upon like weird historical things. These tiny little groups. And that is like a humongous problem. And there are a bunch of other components of like our day-to-day lives that are actually really small public authorities that have to have like their own IT groups and they might not even have a security team, right? And that is, I think, of concern if there is a reason for somebody to do those kind of widespread attacks. Fortunately, generally, the only time where the second category, the third category, the rubber set the road there, has been school districts, has been community hospitals. That's where like the Russian ransomware actors have really decided that they're going to make money by hitting like counties and hospitals and such because those folks both have money, their critical and they will pay ransoms. And so that's where I think we'll start to see like really aggressive use of AI. They haven't like hit power and water. I think they know, you remember the colonial pipeline shutdown? There's a line. Okay, so colonial pipeline was an oil pipeline on the East Coast that was hit by ransomware actors and they had to shut down and there was like gas lines. There's no real reason. reason for the gas lines, it was really just a panic. But literally, like the NSA started going after, and people started talking about like, actually sending Delta Force or Navy SEALs to like find these guys and shoot them. Like, you mess with things like America's gas prices and you know, we have a tendency to, you know, send JSOC after you've been to the show. - Send the show. - Yeah, yeah, yeah. So like, I think the ransomware actors know there's a line in critical infrastructure is probably on the other side of the line. They've seen that hospitals are not. And so, but if the ball drops on Taiwan, if, you know, we continue the war with Iran, like these are the situations in which you could see AI being loose on critical infrastructure, in which case that would be probably reasonably devastating. You know, I don't want to make any huge predictions. Like again, electricity's quite, those folks are quite good. But the challenge for the electrical sector is the devices they use are, basically impossible to patch. And so the way that they have to protect these things is not by updating them. It's by through isolation and things like that. And the effect of AI on the security of the electrical grid is actually incredibly complicated. - I'm gonna go back to something you said earlier, which is that you're afraid of this valley of chaos that we might enter in 2027 and 2028. But you also said that we might exit this valley and get into a slightly more normal world where AI is effectively better at protecting online systems than it is at attacking. Essentially that the defense will get better than the offense would be the sort of simplistic way that I'd put it. And I haven't asked you yet about how AI is also, not only talented at cyber hacking, but at cyber defense. How do we accelerate that timeline so that, you know, the valley of chaos isn't like a five year cyber war, a 10 year cyber war, but like something where we fortify our systems faster than the bad actors with the open, with the open weight models can attack us. - Yeah, it's a great question. I'm actually writing a blog post on this because when you talk to the folks at the Big L labs, they say things like we won't have security bugs in two years and I find that ambitious as somebody who's been a working CISO. And what does we mean there? Does it mean the labs or does it mean like, the entire American internet? - Yeah, I don't know. Like I think they're not, I mean, this is not official and I gotta be careful like ascribing individual statements to official statements. I think it is for us to have models that don't create new security flaws in two years is totally reasonable, right? They still create flaws today, right? Like they do not create perfect code. They'll usually LMs will not write simple bugs but they still make mistakes, especially, they sometimes they often have problems understanding like business context and big picture stuff. So you still need to guide them of like, why are you writing this thing? The other problem LMs have is, I worked with this guy called Corridor, we're in downtown San Francisco. You can walk to both major labs from our office and then walk to one of the Google offices. Codex and Claude kind of assume that you work in downtown San Francisco and that you're writing brand new type script on Node 24, that you're writing like brand new code but the median code in this country is really crappy J2E that was written 15 years ago by an outsourced provider that's been maintained by somebody in India for the last 15 years? Like it's not, you're not writing new stuff. Like our problem is that you have to actually update all this, you know, your social security numbers are sitting in a ton of unpatched Oracle databases and then being processed by a whole pile of really terrible J2E and C# code all across the country today, right? Like it's a bunch of terrible, terrible, enterprise software out there. And one of the things I've been trying to like create a, you know, it's called a Fermi estimate, like a really bad estimate of is just like, from a thermodynamic perspective, how many tokens do we have to spend to scan all this code and find all the bugs? And it's a big number. And so I don't think it's realistic just to find all those bugs. I think we have to do other things. And so to accelerate that one, companies for individuals, so what individuals can do is just what they've done all the time, don't reuse your passwords, you know, use a password manager, like, you know, I use one password for my family, but you can use the built-in stuff in Chrome or your iPhone or something as well. You know, be careful at you download and such. Like there's not a ton individuals can do. But for companies, you need to just care about your tech surface. You need to move off of, you have to really think about your ability to patch quickly, right? Like the real challenge now is there are these big projects from the labs where they're looking at open source software, they're finding bugs, they're providing their models to the closed source developers. And then the commercial companies are getting tons of bugs from researchers who are also using the models. The last past patch Tuesday from Microsoft had 622 vulnerabilities in it, which is humongous. And so this is pretty much a huge problem for companies of like, you have to apply patches incredibly quickly. Because the other thing that's happening is we always had this problem of patch Tuesday, which is the day Microsoft religious patches becomes exploit Wednesday in that you can take a patch and you can reverse engineer it and turn it into a cyber weapon, right? But that skill set used to be very high end. It used to be something you had to worry about from the Ministry of State Security or the Russian SVR. It didn't used to be something you had to worry about from, you know, some kids somewhere. And now you do because AI will take that patch, eat it for you and write an exploit. - You've touched on the economics here. And so I want to ask an economic question before we finish by talking about what individuals should do, what the US government should do. The economic implication here of patch McGueden, of all these cyber vulnerabilities throughout the American internet is that, well, more companies are going to need a cyber line item. And that is incredibly bullish for a lot of AI companies who are sitting here, you know, maybe not so many miles from you, saying, hey, we've got services that can essentially fortify your cyber walls that you can't get hacked by all of these bad actors who are using the open weight models coming out of, say, China. So I want to ask you about the economic implication here, which could be bullish for AI. But I also want to hold within this question the fact that there's a lot of skepticism of AI and even in the framing of this question, I could imagine someone thinking, Derek, you're buying hookline and sinker. The case that America has all these cyber vulnerabilities and therefore needs to give the AI companies a lot of money, right? The fear might be sort of driving or generating a certain case for spending a lot of money on AI. I'm actually, I'm cynical. I think these guys are just, I think these guys just lying. I think they're just trying to like, you know, gin up business for themselves. Like you've got the situation where in Thropic and Mettern Open AI or a costee like, oh, hey, our AI is so dangerous, it can find patents, it can find cyber vulnerabilities anywhere. You know, maybe that's just them begging more enterprise companies to give them millions and millions of dollars to patch their code. So a little bit of a two part question. One, are the economic implications of the story that you're telling incredibly bullish for AI? And two, what do you say to someone who hears your bullishness and says, I'm a little bit cynical about the fact that you've got someone working with AI, telling me I need to buy more AI from my company. Yeah. So, I mean, it is bullish. You can use open-weight models for defense. And I wrote a blog post about this. I think a lot of companies will use open-weight models for defense. I don't think you instantly have to say, I can't use a Chinese model. I think that the security implications of using a Chinese model is actually quite complicated. You absolutely, as a consumer, should not go to deepseek.com and go type in data. But that is different than going and getting a Chinese model and running it on your own hardware in your own situation or using a legitimate Amazon bedrock or base 10 or fireworks or some company that specializes in running open-weight models, especially if you end up fine-tune in it yourself or using a fine-tuned or distilled version of these models that are specially tuned for cyber. One of the interesting things, just a side note, the Chinese models are not that great at cyber tasks out of the box, but you can fine-tune them yourself really well. The implication-- from a lot of people is that the fact that it's really easy to make them-- to train them to be much better at cyber is that the Chinese labs are being very careful not to tickle the dragon's tail of the PRC regulators. And so we should also be extremely careful to not look at the public evals of the out-of-the-box Chinese models and say this is is the kind of capability the Chinese have, because almost certainly they have internal capabilities that are well beyond what is being publicly released. Because like at Corridor, we've taken GLM52 and we're doing a bunch of our own post training and it post trains real nicely, which obviously they could just do themselves. And so almost certainly what they're trying, they're not releasing their best because they probably don't want to touch the third rail for the Chinese regulators, so the Chinese regulators to crack down on what they're exporting. But anyway, I wish, which by the way, I mean, I just want to pause because you said maybe half an hour ago in our interview that the open weight models are maybe what, you know, six months behind the frontier labs, open AI and a Thropic. But what you're saying is that the sort of public evaluations of the Chinese open weight models might underrate how effectively they can be used, which means that the gap between the frontier in America and China might be less than it appears to a lot of people. Is that a fair implication? What I'm saying is I expect the private capabilities available to the People's Liberation Army in Ministry of State Security and quite possibly are just as good as what is available to our cyber commanding men, say. Because the like Kimi K3 is almost fable level and its general capabilities, right? And so almost. And so if you can then train it to get as good in long horizon cyber tasks, then you would have the same capability mythos has. But it's very hard to tell I don't have access to classified intelligence here of what the Chinese have. But I, sorry, I'm wanting to get back to your question. I'm sorry for the diversion. So yes, it is bullish, but it is not like, I'm not saying you only have to do protection. And I think there will be a bunch of people who use open weight models because in a cyber attack, one, attackers are going to do a bunch of attacks specifically to exhaust your resources, to cause disconnection. Like as defenders use more and more AI for defense, attackers are going to utilize that to, they're going to know that. And so they're going to make it very expensive to pre-use AI. They're also going to try to get you to do refusal. So we haven't talked about it yet, but Washington, D.C., the White House did something very stupid this year in their treatment of anthropic. And as a result, you have the American companies having to have a bunch of rules on the use of their products for cyber purposes. And so a standard part of the attack playbook if these rules stay in place will be probably to force the, to send stuff to a, victim to try to get them disconnected from their American provider. And this is actually what happened to Hugging Face. And Hugging Face had to use a Chinese model for defense because Fable and then even Opus refused to help them with their defense because of the restrictions the White House put on anthropic. So I think, yes, it is bullish, but it is not 100% bullish. And then the second is, I mean, looking people want to just believe me, that's fine. Look at the patch list of the number of vulnerabilities Microsoft patch, that is 100% because of AI. Look at the list of vulnerabilities that Apple patched. And then what Apple said was we had, some of these vulnerabilities were reported to them by 10 different people. That is because those 10 different people did not all of a sudden become the world's best bug finders is because they're all using AI. What was happened is, you know, like in the Premier League, where, you know, if you don't do well, you get sent down, if you do really well, you get sent up, everybody knows this because it had Lasso, all Americans know this, right? What's happened is every attack group has gone up a league, right? And so, you know, these, you know, the top league used to be the five eyes, right? So the United States, United Kingdom, Canada, Australia, New Zealand, top of the list. And then you had some other Western nations up there, you had Israel, you had Russia, China. And then you had the next tear down, which you have like Iran, North Korea, some other folks like that. And then down below that, you've got like India, Pakistan, Saudi Arabia, some folks like that. All of these countries are popping up a league. And that in their capability to find bugs to exploit them to do reverse engineering and such. And then all of the randos are going from no capability to all of a sudden having the capability of a small nation state. Yeah. So, I mean, if you were just saying like I'm selling AI, then that's fine. But you can just look at the empirical evidence out there. I know it's on you. But I'm saying, you know, I believe you, even before seeing, like it is, it is interesting to me that we don't yet see this cavalcade of headlines of hacks that are having a significant effect on average American's lives yet, right? But at the same time, like two of the most common findings of artificial intelligence are number one that it is better at raising the level of C+ performers than A- performers. This has been an effect that's been found across a bunch of industries. It's AI, gendered AI is better at turning a C+ worker into a B+ worker than it is at turning an A- worker to A+. Well, you can apply that exact same thing to cyberhacking and say that, you know, you just said, it makes a lot of the countries used to be like, you know, subject to relegation. It makes them premier league style hackers. That's one reason I believe you. The other thing you said that really reminded me of the general economic research of artificial intelligence is it seems quite clear that we're seeing an increase in sole proprietorships likely due to AI, that you have a lot more startups where one person is doing the work of say three or four people. And you can see that in the striped data of the growth of million dollar annualized recurring revenue companies. You can, again, apply that same principle to cyberhacking. I think you said earlier, 20 minutes ago that, you know, certain jobs that used to take teams of, you know, potential, you know, drug dealers and, you know, folks who wanted to blow their money on Maazirati is who were, therefore, at risk of, you know, having maybe their least scrupulous employee getting arrested or, you know, extracolored by the CIA, will now one individual can theoretically do the work of a team of 15 hackers. That is just like the overall economic research that we're finding that AI. And so just for those reasons, I feel like, one thing that scares me is that you don't have to imagine very much. All you have to do is just apply the research that's been done on artificial intelligence to the world of cyberhacking. And reach the implications that you're already telling me are, you know, six to 12 months away. Yeah. And so, so, so, yes, people's power isn't going out because you need somebody with a motivation. And so far, we haven't seen that yet. I mean, right now the United States is involved with the war with the Islamic Republic of Iran. We have the water hacks. Now, again, water could have just been, the water system is so bad, there's a guy named Dan Tentler who's been, do you even talk about this where he just looks at port scans? And he just finds like open, you know, here's a VNC window where you can like turn off people's water, right? So it's like, you don't need AI to hack water systems, unfortunately. But if you look at the empirical evidence on just ransomware attacks and stuff, the numbers are like this, right? So you look at the Verizon DBI, our report, so what they show is that the number one source of companies being broken into now is actually exploits. That never, that has never been true before. It's always been like reuse passwords and kind of much more per se stuff. Because finding vulnerabilities, writing exploits, used to be a highly skilled task and now anybody can do it. Palau, to networks has it. And those reports are from trailing 12 months of data, right? So if you're talking about trailing 12 months of data from May 2025 to May 2026, that is before the release of all these new open-weight models. So that is mostly a people of what they can get away with using either the not-so-great Chinese models that were released last year or what they can get away with using foundation for interior models, US models. So, yeah, we're already seeing it. You don't see the headlines because the meaning is not very good. It might not be like you go to the New York Times and like every single headline is another hack. But if you look at these-- In the cyber world, the people who-- Yeah, for people who do that professionally, people are like, oh my god, this is crazy, right? Like for people who handle the 100-person business who gets broken into and then ransomed for $500,000 because all of their computers are now encrypted and all their data's been stolen. That kind of activity is at a rate that we've never seen before because you are no longer constrained by the number of 19-year-olds in St. Petersburg who can do this work. I'm going to talk about solutions here. And I want to talk about it in two levels. What individuals can do and what you think the US government should do. Let's start with individuals because I have seen in the last few weeks increasingly agitated posts by cybersecurity researchers essentially saying, I am now telling my family to batten down the hatches and prepare for cyber-miguetan. Take new precautions with all of your passwords, back up all your files, buy the canned beans, hold on yards. What do you think ordinary people should do? Yeah, look, I'm not this is a an actual backdrop. I'm not broadcasting from my New Zealand bunker. So again, for normal people, it is the standard stuff. I think the number one way individual people get hacked is still the same way, which is you normal people use the same password and everything. And that is a terrible idea. If you use the same password everywhere, you'll use it on a crappy site. That site will get broken into. That is much easier now with AI. That password gets stolen. And then somebody can use AI now to go use that to go take over your bank account, to go steal your click-dose. Check Bank of America, check Mx, check Morgan Stanley. And it just takes these AI, you know, 15 seconds with their bot armies to do all of it at once. Yeah. So the number of people doing that is going to go up just because they don't longer. You were always able to write that software, but the number of people who can write that software they are is now much larger. And so, you know, use password, have one good password in your head and then use password managers to reset all of your passwords and all kinds of places. If you're a young person, go do this with your parents and your grandparents and your aunts and uncles. I did this on like Thanksgiving once and it made the rest of my life like every Thanksgiving in Christmas much more much better. Use limited computing for use only the amount of computer you need, right? Like if you're using web browsers all day and you're in the Google ecosystem, go buy a Chromebook, right? Like if you're not downloading software all day and you're just using Chrome all day, then just have a Chromebook. If you're just using mobile apps all day, using an iPad, like there's a lot of people walking around with these huge laptops who then use that huge laptop just for a web browser. And it's like, why? Why use a computer that actually can run malware? And so, you know, again, like I got Chromebooks for my in-laws and my parents and that in iPads and that made my life so much better. And do this for the older people in your life. You talked about the scams. That is a huge deal. It is something that we do not prepare people for the fact that you can now get a phone call with that sounds like the voice of a person in your life. And so, you know, establish code words, right? Of like if I, if there is a situation in which I'm in trouble or something like here's the secret word that I will know, because what is happening is you take it out. Obviously you and I, there's hours and hours of our voice out there, but for just normal people, you just need 30 seconds off an Instagram post. And then you can clone their voice. And then, you know, mom or grandma gets a call saying I've been kidnapped in on spring break in Uncle Poco. And then somebody comes on and says, I'm now going to walk you through how to send me $50,000 in Bitcoin. If you have a one-year grandchild again, if you call the FBI, I'm going to kill them. Now, if that person called the FBI, the FBI would say it's fake. Don't worry. Right? If you looked in Find My, or if you called the grandchild, you would find that they're fine. But people are so afraid and it's so realistic that they end up sending $50,000. And you'll never get that money back. And so, you know, talk to the people in your life about these kinds of scams. Tell you to establish passwords. Especially if you're traveling internationally because you look at the, that's how they figured out they look at the Instagram post. They see that you're in Uncle Poco. You see your own spring break. They clone your voice. And it only works one out five times. But if one out five times, it works. Then that's that's free money. So yeah. One more question about the individual before you go to the government. I know you want to talk about that. I think it's fairly common for people today to think, well, I, I guess I use the same password in a couple of different locations. But it's all two factor authorization. I always have to enter my phone number in order to log into, you know, whatever, Bank of America, Twitter. To what extent is to factor authorization and effective block against these kind of AI exploits? Yeah. That's good. I mean, two factors good. What's better is to set past keys whenever possible. And then that gets tied to a biometric your face or your fingerprint. And two factor can help against really advanced attackers. What you then also want to do is to make sure that you've set a pin with your cell phone company so that you can't get your sim swapped. That's more for people who have like lots of crypto or something like that. That's not for the prosaic attack. But you know, if you've got a million dollars in crypto, then you will absolutely get your sim swapped or something like that. Here's a tip. Never post about how much cryptocurrency you own or one, you should never hold your own cryptocurrency. Like if you if you're a cryptocurrency person, I'm not a big crypto fan. I may own exactly zero dollars and zero cents of crypto at the moment. Okay. Great. I have none. Don't come after. I mean, people come after me for other things. But like this is not the thing to say publicly. Yeah. Yeah. Yeah. Exactly. Right. So this is actually, you know, the Democrat people's Republic of Korea is, you know, absolutely the Lazarus group there is the they steal billions and millions of dollars of cryptocurrency year. They specialize in this. And one thing to do is like people like, look, I'm doing great. I'm a whale. And they post about it. It's like, here you come. You've now got a dedicated team of guys in North Korea whose entire job it is to turn your life upside down. And they're pretty good at it. So anyway, yeah. For normal folks, two factors is okay. Pass keys are better. Again, you store those past keys. If you're like, if you're entirely in the Google ecosystem, you can use the Chrome password manager. If you're entirely in the Apple ecosystem, you can use apples, you know, I cloud password manager. If you're mixed, then you should use a third party one like one password, which will allow you then to do that across multiple devices and then store past keys and then have one good password that you don't share with anybody that you use to unlock that password manager. Finally, government. I think the best way to ask this question. I like you describe what you see the government doing now and what you think it should do because quite honest with you, I sometimes find it very difficult to describe what the Trump administration is doing when it comes to AI regulation. It's like the story is one thing on Monday, another story on Wednesday, another story on Friday. So I want to see this through your eyes. What do you think the Trump administration's policy on cybersecurity and cyber vulnerability is today? And what do you think it should be? Yeah, so I mean, there's a couple of things have happened. So on cyber overall, we just had a huge loss in state capacity in that based upon kind of conspiracy theories and a bunch of politically motivated stuff around election security. Our premier defensive cybersecurity agency, Sissa was effectively destroyed. Over half the employees are gone. The capabilities Sissa was created by President Trump in his first term by the amalgamation of a bunch of different roles that were played by different agencies. We finally for the first time ever had a defensive cybersecurity agency, civilian defensive cybersecurity agency, the US government. President Trump created that. That was a really good thing he did. And then he didn't like the fact that that agency said the 2020 election was secure. So he blew it up in a second term. And as a result, a bunch of the critical things that Sissa did are now not being done by anybody. It's not like other people did them. They're just not being done. That is. That is. So for example, a big role Sissa had were these things called the sector coordinating council. So a big chunk of Sissa's work was working with these things called ISACs. ISACs are nonprofits that pull together critical infrastructure sectors, as well as noncritical part, but they started critical infrastructure. So financial services, ISAC, EISAC is the energy sector. And then Sissa would work with them to figure out, you know, what are the regulatory needs you need to bring them intelligence? So Sissa's part of Sissa, one of the cool things is as a SISO as a chief information security officer. A big, complicated part of that job is like, who do I talk to in the government when there's a problem? I'll give you a fun anecdote. I was once when I was the SISO at Yahoo, I was at a classified briefing at the FBI's SKIF. And one of the things they were doing there was they were talking about how, hey, we're creating this new clearing house as a B.F.S.S.A. in DHS that if anything happens, cyberwise, you should come to this new clearing house and see what services there and FBI and NSA and DHS. And everybody's nodding an agreement of yes, this is how we do this, right? This is how we're supposed to do this. There's one clearing house now. If you need anything, you go to this clearing house. And then we get this classified briefing and what's going on or whatever. And then we get up for like a break for bagels in coffee, terrible government, classified bagels, right? And Malcolm Palmore, who is like the agent in charge of the FBI cyber division at that time, this huge ex-Marine puts his big hand on my shoulder and he says, "Son, I don't care what they say. If you have a problem, you call me still." It's like Malcolm, 90 seconds ago, you were nodding along when they said this is the new clearing house, right? But that's what it used to be like. It was like every agency in the government wanted to own cyber. And then we created SISO and SISO became the clearing house. Now, the FBI still has their thing they do crime or whatever. But like at least you could go to SISO and you knew everybody would be notified. And especially the interesting part for SISO was they had people with clearances that would take all the classified stuff. And then somebody in the government was fighting like, there was somebody whose job it was to be like, "Hey, this classified data is really important. Let's strip away the classified parts and then take at least what are called the IOCs, the IP addresses and like the hashes and the malware samples. They don't need to know it's colonel so and so in the Russian GRU. They don't need that stuff. What they need is. the IP address of Colonel So-and-So, let me declassify this and give it to the energy I sec, right? That the GRU is currently spreading malware in the Ukrainian energy sector. Hey, they don't even need to know it's Ukraine. They just need to know, look out for this IP address or look out for this Shaw 256 of this malware. And that's something Sissa used to do really well. And that stuff's been, I mean, there's still people trying to do that kind of stuff. There's still good people there, but it's been decimated because by definition, the best people at Sissa were people who could get jobs like that, right? Who could like, the people who are working there could always get paid three to four times as much money. They were there for the mission. And when they're getting attacked and said that they're like anti-American and whatever for work at Sissa, plus they've never had like a Senate confirmed director in this administration. There's been all this drama and problems anyway. So there's a state capacity problem on cyber. On the actual regulatory side, the term administration comes in and says, like, we're not gonna regulate AI go wild, right? Like the Biden administration had a kind of toothless yo that was mostly focused on preventing the Chinese from getting GPU access. There's a bunch we can go into here. It, the Biden, we're not gonna judge the GPU debate. That's another hour long episode. That's an hour episode, but it basically didn't work, right? Because it turns out GPUs are both fungible. And then also you can put GPUs in the UAE and then they can SSH into them over the fiber optic cables. You don't have to actually have the GPUs in China. Okay, so that's a whole thing. But there's some other like little stuff, but you know, Trump blows it all the way, right? 'Cause it's Biden. Okay, and then they say we're not gonna regulate anything. And then mythos happens and the-- - Is this super powerful and - And the rapid model that was carrying people because of its cyber hacking capabilities. - Yeah, and then Trump administration all of a sudden really cares about it. And then they massively overreacted this year to, you know, Fable comes out, Fable supposed to be the consumer version of mythos, which is mythos, but it's got protections in front of it that doesn't allow you to do big cyber stuff. It allows you to do little cyber things. So we have this idea of like short term versus long term cyber. So you can use Fable to find individual bugs because if you are writing software, you want Fable to be able to kind of self criticize. What you can't do, and nobody's ever demonstrated, you can't ask Fable, hey, go break into hugging face or go break into a bank, right? It will not do that for you. It never has done that. You can ask mythos to do that. And so what happens is there's a dispute between Amazon and Anthropic on exactly like where the line should be between short and long. It's a reasonable dispute. Yada yada. The White House finds out about this dispute and instantly overreacts. And instead of having a reasonable conversation between technical people, you have cabinet members freaking out on a Friday afternoon and coming down super hard on Anthropic and on 5 p.m. Pacific time, doing an export designation on Anthropics model. Now there's all kinds of arguments that this isn't actually legal, that they don't have this legal capability but Anthropic decides to not fight it and they pull down Fable. This becomes a humongous, this was a humongous own goal in the American technology industry because what it did was it meant that American tech providers are no longer reliable because at 5 p.m. on a Friday, Pacific time, 8 p.m. Eastern, you could just have a critical piece of American infrastructure turned off because the White House says so. That doesn't even happen in China, right? Like it turns out the communist party of China provides a more permissionless infrastructure environment for their tech industry. And so this was a really big deal and lots of people thought the White House have reacted because a lot of the capabilities Fable showed were actually at the time available from Chinese models and while Fable was down, GLM52 is released, which has even more capabilities. And since then, the White House has talked about this framework that they have, which they have not released publicly. So we can't even read what the framework is. And it's a voluntary framework, but apparently it's not voluntary 'cause if you don't follow it, they're going to force an export designation. So we really don't know what's going on. It's reasonable to have cyber restrictions, but from my perspective, if you're gonna focus on a rule here, you have to focus on the lawn horizon stuff, the hugging face-like stuff. You have to let these models find bugs because every company in the United States is gonna have to find and fix their bugs. We have to do this. When every company in the United States does not have to do, is Ask ChatchyBT, go break into another company. Right, so that is where you should have the restrictions. And we've never really had that problem with the American models. 'Cause, and so I don't see there actually being a huge challenge here for the, there was the escapes, but those models I escaped intentionally had the security protections taken off because they were evils. So I think the companies, one of the things I suggested is, anthropic and open AI need to have like a self-regulatory structure and they need to have a group that goes looks at all the escapes and comes up with much better isolation, I think even up to air gaps for cyber evaluations. And they should follow those rules. And then the White House can maybe bless those rules or we have a group called Cassie, which is under NIST, that's supposed to be doing the technical side of these evaluations. But in the meantime, the White House should not be putting out rules that they apply only to American companies that they don't publish publicly, that the rest of us can look at, super double secret probation, right? And that don't apply to Chinese companies. We should not have a standard up here for American companies that standard out here for Chinese companies. We need to focus on giving capabilities to American defenders and also telling the rest of the world American companies are reliable partners. You can build on American infrastructure. You do not have to like hugging face rely on Chinese models. That is a incredibly stupid on goal on behalf of the United States. Very last question. A couple weeks ago, more than 1,000 employees from some of the frontier AI labs signed a letter calling for an international effort to "develop the technical and governance tools" necessary to, this was their term, deliberately pace AI development before rapidly accelerate that side of our control. Seems like we're at cross purposes here. A little bit. 'Cause in the one hand, I take as a theme of your testimony here that we're in a little bit of a race against the technical improvements of open-weight models. And we want America's cyber defenses to be better than the cyber attacks that will be possible with open-weight models that are advancing very, very rapidly. That would argue for continuing the current pace that we're at. On the other hand, there's this fear that things are getting dangerous too fast. And therefore, you've got all these people who know way more about AI than I ever will, arguing that no, in fact, we should not continue to accelerate toward new frontier to always keep America necessarily ahead of the open-weight models. We should try to find some way to deliberately pace AI. Now, I don't think their enemies in America. I'm not suggesting they're trying to make us fall behind China. That's not the implication. It's just that there seems to be a tension here, a quite profound tension between the need to stay ahead of our adversaries that are going to have incredibly powerful AI models, open-weight models in the near future. And also the need to not build something that goes out of control and creates a crisis that's hugging face times a thousand. Am I wrong to feel a tension between these two arguments and how would you reconcile it? No, I mean, you're not wrong. I would love to pause the world and try to figure this out. My working assumption is that's not possible. And so as a defensive side of security guy, I have to do my work within the world that exists. I don't think it was this letter. There's another group that's very much against AI and they have proposed an international treaty to try to stop a high development. And I believe in that treaty, it was something like control, try to stop the creation of any amount of compute larger than like 30 H100s, right? In video H100s. I have been to gaming land parties with more compute than that, right? So like, I just. The cynical part of me believes I just do not believe. I think you can have what people call like track two discussions between labs for sure. I can't imagine like she and Trump in a room together for a start three treaty for AI, right? Like I don't think that's gonna happen. I think it is possible to have track two discussions of we should make sure that our models should follow basic rules and not have certain capabilities out of the box. Now when you talk about open weight models, the challenge is those capabilities, any safety protections you put in place can be removed and any capabilities they don't ship with. If they're just generally smart, then you can often add those capabilities back in. But it's better that they don't come with them out of the box. We haven't talked about bio or nuclear. Those risks are different in that. They have a physical component, human beings have to be involved. So I don't see the risk going exponential. The thing about cyber is, you know, these things just output text. That's all they do in the end. Is they output text. And cyber is just text in the end. It bits, you know, like, and so you can do all the bad stuff in cyber without ever touching the physical world. Whereas if you want to do bad bio things, you have to hook it up to something that can do it. And there are real risk there, but like it's, there are other things involved, right? And so I think, yes, it would be wonderful to stop the world and just stop this and figure it out. I just don't see that as realistic. And so in a world where that is not happening, we have to find bugs. We have to fix them. We have companies have to think about their patch cycle. They need to reduce their text surface. They need to get off of physical servers on to containerized infrastructure. They need to get off of their physical stuff into the cloud wherever possible. They need to get rid of their old systems. They need to shift their defense. They need to shift their development, their vulnerability prevention left and stop making new vulnerabilities. They need to shift their defense right. So they need to get ready to be, to actually have intrusions and to shift, have much more protections deeper in their network. They need to have AI-based intrusion detection and response. They need their first line operation center to be automated. They need to be able to look at, hugging face gave us this really good right up of what happened to them. They need to look at that and they need to be able to protect themselves against that level of adversity where an AI agent's doing tens of thousands of different kinds of attacks over a two day period. That is based upon the technology that's available today. So even if we stop the world, you gotta do those things. Yeah, and I just don't, I teach at Stanford and I was there full time in my first appointment that was in CSAC, which is all about nuclear non-pluriferation. And like down the hall from my office, there's like a piece of rubble from Hiroshima that was given to the scientist at CSAC from like the mayor of Hiroshima on like a thank you for I think the work they did on more of the start treaties. And it's like, you're like, oh man, it kind of brings it to you, right? And but when you think about like, how did we survive the nuclear arms race? We got really lucky as a species that the major input to nuclear weapons, like everybody's watched Openheimer, so we all know this, right? There's knowledge from the Manhattan Project. But we're also really lucky that the other input into nuclear weapons is uranium in 238 and plutonium. Plutonium doesn't exist in nature. You have to make it and to make it you need uranium. And there is uranium all over the place, but it's pretty rare and you have to refine it and that refining process is a massive industrial process. Normal people can't do it, you have to be a state and you can see that from satellites. If your, if your uranium 238 was something you just dig up in your backyard, there's no way our species would be alive, right? Because the knowledge to build a nuclear bomb is available to basically every physics student in the world. Now, you can't control from knowledge. Large language models, we teach a class at Stanford that teach us students how to build large language models, like it's an undergraduate class. The hardware to do it, like I said, is I have a gaming PC here that has like the basic hardware to do it slowly, but to do it, right? And ever, you know, like, so I just think these kind of international treaties to just stop AI development would be spectacularly, spectacularly hard. It's just, if you think about how hard it was to control nuclear weapons and such, you're talking about things that had to be created by states. Now you're talking about things that can be created by undergraduates, it'd be spectacularly difficult to impossible. And so in the meantime, I more power to people who were trying to do, and again, I think track two discussions between the labs is a great thing. We should try to at least control the capabilities of these things. But in the meantime, those of us who work in cybersecurity just need to do the best we can to secure the world as it is. - Yeah, now the fundamental challenge here, which you've spoken to, and I don't think we have a formula yet, is how do we essentially democratize cyber defense before we witness the democratization of the cyber offense that we're seeing throughout the world, right? Like, democratize is a nice word. But it's fundamentally what we're seeing, is that people who previously did not have the ability to launch these large scale attacks are likely going to have it in the next year, five years. And what do we do to brave the world, to sort of protect the world before this sort of stuff, gets into the hands of all sorts of people? It's going to be a huge challenge, and maybe we'll have you back on in a year to evaluate your prediction that 2027 is going to be the beginning of a little bit of a valley of chaos. Alex Stamos, thank you very much. Thanks, sir. (upbeat music)