EP.25 - Wondder - Mihai Streza, Founder&CEO - Empowering People through Virtual Reality

Arguably, the only topic that's more controversial than AI criticism at the moment is really any mention of the Trump administration. Well late last week, both of these topics came together creating a tidal wave of chaos and recrimination. Here's what happened if you haven't been following. Back in April, Anthropic announced that their new large language model, which they called Claude Mythos, was so good at finding bugs in computer code that it was too dangerous to release to the public. Here were their exact words. The fallout for economies, public safety and national security could be severe. Okay, fast forward to last week when Anthropic essentially said, "Hey, good news." We added guard rails to Mythos and now it is safe. They called this protected version of the model, Fable 5, and they made it widely available. On Friday, the US government said not so fast. Now according to David Sachs, who was until recently the White House AI's are, the administration had heard from an independent researcher they trusted who said that he had easily evaded the guard rails that had been added to Fable 5. The Commerce Department promptly placed Fable 5 and its unprotected version Mythos 5 on an export control list, which means the company must suspend access to the model from all foreign nationals, which presumably includes many of Anthropics own employees who are here on visas in our foreign nationals. They said they won't lift this restriction until Anthropic fixes the guard rail issue in response. Anthropic had no choice, but to temporarily shut down all access to these two new models. All right, so that's what happened. Most immediately, the internet exploded with most of its criticism focused on the typically haphazard and inscrutable manner in which the Trump administration, as it usually does, acted in this case. Here's a headline from the economists that I think captures this mood well. It reads, "Doddle Trump's blocking of Anthropic is capricious and chaotic. America's closest allies are shell shocked." Dean Ball, who previously served as a senior policy advisor for artificial intelligence, voted this way on X, "Make no mistake post Mythos. The United States has a licensing regime for AI. It's just informal with no consistent rules or firm boundaries on state power or public transparency." Even Gary Marcus, who is no fan of Anthropic, was uneasy by the way this all went down, saying on X, "Whatever you may think of, Dario or Anthropic, Friday's decision and the impetuousness and arbitrariness of it was a terrible mistake that has left a stain that will last." This is not a political show, so if we put the politics of the situation aside, there are some key deeper questions that lurk. For example, are these new models actually national security concerns, and two, in the hands of a more competent administration, is something like this more hands-on regulatory approach actually warranted? Well, it's Thursday, which means it's time for an AI reality check episode of this show, which is the perfect opportunity to go seek some measured answers. Before we get into it, I want to add one warning. This is a fast-moving story. I'm actually recording this episode on Monday, June 15th. By the time this episode airs on Thursday, the situation on the ground with these issues will almost certainly have changed. In fact, my read, trying to see the chatter and the writing on the wall, my read is that the government is going to soon reach an understanding within Thropic, and they will release their export control list in the next probably one week or so. I don't know quite when, but I think that's what's probably going to happen. This restriction was done too hastily, and Silicon Valley has too much influence on the current administration for it to stand. With this in mind, we're going to try to focus on here in this episode of these, those more fundamental questions, which I think are relevant, no matter how this particular story with Fable 5 evolves in the next week or so. Alright, that's enough preamble. Let's get into it. As always, I'm Cal Newport, and this is Deep Questions, the show for people seeking depth in a distracted world. Alright, so let's start with our first question here. Is the government right to be worried about Fable 5, the consumer version of Cloud Mythos with the Guard Rails added? Now they haven't actually provided many details about their decision to restrict this model. So our best source of information, at least as of the moment of this recording, is David Sachs. Let me read a little bit more detail about how he explained what he understands happened that led to this restriction. So I'm going to read him here verbatim. A highly credible trusted partner of both Anthropic and the US government, who was testing Fable, came forward with a jailbreak of those Guard Rails. The admin asked Dario to fix a jailbreak or de-deploy the model, Dario refused. And their blog post Anthropic defended his decision by saying the jailbreak isn't serious. This is not what the trusted partner and the US government believe, nor is that kind of minimizing language consistent with Anthropics brand as the AI Safety Company. It's difficult to fathom how they could claim a jailbreak allowing operability of a cyber weapon could be defined as not serious. All right, so stepping back, the government's argument seems to be the following. Hey, Anthropic, you told us that mythos is essentially a cyber weapon that will create grave national security concerns if bad actors gain access to it. Your protected version, Fable 5 is not really that protected. If we can get around the protections, then the cyber weapon is now available. We have to put a stop to it. All right, so let's break down the government's argument a little bit more. The first part, do we believe David Sachs that a trusted partner of the government was able to bypass Fable 5's guardrails? My answer to that is like, of course, that's not surprising. What do we mean by guardrails? All right, so guardrails, when it comes to applications built on top of large language models, usually means you have done fine tuning of an already pre-trained model where you have a bunch of sample answers and responses. You'll have our questions and responses, questions that you don't want the model to answer, and then your sample response will be something like, I'm not going to talk about that. Using techniques that come out of reinforcement learning, you can then fine tune the model so that when they see questions similar to those sample questions in the training set, they will divert the flow of token output logic towards those predefined answers. We've been doing this since GPT-35. This is why if you just say, hey, give me the instructions for making a bomb, it will say I'm not going to talk about that. It's because they gave it many different variations of people asking for bomb instructions, and each time reinforce the correct answer there being, I'm not going to talk about it. Sometimes, I'm going to be clear about it. Sometimes guardrails can mean something else. For example, you could have a more old-fashioned symbolic pattern matching logic where before the prompt even gets sent to a large language model, the computer program, the control program says, oh, I recognize this as a bad question, and I'm just going to short circuit it and not even send this to the language model to answer. There are a lot of credible reports out there that actually that pattern matching guardrail is how these companies have been dealing with some of the more embarrassing examples that circulate the internet, like asking a language model to count the number of R's in the word strawberry, which is most of the major models struggle to do. The accusation is they just hard-coded a pattern match. If someone's asking that question, just give them the right answer. There is another embarrassing example where with many of the major models, if you ask them, I only live a quarter mile from the car wash. Should I walk or drive to get my car washed? Most of the major models said, oh, that's not that far you should walk. Again, supposedly, they have now hardwired with pattern recognition. If we see any variation of that question, don't even send it to the LLM, just say like you should obviously drive because you have to wash your car. So this what guardrails are, we've been doing these since GPT-35. They're very available. When someone says they've jailed, broken a model, what they normally mean is they are able to get it to give a response about something even if you had fine-tuned it not to give responses for it. There's a couple ways to do this, but to understand how this works very conceptually. This is rough. RL nerds, please restrain yourselves for this explanation. Essentially, when you reinforcement train these models with these sample questions and safety responses, you can imagine what you're doing is when the model has recognized the pattern of one of these type of questions, you have sort of burned its weights to sort of divert the answer from those type of activated patterns towards the safety answer. So any sort of question, like the questions you trained it on, when those neurons are fired up in this sort of virtual parameter space, the model has been adjusted its weights to sort of go downhill from there towards one of these safety answers. So the way you get around these guardrails, you jailbreak them, is if you're getting at the question in a way that really doesn't fire up those same neurons, you're coming out in and off-uscated around the valet way, that will often evade the guardrail. This is why, for example, I've read a bunch of these papers where they try to convince a chatbot to give it information about stuff it's not supposed to. I read a paper recently, I think from last year's NURRIPs, where they were able to get the major models to give conspiracy theory information. What they found is it took a while of prodding it until they finally done, until it would finally tell it the earth is flat. And again, that's a similar type of evasion. you make your context window long enough, it's the attention space is so calm. complicated that it no longer matches with the patterns from the RL training and now those diversion stores the safety answers are bypassed. So guard rails putting the nerd stuff aside, they're available. And this has just been the case with language miles up to now. So yeah, I have no trouble believing that this sort of whatever guard rails they added, from what I understand, they're guard rails to say, I will not answer cybersecurity questions. I'm sure they're available. You know, we've never seen a guard rail that we couldn't jailbreak as one way to think about it. All right. So the second sub question here is "Fable five without guard rails or jailbreakable guard rails, a unique national security concern." This is where I'm gonna differ with the government. I'm gonna say no. Look, I did an episode about the original mythos model if you, you know, back in April, when it was originally announced. Now remember, when the original mythos model was announced andthropic went on this PR campaign to try to terrify people, they gave briefings to government officials, the banks and the reporters, about how scary this model was, how it had this sort of unique revolutionary ability to find and exploit bugs and software that was going to like take down the whole infrastructure of the world if we didn't protect it. This is when Tom Friedman wrote a very scared article about kids would now be able to basically shut down nuclear reactors in some such, right? At the time, here's what I said. I said, "I don't think this is revolutionary. I think it's evolutionary." We have been using large language models to find bugs and security vulnerabilities since the beginning of large language models. With each new generation of models, they've beginning steadily better at this. We have no reason to believe. We've seen no evidence that mythos is a revolutionary leap in that ability as opposed to just continuing the normal trajectory that in other words, there was nothing shockingly new with mythos that made it unusually dangerous compared to like, Opus 4.7 or GPT-5. We had some evidence for this point. For example, in its scare marketing campaign and Thropic talked about the fact that they had found over 1,000 zero-day vulnerabilities in code some of which had been around for decades. This really got a lot of people scared. The problem is you can go back to one of their earlier models when they're earlier Opus models. And if you read like I did, a reported anthropic release on their blog the same day as that earlier Opus model came out. They said, "We found hundreds of zero-day vulnerabilities some of which have been around for decades." So it wasn't some brand new thing that mythos could do that earlier models could not. Then we got multiple independent security researchers that said, "Okay, well, we took some of the marquee bugs that anthropics had they had found with mythos and we gave that same source code to other models, smaller models, pre-existing models, cheaper models." Said, "You'll do a bug search on this and they found the bugs as well." Then we got other sort of independent benchmark testing of mythos once it became a little bit more available and it really fell into this pattern of like evolutionary incremental increases on these type of capabilities. And of course, maybe the biggest sign that mythos was not this world-changing bug finder is that anthropic's own software remains very buggy and has security vulnerabilities, even post-mithos. So I guess it hasn't been able to fully find all of their bugs. So what was really going on here, my contention was the original mythoscare campaign was marketing. One of the other biggest pieces of evidence for that is bug finding pre-mithos was not what the AI companies were bragging about. These were not the capabilities that they were touting to try to emphasize the power of their software and all of its possibilities going forward. Bug finding is what we were doing with like GPT-2. This is not exciting. So the fact that that is what they emphasized by mythos in my mind was, uh-oh, we trained this new massive model and it got like a little bit better at everything. That's not exciting enough. We need headlines. What can we say it? What if we say it's bug, let's focus on it's bug finder? We haven't talked about that in a while. Oh my God, we built an ultimate bug finder. No where before mythos were they saying that was their goal or this was one of the big uses of LLIM. So to me, that was a red flag. So what I think they did is they were hoping to play us both ways. Say this thing is really terrifying, get a lot of terrified columns, get everyone really worried about it, wait a few months, made a month and a half and then release it and try to ride on that sense of, this is a super powerful model that we then release a version to the public. They'll be willing to pay the much higher token prices that these larger models command. So they're trying to play it in both ways. Terrify us and still just go right ahead without restriction with their plan of releasing and trying to make money off of that software. So I think that was just pure marketing. Again, this doesn't say that LLIMs are not good at finding security vulnerabilities. This is not the saying that the new coding harnesses that took off starting last fall aren't good at writing exploits if you explain to it. A vulnerability. But what I am arguing is that we never had evidence that mythos was some sort of revolutionary leap in these capabilities that would merit the way that Anthropic talked about it or it uniquely being put on an export control list as compared to other cutting edge frontier models. All right, the second relevant question is of course, should the government be more involved with AI? So if we are putting aside politics, like is there a way we could imagine a more hands-on regulatory regime that would make sense? Well, in order to understand what role the government showed us on the play here, let's look a little bit closer on the question of why the Trump administration did what it did with mythos because there's three different explanations that are out there right now floating around. The first explanation is corruption. The Trump administration has multiple ties to Anthropics main competitor open AI. They've also already been in a bit of a feud with Anthropic dating back to March and the issues they had with the Department of War, would they place it as a supply chain risk? So this would be very Trumpian. You are my enemies, here's a way to squash or screw with you. I'm just gonna do it. The second explanation given for what happened, the government is legitimately worried about Fable 5. And the idea of China using a jailbroken Fable 5 defines security vulnerabilities and key software that's a part of our infrastructure and economy really scared them and they said, we don't want this to happen on our watch. Reason number three, explanation number three is that they're calling Anthropics Bluff. Anthropics spent six weeks trying to convince people that they had essentially summoned the demon with mythos and that it was a grave powerful source. They almost regretted having brought into this world and that they were just trying their best to steward this inevitable darkness and keep us safe. And then six weeks they were like, "Yeah, never mind, here you go. It's gonna be 75 cents token." So another way of thinking about this is the government said, no, you don't get a talk about your product as the worst thing since nuclear weapons. And then just go and sell it without restriction when you want to. We have some evidence for this bluff calling rationale. If we go back to David Sachs who said the following, keep in mind that Anthropic itself widely promoted the idea that mythos was a cyber weapon and needed to be regulated as such. They asked for government regulation of mythos and championed the guardrails unfaithbled. If there's a vulnerability, bigger, small, it is Anthropics responsibility. The patch. All right, so let's go through these three possible rationales for what the government just did and said, are any of these justified? Could we imagine any of these being the foundation for an actual sustainable approach to AI from a regulatory framework? Well, the first explanation was corruption. And of course, no, that's never justified. Now, unfortunately given the recent history of our current administration, we have to assume that corruption or arbitraryness of revenge probably played some sort of role in whatever they were doing. We cannot have regulatory actions be implemented so capriciously. I agree with Gary Marcus and Dean Ball about the need for a transparent and consistently applied regulatory regime if we are to create one of those. All right, the second potential justification is the national security concerns. Is the government in general justified for blocking or restricting an AI model if they have concerns about its ability to cause harm? Yes. That is a good justification for the government to step in. You do not, as an American company, have a right to ship any product you want without interference from the government. If it can cause massive harm, especially harm to us caused by our enemies, this is where the government should say, hold on. Again, you are not as the AI companies like to make themselves seem inevitable stewards of an inevitable technology. You're building products. If you build a product that's going to screw us, we don't want you to release it, just like you can't release medicine this poison or cars that will explode. What about the third just possible explanation here that they are calling the AI companies bluff? Is this ever a justified rationale for government intervention? I'm gonna say yes. I'm gonna say yes, because I think this is a public health crisis. These companies have been trying to terrify the public for at least the last two years. And I find this strategy of wanting everyone to be unsettled all the time, unconscionable, strange, and baffling. And it has been incredibly successful. There is a pall of anxiety and fear and uneasiness that tens, if not hundreds of millions of people are feeling right now because of the direct communication strategies of all these companies to be continually trying to keep people unsettled and fearful and distrustful of what this technology is gonna do. The psychic damage this has caused to our country unquestionably as much larger than any benefit to the AI technology to date has actually provided us. And I think the government has a role to get involved here. You do not get without restriction to run a siop's on 300 million people because you think either it makes you feel. important or you think it's going to help you in a small number of early investors become richer than, you know, Mammoth. That is the government steps in now if you make a claim in your dog food ad that isn't quite right. And yet, hey, you want to just like continually see, have a game of who can verify like the average, you know, the average citizen more about AI without any restriction or change to what you're doing, we're not just going to applaud it. I think that actually is a reasonable place. These are extreme circumstances. It is a reasonable place for government involvement. So what would it look like for the government to act on those justifications better than what we're seeing right now out of the Trump administration? Well, I want to return to the Trump administration because they did something recently, which is a step in the right direction. On June 2nd, Trump signed an executive order titled Promoting Advanced Artificial Intelligence Innovation and Security. This report, this executive order requests that AI companies voluntarily provide the federal government access to covered frontier models for a cybersecurity review up to 30 days before their plan released other trusted partners. That's a step in the right direction because it shows like, wait a second, the government should have some role in these technologies powerful enough. I would go farther, however, I would say it is mandatory, not voluntary. Companies must preemptively establish the safety of frontier models, which you would define probably in terms of parameter size before public release. In these reviews, the communication of the companies itself should be taken into account. So if you have gone around and talked about how terrifying your model is, or if like anthropic did a couple of weeks ago, and I talked about it last week on the podcast, they came out with report. It's like, you guys were looking at our own cloud code. And like, this thing is getting close to recursively improving itself. We might lose control of it, but don't worry. We have a white paper in which we're going to walk through the possible future. So we're thinking about it at least. You don't get to write that article. And then release a new version of your cloud code. The government should be like, okay, you said this a technology can get out of control. Just like if you were a virology lab, at least is the apt comparison. If you were a virology lab, talking about your gain of function research, writing reports about this research we're doing, it lead to out of control pandemics. And we're going to write reports that walk through the possible ways like society could crumble when these out of control pandemics come. The government would have every right to say, you have to stop doing the gain of function research then. If you think this is a possibility, well, you got stopped right away. All right. So I think you should have to put frontier models up for review. How you talk about them is included in that review. The government should have the ability to also retroactively go back and say this model we previously approved. We are now seeing signs that is causing grave safety concerns and we can revoke its license. Safety is actually going to have to matter. Now, this has to be done in a transparent way. There has to be consistent mechanisms. These mechanisms somehow have to be free from direct influence by Trump as mediated by whoever he happens to be friendly with or whoever gave him a lot of money. None of that's easy. But I think it is time for something like this. The owner's for safety has to go back on the AI companies themselves just like with every other consumer product we see. You're welcome to talk all day about how dangerous your products are but you can't release them. You got to convince us and the government that of course we're not going to release something that does this harm. Of course, this is not going to happen. This is a specific product. Here's why it's useful. We are responsible for its harms. Now, if we did have this regime, I think two things would happen that would be immediately positive. One, I think the worldwide siop's about trying to keep everyone terrified so these companies can feel exceptional and push forward to these big IPOs. We'll come to an end because we're not going to allow them to release products that they're trying to terrify us about. Two, I think it would lead to a narrowing of their products which would be good. Instead of trying to do this, let's build the biggest possible frontier model and then after the fact try to probe it and see what it can and cannot do and kind of ring our hands. I hope it's not too smart. Build specific products. Here is our coding tool. We have to justify for you why we think it's worth the money it costs and of course if it cut we're responsible for any harms like any other consumer product company. Here is our product for memo writing and we want to justify why it's worth the cost and of course we are responsible for any harms. You get narrow and responsible. That is the future of AI that is going to be way more predictable, it's going to be way less distressing, it's going to be way more safe, it's going to have a lot of variety. The giant AI companies hate this model because hey guess what? When you start releasing specific products, you don't need a 10 trillion parameter model. I don't think the AI companies want you to know that these frontier models are F1 cars. It's like McLaren or whatever Ferrari or Red Bull, whatever, F1. We put out these 20 million dollar cars to try to win this ludicrous race. We go 180 miles per hour so that people in general are like, I like that car company. That's a good car company. I'm more likely to buy the $60,000 car which is all I need. That's what they're trying to do here. These models are F1 cars for most of people's needs. Much cheaper models would suffice. Remember when I said a lot of independent researchers could duplicate bug identification using smaller models? Yeah, that's probably the case. You could have a bug finding system tuned to do bug finding with a good harness on it. This made to like test and look for bugs and be better at it. They could run on probably like a 50 billion parameter model. Right? A lot like coding harnesses. I think Cursor is discovering this. You can build these custom models that are not massive and are perfectly good at producing computer code and you put the right harness on it. It's very effective. Much cheaper to run. This is the reality. We need the, I don't know, cars very well. I was going to say like the Ford Tories. That's a 90s reference. Right? But we just need the normal consumer cars. These companies are still working on the F1 cars to try to impress everyone so their IPOs can succeed. They don't like this message being out there because a world of narrow AI application, they don't have an advantage anymore. Who's going to have the advantage of computer code? Well, everyone, these models aren't so big and then it's who's going to build the smarter harness and some company that does nothing but think about harnesses is going to have a tool that people prefer. There's no mode. So they want it to be about mythos. They want it to be about GPT-5-5. They want it to be about these massive models and finally one way or the other to justify them. That's why they have to constantly put out these articles of like this thing as a national security threat. This thing is going to come, is going to improve itself until we lose control. They have to make it seem like only these frontier models that only they have the capital to make are the only things that are smart enough to be useful. But they're not. We can have much narrower tools to do useful stuff and they're justified and they're affordable and they're not at all something we worry about. That's the future we need. In a real consistent and transparent regulatory regime that says, "You have to convince us your model is safe before we release it. You don't get to just build whatever you want. You don't get to just say whatever you want about a model." The time has come from that. The damages being caused right now psychologically have been massive. The damages economically that are going to happen if both open AI and anthropic have big IPOs and then the bottom drops out on this. The impact that's going to have on 401(k)s that are holding index funds is also going to be calamitous. This is time for the government to stand in but they have to do it with more transparency and honesty than I don't know if the Trump administration is even capable of doing. But that is how I feel about it. All right. So there's a lot of things going on in this story. It's a Trump administration handling this in a way that's haphazard and at best and corrupt it worse. Probably is Fable 5 a unique danger, probably not. What I care about most is that this incident points towards a potential future in which the government gets off the bench and stops treating these AI companies like some sort of untouchable priestly class and say, "Wait, who says you get a launder anxiety and potential destruction to your own personal wealth? You don't get to do anything you want. And if they're not willing to be responsible in both their rhetoric and product design, then someone needs to step in and help put these AI companies back in their proper place as normal consumer product companies that are beholden to the same type of rules, restrictions, expectations as any other consumer product company. All right. That's all I have for today. Join me on Monday for an advice episode of this show and I'll probably be back the Thursday after with another AI reality check. Until then, remember, care about AI but not everything you read about it. All right. See you next time. So far, you must be ready to join my fight for depth in a distracted world. Now the best way to do this is to join over 125,000 people who receive my email newsletter each Monday. You can sign up at calnewport.com/ideas and when you do, I will send you a free guide to my seven best ideas about cultivating a deep life. Join up today at calnewport.com/ideas.