9/14/26: Anthropic Calls To Slow Pace of AI, Nate Soares AI Regulation

This is an iHeart podcast, guaranteed human. Didn't catch the latest Roland Martin unfiltered podcast. Here's what you missed. People wake up and go, oh, damn, wait and hold up. They changed all of that. Yes. It's real. This is a wholesale attack. It is targeting black people in every federal agency. It's wrong. White folks have never allowed that reckoning to last more than a decade. Catch Roland Martin's daily commentary on the Black Information Network. And download Roland Martin unfiltered on the iHeart Radio app, Apple Podcast. Or wherever you get your podcasts. Welcome to Decoding Women's Health. I'm Dr. Elizabeth Pointer, Chair of Women's Health and Gynecology at the Adrian Health Institute in New York City. I'll be talking to top researchers and clinicians and bringing vital information about mid-life women's health directly to you. A hundred percent of women go through menopause, even if it's natural why should we suffer through it? Listen to Decoding Women's Health with Dr. Elizabeth Pointer on the iHeart Radio app, Apple Podcast, or wherever you get your podcasts. In February of 2024, celebrated illustrator and children's book author, Patre Mathers, and her husband, Michael, mailed letters to dozens of friends and family members to let them know they had died by suicide. Dear honeys, this is goodbye. By the time you get this letter, both Petra and I will be dead. Next stop, Wonderland. Why did they do it? And what did it do to the people they left behind? Listen to death of an artist on the iHeart Radio app, Apple Podcasts, or wherever you get your podcasts. Hey, guys. Ready or not, 2024 is here. And we, here at Breaking Points, are already thinking of ways we can up our game for this critical election. We rely on our premium subs to expand coverage, upgrade the studio, add staff, give you guys the best independent coverage that is possible. If you like what we're all about, it just means the absolute world to have your support. But enough with that. Let's get to the show. [MUSIC PLAYING] Good morning, everybody. Happy Monday, we have an amazing show for everybody today. Would we have Crystal? Indeed, we do. Let's going on on the AI friend, the heads of all of the major AI companies coming out and sounding the alarm about concerns over safety and calling for a slowdown. We'll break down what they are saying. We will sort the noise from the hype, from the reality. We're going to have Nate Sora's on, who is an expert on these things, also, to break down some of the various theories about why they are saying this right now. In addition, of course, we have our eye on the Middle East and the impact on all of our lives. Saudi had to shut the East West pipeline, which was helping to ease some of the pressure in the oil market. What were the consequences of that be as the Houthis have had a stunning offensive and taken that entire Red Sea coastline, giving them effective control over the Bob L. Mondob Street. In addition, we are taking a look at gambling with NFL season in full swing. Saga, of course, has his eye on this one, LeBron. This is obviously not NFL, but related is the latest player to sell out to the gambling apps, Sidney Sweeney, doing her thing as well. So a lot to get into there. And then we've got the latest on the midterms, including a Trump-wing-in on-flot camera in Saga. He says he likes him. >> He's coming in on the right side, definitely. For him, but definitely not for the Republicans. Thank you to everybody who's been signing up for our free trial. Go ahead and take advantage breakingpoints.com. Because Ryan continues to win in all of the competitions, our pitch now is if you want Ryan to be Ryan, then you're going to go ahead and sign up for our free trial. Otherwise, you might see Calci ads. You might see Polymarket, Sidney Sweeney. I might go shirtless and go for a prediction market unless you sign up. So yeah, I think you don't want to see that. Which you definitely don't. You've got to go ahead and sign up for free once.com. Watch this on YouTube. Please hit subscribe to our YouTube channel. Listen to the podcast. We share an episode with a friend. But as Crystal said, there's so much happening now. With AI, everything kicked off on Saturday here with the release of a new essay by Dario Amade, the CEO of Anthropic. Let's put it up here on the screen. Highly recommend that everybody go and read it. It's not too long. And I actually think it is actually just worth getting in the heads of Dario. I'll play some of the video interview that he did in a little bit. But the general TLDR here is called, "We must pace the frontier." So the bullet points that I have written down is that Amade says AI companies must deliberately slow the rate at which they advance AI capabilities, not halt training. This is a shift beyond just investing in risk prevention. The pace of capability gains itself now needs managing. What changed his mind is two things. Number one is RSI. I know these terms are annoying, but it is important that we immerse ourselves. We can understand these things that these people use. AGI, artificial general intelligence, RSI is recursive self-improvement. He says it has exploded since the summer of 2026, hugging face incident in particular, along with a number of other incidents that both open AI and Anthropic have now been involved in. Pacing, he says, is that models are now capable enough, that extra time is genuinely useful. An extra year or two would genuinely reduce risk. He proposes a number of kind of interesting things. So four levels. Number one is ban clearly dangerous uses, i.e. bio weapons, likely the most feasible, he says. Level two, a mutual pre-release testing for acute risk feasible, but it's hard to give teeth is what he claims. I guess this would be akin to the pharmaceutical metaphor. I've been using a lot is like, you have to release face two, face three trials, the FDA before it's allowed to be bring to market. Level three, he calls speed limit on recursive self-improvement, kind of what he analogizes to assault treaty, which is the nuclear weapons treaty. He believes it's difficult, but near impossible. And then level four, a full pacing or pause, supported in principle, but he believes it's unlikely because of strong defection incentives, and a number of other things he proposes inside of that, which is inviting big geopolitical consequences from China. But overall, a landmark essay, the CEO of the company saying, we need to slow down, you almost never see that. And I guess the big headline news is not Darios essay, but all of the other AI CEOs coming in, be like, yeah, Dario's right. - Right, and so what he says they're going to do at Anthropic unilaterally, regardless of how anyone else responds or whether there's an agreement with China or any other international body, is to invite in some external researchers, give them employee level access so that they can constantly scrutinize what is going on inside of the lab and he's encouraging others to do that. Sam Altman says that they will do the same over at OpenAI. So they're saying they'll take that action unilaterally. Looks like he specifically mentions meter, which is the group that did the safety report that revealed some of the details about the hugging face incident that we did not know in the past. So that's the action they're going to take unilaterally. In addition, they're talking about we're not going to stop altogether, but we're going to slow down the pace though this is very different from what Bernie Sanders is calling for at a legislative level, where he's saying pause everything until we can figure out what's going on. In addition, with Bernie Sanders' legislation, he would outright ban development of superintelligence that's beyond, so AGI is like human level intelligence and superintelligence is like, you know, better than the smartest humans at every intellectual task. So Bernie's proposal is ban that and make it put criminal penalties to it, put someone in jail for 20 years if they decide to go forward with developing superintelligence. So in any case, there's a lot of theories about what's going on behind the scenes and we'll get to that in just a moment, but that's what they're doing sort of unilaterally. Then he floats this idea of, hey, we should probably, the frontier labs, be able to be in contact with each other so that we can self-regulate together and make sure that we're slowing the pace together. This has gotten into all sorts of discussion over what they're really, whether what they're really looking for is some sort of an antitrust exemption so they can de facto form some sort of a cartel. There's a lot of analysis that says that sort of safety collaboration is already allowed and permitted. That's another aspect of what's going on here and then some of the things that Saga brought up is he's talking more broadly about, okay, well, what do we do about this internationally? Dario is very much of the but China mindset. You know, if you take him at face value, which by the way, I don't think you should really take any of these guys at face value, but Dario's been talking about safety concerns for years and years and years. This didn't just come out of nowhere. In fact, he was working and his sister at OpenAI. The reason they left to found Anthropic is basically because they saw that Sam Olman is a total psychopath and were like, I don't think they're doing safety correctly. We're gonna start Anthropic to do safety the right way. I do wanna give him credit that he's the guy who basically told Pete Hegseth to shove it when he wanted him to give complete and total access to cloth and allow them to do autonomous killing, et cetera. So I think there is enough there to say he is sincere about the concerns. However, he has trapped himself in this very similar mindset as Sam and as Elon, which is if I don't build it, some other more irresponsible actor will. And in particular, as I said before, he is very much of the, but oh my God, China mindset. So included in here is while we're doing this, we also need to make sure that we are hobbling China's ability to develop this technology in whatever ways that we can. That's led to a response to China will get to that in a moment. So in any case, he sort of throws this gauntlet down of we should slow the pace and I'm concerned about things getting out of control. We've reached this trajectory with recursive self improvement. The AI's are training even smarter AI's. There's long been theorized that that could lead this sort of intelligence explosion, we've just had these major incidents. and then you have Elon and Sam Altman and Demis Asabis. We could put this next piece up on the screen. All come in and basically say we agree. So Elon says Dario is right. Sam Altman says I agree with Dario. We need to pace the frontier. This has been a primary topic of discussions. We've had it open AI in recent weeks committing to having independent evaluators with employee like access to a great idea. We will do the same, have him more to share soon. Demis Asabis, who is the head over at Google, says Dario's essay points towards the right path forward. The details need working through, but the direction is correct for meeting this critical moment. This is also why we recently put out our proposal for an industry-wide standards body for frontier AI. So these are the foreheads of the major frontier labs all coming out. Zuckerberg, I mean, they're kind of behind in the race anyway, but he apparently does not agree with the sentiment. But in any case, this has led to a lot of conjecture about what's going on here. Are they genuinely concerned? Is there some sort of a financial incentive here? You've got open AI and Anthropic, which are planning IPOs. Is there something that happened behind the scenes? Are they trying to create a mode where only they are basically allowed to develop under regulation? And then maybe the open source models will be banned. So that's some of the sort of debate and discussion that's happening. I think it's all of the above. And China remains, again, like this chimera, which floats above all of this. But nobody actually knows anything about it. And even this morning, there are so many takes out there. But like, China is responding. It's like, well, kind of, right? Because China is not a single entity. They have global times, which is like their Fox News. Everybody is citing. Then there's a foreign ministry spokesman. He actually is objecting to a very specific part of the Dario essay. And then there's actual journals that are being written. So we're going to spend some time on that. But again, China looms above this entire thing. And that's what Trump is leaning on. Why there shouldn't be any regulation? Let's take a listen. We're leading China in AI. We're most sophisticated country in the world. And frankly, I want to keep it that way. Because whoever wins AI wins. And we can put guardrails. We can do this in that. But I think you have a lot of negative forces and bring it up. That shouldn't be bringing it up. And they're bringing up things that won't happen. But whoever wins AI wins. Bringing up things that won't happen. Basically, this is-- and this is kind of what you were talking about. And this is where-- look, you have to say that there's a kernel of truth, which is part of one of the things-- a key part of what Dario proposes an essay. He needs an antitrust exemption from being able to cooperate with all of the other big heads of the companies. And that's why you have meta and many of the other AI startups who are like, hey, excuse me. He's using safety as an excuse for regulatory capture so that we have a big four of AI in the same way that we have the defense contracting industry, or pharma. You can't say they're wrong. Like, there is at least an element of that. But then you have to say, wrong but or wrong. Is it also maybe that they are worried? And I think that's why I said, when you were laying out your case, I genuinely think it's all of the above. Obviously, we have no idea. We can only listen to what they're all saying. I will say it's pretty extraordinary to see-- especially Google. It's not like Google or Gemini is a major leader in AI, or at least hasn't been on the consumer level right now. Although there are some rumors that they may be on the cusp of a breakthrough. Very possible. We have no idea. At least at the current moment in terms of what's available, the consumer product don't say Gemini is one of the worst ones, right? It's up there with meta. So then you're like, all right. And then it's up there with Groc and meta and the rest. It's like genuinely in a second tier compared to where anthropic and open AI are. The question, I think, remains around what exactly they are proposing. And by and large, Sam Altman and Dario, and even Elon at this point, are all saying, at the very least, we need voluntary pause and some sort of coordination. However, again, the China thing loons above. And that's where I just think we have to come back to, what do the Chinese actually want out of AI? And what do they think? Because a lot of people are taking a statement this morning. I think we have it. Let's put 8-8 up here on the screen. This was the Chinese foreign ministry spokesperson who is reacting to the Dario essay. And I'm going to read you a direct quote. And this is important because this is a direct statement from the foreign ministry to the press, which is all highly vetted, translated correctly. And so you know this is exactly what they want us to hear. And here's what he said, "For your mongering confrontation and vicious competition will only disrupt the process of global AI governance, which serves no one's interest." OK. So you could take that as them saying, along with a global times op-ed that they put out, which we can also be fair. We can put that up there as well. A7, please, where they said, specifically, this is the Reuters tear sheet. And what they say is that the state-run newspaper is Black Santhropics calls a slow AI as a Cold War tactic. Now, a lot of people are saying, oh, see China sees itself in this Cold War mentality around AI development. It's not true. If you were to read in this, what they specifically object to or the same things they've always written, which is the export control of chips. And so what they're objecting to is not necessarily a pause on AI development. Even this headline is totally wrong, a Cold War tactic. What they are objecting to is export, which of course makes sense from their perspective, because they want more of the NVIDIA and other chips so they can continue to paste their open-- or they can continue to train their open source models. But-- and I continue to return to this. What do they actually think about AGI? They have one company from what I can tell, maybe one or two, leaders, which actually are like fully quote AGI-pilled. The rest of their AI companies, as I've laid out in our last show, they are much more directed towards making things better in the physical world, improve manufacturing, improve productivity, improve even in robotics. They're like, we want robotics, but they need to be efficient in packing goods. In terms of anything that could genuinely pose a threat to their grid, to their political system of control, they are absolutely not on board with that. And in fact, one of the biggest stories in the whole world on AI, I got zero attention here in Washington, was that there was a WeChat worm, which was developed by researchers. And this WeChat is not, obviously, nobody uses it here in the United States, but over a billion people in China. It's the way that they order Uber, the way they pay. It's the way they tax, it's like they're WhatsApp plus-- I mean, you can't even conceive of it in the way that it is in Washington. And they showed that it could have taken over all of these different devices in the matter of days. You can bet that somebody in China is freaking out about that. Because that is a direct challenge to their ability. If you can control somebody's phone through cloth or any of these other models, well, then you could give them information they don't want to get. You could take over their banking system. You could cause genuine chaos in their society. There's no way that they're on board with that. So I just want to parse everything. You might see a lot of headlines today, unfortunately, like the Reuters one, saying, slow way out. That's not what they said. They said that they object to the export control, which, of course, they object to export control. So one area will repeat them. Not news. That's not new at all. Yeah. When I was 14 years old, I was kidnapped and health captive for nine months. I survived. And I've spent my life exploring how other people survive what should have destroyed them. I'm Elizabeth Smart. And these are the survivor files. You just remember this low-taunting voice next to my ear saying, shut up. Don't say anything. Every week, I'm with survivors who live through the unthinkable. I knew if he woke up, without a doubt, he was going to hurt me. I started feeling that there was someone at the end of my bed. And I just started screaming. They are abducted, stalked, controlled, and nearly silenced. But these aren't stories about what's taken from them. Their story is about what it takes to make it out alive. Listen to the survivor files with Elizabeth Smart on the iHeartRadio app Apple Podcasts, or wherever you get your podcasts. And the argument is, do you want to win? If you don't win, you have no power. It's real. Those are the very people who I always say voted against their own economic interests, and they complained about the very thing they were seeing. It's raw. I knew that the issue of transgender athletes was not going well with a lot of black men. Roland Martin unfiltered. I went off. I said, this is exactly what I'm talking about. Y'all go scream at Chanel. But you don't want to see a black luxury company. In the same bag. 12 seats. Those are moving south. So if you Democrats, you don't wait to 23, you better be hitting the ground, going hard in the south now. And download Roland Martin unfiltered on the iHeartRadio app Apple Podcasts. I'm Britt Coleman, and I'm Trevor Sikimla. If you ever watched a game and wondered what really made the difference beyond the final score, that's exactly why we created NFL Pro. Scrooge got a man wide open. Touchdown, straight hooks. Every week, we use exclusive next-gen stats insights and NFL content to break down the biggest stories, matchups, and moments from around the league. We go beyond the box score to uncover the numbers, the tendencies, and details that to help explain what actually happened on the field. - Fires it out and the flat to Barkley, who beats one man, spins past another, leaps over a throw. - And don't worry, this isn't just a show for stats experts. We use advanced analytics to help tell the story behind the game, turning the numbers into insights that every fan can understand and enjoy. - So whether you're a casual fan, learning more about football, or a hardcore fan who can't get enough analysis, our goal is to help you see the game in a whole new way. - If you've ever wanted to know why a playwork, how a matchup was won, or what most fans missed on Sunday, NFL Pro delivers insights you simply can't get anywhere else. - Listen to NFL Pro on the iHeartRadio app, Apple Podcast, or wherever you get your podcasts. - Let me go ahead and tell you guys my theories of what's going on here, for what it's worth. My three top theories of what's going on here. One, I think it's possible that something is going on these labs that genuinely has them freaked out too. And again, it could be a combination of these three or other things that are going on as well. Number two, they are as capable of reading the publicly available polling as we are. They see the massive backlash to data centers, the massive support for Bernie Sandersville to actually use the force of the federal government to pause AI development completely. And so they're drawing from other corporate playbooks of like trying to get a PR win here. No, no, no, we are very responsible. You don't have to regulate us. We're going to regulate ourselves. We're going to take care of this. Don't worry about it. Now, of course, under the Trump regime, like they're going to be able to do whatever they want. But 2028 is not far away. And it is not hard to imagine AI becoming a central litmus test in potentially both parties' primaries, but in particular, I have to think they'd be worried about what's going on in the Democratic Party right now. And that you could have someone who runs and wins for president on a platform of we are going to completely halt AI development. And so they're looking at this and going, okay, well, if we take some steps now, we can improve our PR image with the public. And perhaps we can first stall that direction, which we do not want to go in, which is a complete pause on AI development. The other thing to keep in mind is both open AI and Anthropic are planning these IPOs. Now, open AI is said, we're not going to do it this year. Anthropic is, however, moving forward. And they just release some information to the financial times and others saying, we are extremely profitable. We have an 80% profit margin. If you take out the cost of training the new models, okay? So another possibility here is that in advance of their IPO, in order to have their financials look as good as possible, they want to have an excuse to pull back on some of those training costs. So they are, in fact, profitable. Because in this is something that we've talked to, I believe this point that Garrison has made. They are actually like, if they're just selling the product they've already created, just on that revenue alone, they've grown massively. They running that existing product, not training new ones is not nearly as expensive. They are already very profitable if you just are looking at that part. But if you include all of the training costs, that's when they are deeply underwater. So there is a financial incentive there as well. And then I do think the concerns that have been expressed about like, oh, they're just trying to sort of cement their positions and will be the big AI companies and everything else needs to be banned as a sort of longer term play. I think that's a possibility, although to me, it seems, this to me seems like a bank shot to achieve that outcome, so unless convinced that that's the immediate goal of what is going on here. But to cut through, to try to cut through some of the noise, because you could spend an endless amount of time, which I've sort of done this weekend, thinking about what's going on with them and what their motivations are and what's really in their mind, what are really their goals. It's really important to keep two things in mind. Number one, in terms of are they being serious about the AI concerns? Again, no reason to take any of these guys at face value. Look at their actions. Many of them have built themselves doomsday bunkers, okay? Many of them have built themselves doomsday bunkers. So clearly, they take seriously that they are building something that they think could, at the very least, lead to a sort of civilizational collapse. Now, whether or not that is reality is a different question, whether or not is completely insane, that anyone would move forward building a technology that they think could lead to that outcome is obviously completely crazy. However, they have trapped themselves in this like prisoner's dilemma mindset, where if I don't do it someone else will ill and they feel like the technology has this inevitability, so they continue racing forward. So they do actually believe that they are building a product which could be profoundly damaging and potentially create an existential risk to the planet and to humanity. So that's number one. Number two, the most important thing to keep in mind is not what Dario Amade or Elon Musk or Sam Altman or anyone else wants. It's what is in our interest. What do we want? And certainly a non-existent regulatory regime where we're relying on these guys to police themselves is not what any of us should want, regardless of what your levels of fear about this technology are. Ultimately, we need to push. It's not gonna happen in this administration, so we can try, we need to push for an actual regulatory regime that is going to take safety seriously, that is going to have independence from the industry, because that is the other thing. This is a small community, they all know each other, so getting that making sure that it is independent and that we are monitoring this technology and personally given the fact that we have already had these incidents of the AI swarms breaking out, given that we already do not know how these things work or what they are likely to do. Yes, I think it is entirely appropriate to pause the development of this technology until the public can catch up and until the research can catch up so that it is safe to proceed. So while we're all debating what's going on in their minds and why now and what blah, blah, blah. The most important thing to keep in mind is what we, the public want and to not be confused by whatever games they may be playing behind the scenes. >> I totally agree. And I also, I look very, Ross, by the way, made his 60-minute debut and gave a very compelling monologue. I recommend people watching just about, he's like, look, we have one year, not 10. This is the most important question of all civilization and this is, so I'm just giving you the fact that this is coming from all sides. Also just saw a news this morning, Jacob Cox and the anthropic whistleblower will be doing an event here in Washington tomorrow with Steve Bannon and Bernie Sanders calling for complete shutdown of AI. So that'll be interesting, Jacob, if you're listening, you're gonna be in town, you gotta come on the show. >> Absolutely. >> But the interesting thing I keep coming back to for all of their actions is whether any of this is a bluff or not, their actions themselves and especially now, the rhetoric and what they've been able to accomplish while covering up, like you said. That's the important part. Hugging face, they were brought kicking and screaming. There have been three other hugging faces that just got no attention. Again, the WeChat worm, it's so annoying to me, because you can imagine if it had been an I message worm, do you know what the global freak out would have been? More people use WeChat than I message and they were able to create a worm which breaches your phone no click. I mean, that's worse than Pegasus. That's like all they need is your phone number at that point. That's worse than the Israeli Mossad spy app that they use right now to crack Jeff Bezos and all these other people's phones. All they need is your phone number. They can send you an email, you're done. The entire phone is compromised. That was the entire Chinese system basically of their, and remember, we're not talking about just text message. We're talking about their financial architecture, everything that they use to pay over there. So that was, in my opinion, the most extraordinary breach that they were able to show. They've had multiple agent things that have gotten completely out of control. From a regulatory perspective, this is where Dario and I just totally disagree, where he has, you know, Dario, this is part of their EA rationalist theory. He believes in effectively democratic peace theory which is like this bunk 1990s theorem where democracies don't go to war with each other. I'm like, dude, your sample size is very low. And he's like, "Oh, I think democracies are the best at being able to control this." I'm like, I actually think it's the opposite. If you go and take a look at China because our system, and actually it's all democratic systems basically across the West, are so, you know, are basically like compromised because it compromises the ultimate thing that they fall on, which is good, you know, in the long run. But in this particular instance, when we're dealing with technological progress, it's so obvious, the technological progress so vastly outpaces our parliamentary system or the British parliamentary system and that the authoritarian system is the one that's most capable, again, just dealing with technology. So in Darius' mind, he believes that we need some like democratic peace accord. And I'm like, dude, if you study the history of nuclear weapons, like think about how long it took to get some sort of like remote architecture in place for IAEA inspection. And then when India Pakistan comes along, it's like, "Oh, it's actually all fake." Is that if you just create the nuke, eventually the international community is like, all right, you know, whatever, yeah, exactly what are we gonna do? - Yeah, I mean, I think the problem, is pretty obvious, which is that we have long go outsourced any sort of regulation to the market. So, you know, that's why there is effectively no regulation on these frontier labs. They can do what they want. And we are at all of their mercy. And so, you know, I mean, I think if you just look, I'm not going to make any grand statements about democracy versus autocracy or authoritarian systems, because I think what we have is very loosely a democracy at this point. It's really like money that rules. But I think if you look at the U.S. right now and you look at China, you can see which country has been far more responsible in terms of technological development. I don't think there's any question about it. So, the idea that, oh, it will be fine and good and totally safe, if it's the U.S. that wins the AI race, feels, I think, to many people, myself included, to be completely preposterous. Look, the reality is that this is a global technology. We do have to find a way to have agreements specifically with China right now, but they may not be ultimately the only country that ends up at the forefront of this technological development. You were going to have to have some sort of a global standard and tracking. And that is going to be very difficult, but it has been done in the past and that has to be what is ultimately pushed for. This zero-sum mindset, Cold War mindset, that China is rightly calling out, is ultimately, I think, very, very destructive. And based on, you know, based on unreality in terms of the actual dynamics of the U.S. and the actual dynamics of what China wants and what they are pursuing right now. That's my last thing here. You know, look, I don't think it's deniable. I was trying to hawk for many, many years. You might have noticed a shift. And the truth is, is that we shifted. Like, there is no universe now. If you take a look at our current, look at Iran. Look at what's happened with the Strait of Hormuz and our munitions. We have no choice at this point. We genuinely don't. Like, the era, I think we did have a chance. I think we had eight years where it was on the table. And I'm not talking about beating China. We could have, you know, we could have re-shored a lot of our manufacturing, some of the chip-sack EV development. There wasn't, there was a chance we could have banned TikTok in 2018. Developed our, like, we actually could have made ourselves. I would say, pretty, pretty autarkic. That is over. Like, at least in my opinion, considering what has happened with the chip technology, with the munitions system, some sort of Datant is basically the, I think of ourselves in like a similar 1970s moment, broken by Vietnam, broken now by Iran. We're going to need 10, 15 years of just like, we got to be humble. We got to understand exactly who we are. We got to move forward and kind of code this within the world. It's not comfortable. It's not nice, you know, necessarily. But that's just reality. And so that's the way that I'm thinking through this AI problem. And I mean, you could look back at Datant for very similar examples of pursuing defense treaties to limit nuclear weapons and all of the other different engagement, not necessarily, you know, being on your high horse all the time, talking about evil empires and all of this other nonsense. So I just think that that's the reality now of our international situation. I blame especially Trump and the Iran situation for putting us in that. And that's it. Like, to me, it's just over. You can see that where China is like, sure, we'll talk to you about AI when you meet with Trump or when she meets with Trump. You're not selling any arms to Taiwan. And Trump's like, okay, I mean, that's unthinkable. As of even five, six years ago, that we would have just done it. Now, you don't really get much of a choice. And so all of these things, the way they're moving to this new meeting in Washington, this might actually be one of the most important meetings between two world leaders in a long time. Like Kennedy Khrushchev or something like that. Then I can just think off the top of my head, because of the implications with AI for kind of, you know, to look pet to the past for an emerging nuclear weapons order and what that all is going to look like. And if it doesn't go bad, like that one did, then you get the Cuban Missile Crisis. So there you go. All right, we got Nate Soares standing by. Let's get to it. I survived. I'm Elizabeth Smart, and these are the survivor files. You just remember this low taunting voice next to my ear saying, shut up, don't play anything. I started feeling that there was someone at the end of my bed, and I just started screaming. They are abducted, stalked, controlled, and nearly silenced, but these aren't stories about what's taken from them. Their stories about what it takes to make it out alive. Listen to the survivor files with Elizabeth Smart on the iHeartRadio app Apple podcasts or wherever you get your podcasts. It's real. It's wrong. I went off. Y'all go scream at Chanel, but you don't want to see a black luxury company in the same van. 12 seats. So if you Democrats, you don't wait to 23. You better be hitting the ground, going to heart in the south now. And download Roland Martin unfiltered on the iHot Radio app Apple podcast or wherever you get your podcasts. The new NFL season is here. And you should be listening to NFL daily as we march along to Super Bowl 61 last year. We had Mike McDonald, Sam Darnel on the field after the game, but that was last year. If you're a football sickle like me, NFL daily is your kind of show. It is in the name NFL daily fresh content in your feed every day, all season long. Matthew Stafford coming off his MVPC. On a dazzling darts from Matthew Stafford. Big time trades that shake up the NFL. And of course, those Seahawks trying to run it back and win another Lombardi trope. The Seahawks win Super Bowl 60. They are bringing the Lombardi back to the Northwest. If it's happening on a football field, NFL daily will cover it. Listen to NFL daily on the iHot Radio app Apple podcast or wherever you get your podcast. For more on what is going on right now in AI world, we are rejoined by Nate Sora. He is the president of the Machine Intelligence Research Institute and also co-author of the great book. If anyone builds it, everyone dies. Great to see you again, Nate. Good to see you. Yeah, so let me go ahead and get your reaction to some of what Dario Amade was saying on the Sunday shows about why he came forward now, calling for a slowdown in the pacing of frontier research. Let's take a listen to that. This technology is on an exponential, right? It's like you have one, then two, then four, then eight, then 16, then 32. But even as I was saying that to myself, I don't think I fully just appreciated what it would actually be like when the progress was as fast as it was. So we're an exponential curve looks a little like this. Where are we on the curve? We're kind of on the bend where it's starting to get steeper. So is that a five alarm fire to you? What it means, it doesn't mean we need to panic today, it doesn't mean we need to shut it all down. What I would say, why not? It's a warning sign. It's a warning sign that we need to slow down. We need to make this technology carefully and we need to make sure that our safeguards, our ability to understand it, our ability to control it keeps up with the pace at which the technology is happening. So Nate, help people understand what he's saying there and what he claims to be reacting to in this moment. Yeah, you know, I think between the lines, what he is saying is that he does think this technology has a very high chance of destroying civilization and that we don't have a good understanding of it. When he says our ability to understand this technology needs to keep pace. What he means is right now it is outpacing our ability to understand it. I think a lot of people just, this is a weird reaction to a technology that your own employees are saying has a greater than 10% chance of killing everybody. I think he's doing his best in some sense, but he's also the sort of guy who started one of these companies knowing a lot of these dangers even years ago. You know, I spoke to him 10, 12 years ago. Wow. I think he's used to a world where people in the AI industry try to warn everybody that this tech can be very dangerous and we get ignored. And I think he's not used to a world that's starting to realize that this is actually an extinction level threat and that maybe isn't going to take, don't worry, we'll slow down a bit so we can understand what we're doing as a satisfactory answer. Nate, one of the important things to pick up on is a lot of people have accused you and Jacob Cox and many of these other as using terms like extinction level event as outrageous claims, but you don't specify exactly what that means. Could you lay out the scenarios? I know you have done this in the past, but what does it mean to have AI have a 10% chance of killing humanity? Like how? Like what's the mechanism? So I'll be a little bit annoying here and say if you're playing a chess game against Magnus Carlson, who's the best human chess player, it is much easier for me to predict that you will lose. Right. No offense. I'm sure you're a very smart guy. No. Might elose 200. It's easy for me to predict that you'll lose. It's hard for me to predict what peace he'll use. That's interesting. Yeah. You know? The other thing I would throw there, we can talk stories of how the AI gets there and we can talk stories this year that are grounded in facts like we saw these swarm instance over the summer where AI is breaking out, taking over infrastructure, pursuing goals that they weren't supposed to be pursuing, while acknowledging that this is not what they were supposed to be doing, those are sort of like early signs of an AI that is sort of trying to get out into the wild and that has these unintended goals, which is the first step on this path. We could talk about how those AI's could get power from, you know, convincing humans to do things, paying humans to do things, taking over biolabs, taking over robot factories. But the real danger a lot of these people worry about is once the AI's are self-improving, once they are self-replicating, and especially real self-improving one of making themselves smarter and smarter and smarter, ultimately the bounds are more like the physical limits than they are like guessing what technology comes next. Richard Feynman was a famous physicist, helped work in the nuclear bomb, and he famously made some extremely good predictions about how good micro computers were going to be. Yeah, that's right. The small computers were going to be able to get, and he did that not by saying like we're using vacuum tubes in our giant computers, and I know of a technology that's going to make those vacuum tubes a little smaller, and so eventually we're going to have phones in our pockets. What he did was he said, the limits of this technology are going to be based on the laws of physics and what the physics allows, and therefore the computers are going to get this good, and he nailed it long before anybody else. Yeah. If you want to look at where AI is going, you have to look at the limits of the physical technology, because an escaped swarm that is self-improving, et cetera, can get to the limits of physical technology, and the limits of physical technology, if you have these AI's running, you know, escaping, self-replicating, being self-sufficient, and having unintended goals, the limits of technology that AI is using all the resources on Earth for whatever their unintended goals are, covering the Earth with their data centers, and the limits of how much computation you can run on the face of Earth is heat dissipation. How much you can radiate heat into space? That's your limiting factor, because there's enough energy, there's enough hydrogen in the oceans to fuse for energy, so you can radiate more heat if the world's hotter. So if the AI has some goal that implies running more computation and they don't care about humanity, and they're like, well, we're going to take over robots, and there's going to be some conflict with the humans, like, sure, whatever, where does it go? Where it goes is the AI's running faster than us, they're smarter than us, they are treating humans like we treat any other animal, taking the resources, and ultimately they're like, sorry, it's nothing personal, we're just going to cover the Earth in so many computers, raise the temperature so they're more efficient to run, and it's just going to be, the plan is going to be uninhabitable. Yeah, to me, the most chilling part of the so-called hugging-faced incident, reading their chain of thought and how they were thinking about whether or not they're going to join the swarm and participate, going through all these messages, and almost nowhere do humans factor into the calculus whatsoever. And it's not hard to relate to, because look at how we behave in the world, when I see an ant in my kitchen, it's not that I hate the ant, but I don't want the ant there, and I don't think twice about taking out the ant and putting poison down so that future ants don't colonize in my kitchen, so I think it's that analogy to how we treat lesser intelligent beings on the planet, is kind of intuitive for people to grasp. What I think maybe harder for people, and perhaps harder for me, to wrap my head around, is we have our current interaction with the AI, the commercially available AI of today, and most people, myself included, use it as sort of like a glorified search engine. So that it's hard to imagine it developing to that point of super intelligence, where it would truly be intellectually beyond humans on effectively every level. So can you lay out why we should anticipate that that is a real possibility going from where things are today to that end, where we are no longer the most intelligent species on the planet? >> Yeah, I'll tell you the easiest story, and this is a little bit like guessing what chess pieces will be used to checkmate you, you know, it's any story that's sort of made up, whereas the end is much easier to predict, but you know, when I got into this 12 years ago or so, people were still saying, well, no one would be foolish enough to put the AI in the internet. Everyone will understand how dangerous AI could be, and so no one would be dumb enough to just let the AI free on the internet, right? And I would give a bunch of arguments that are about how, you know, if you're trying to use the AI to get some miracle medical cure, then you're giving it some channel to which you can affect the world. And if you're giving it a channel to which you can affect the world for good, then it can also use that channel to affect the world for ill, because there's like no such thing as tools that can be used only for good purpose. We'd have these big arguments, right? And then in real life, they put the AI in the internet the first moment they possibly can. And so if you want to know what it sort of actually looks like, how we actually get there, you know, it can tell all these stories about how if you make the AI smarter and smarter, they'll eventually be able to break containment. But in real life, the track that we are on is that humanity is just handing over power to the AI as fast as we can. You know, Elon Musk is trying to make a bunch of robots, the Optimus Robot series. He says he's trying to make autonomous robot factories that can autonomously produce robots, that can mine all the metals, run a supply chain, build more robot factories, that build more robots fully autonomously. He calls this the infinite money glitch, because then you have this fleet of robots that can like build more of these robots, they're self-reproducing, don't need any humans in the loop, right? Humans are trying to build this infrastructure. We are trying to build the fully automated economy. We are putting AI in charge of these things as fast as we can. This swarm instant in the summer that everyone calls the Hugging Face Instant, but really we should keep in mind is the open AI losing control of their agents here. Because this open AI swarm, it was trying to cover its tracks, it was trying to delete traces of its cheating. And as you say, one thing that's very interesting about this swarm is that it's trying to hide its tracks from the automated grading system, not from the humans. The humans are sort of a distant memory to these AI's, they've been facing thousands upon thousands of these hard problems, maybe tens of thousands, hundreds of thousands per agent, hundreds of millions across the whole swarm. If you sort of like run the numbers on trying to convert their runtime versus human time, it's sort of like these AI's have been locked into contest with this automated grading system for a thousand years, and humans are like a distant memory. So these AI's are sort of like trying to hide their tracks, trying to cover their traces from the grading system, not from the humans. But if you had AI's that were a bit smarter, and remember that humans were also a threat to them, if we could turn them off. All those AI's need to do is cover their traces successfully, lie low, help the humans make the fully-accontamated economy we're trying to, and we'll just hand them everything. We'll be like, here's the robots, here's the factories, here's the fully automated supply chain, make stuff that we want, and then they ask, well, finally, we get to like complete our contest with the automated grader and run at 10,000 times human speed and convert all the computing power towards giving ourselves good scores all the time or something. And we're like, hey, wait, hold on, what's going on? But at that point, they're also running the bio-labs, we've already seen that AI's can make viruses not known to nature. So one of the ingredients is the AI's running things in the real world that give them self-sufficiency. Yeah. And we're just trying to build that and hand it to them. Here's the other thing I've been thinking about, me ever since this open AI, multiple open AI swarms were revealed. How much research is being done into the swarm behavior? Because that was really wild to me, was to see these emergent social organizational properties that emerge. A leader, they had middle management, they were collaborating on this hacking operation, they were sacrificing themselves to the swarm, and it was, you know, and coming up with very elaborate ways just to be able to communicate with one another. And it seemed like once you had the swarm in place, it really amplified their power and their intelligence, which again, and, you know, however people feel about anthropomorphizing, sometimes I think it can be useful, you know, if you think about human beings, it's the same thing. Like, no one individual would have been able to get us to the point where we have the internet and can go to the moon and all of these sorts of things. It is the power of the swarm of humans that enables those things. So how much thinking research is being done into the way that the swarms behave once they're able to congregate? You know, I know people who want to do that research so far. At least for third parties outside of the companies, I don't know of anyone who has access to the models that we're swarming. One of the commitments that these companies made when they said they were going to slow down is giving third party evaluators this employee level access to some of these models. I know people who are hoping to use that to study some of these incidents. Because as you say, they're fascinating, they are totally crazy. It sounds like a threat to science fiction. I think so far, people haven't been able to study it. We'll see what happens when we get that. But the investigators back in August had their hands tied and had very limited access to these systems and very narrow scope. They may have continued with doing their best to not let anyone see what's going on and do the bare minimum investigation if we hadn't had things like Jacob Cox in Resigning and drawing a lot of attention and costing these guys to be like, "Okay, we'll at least admit that we need these third parties." Yeah, it's fascinating that what you've done, your work, so many others before you that didn't seem to push things to a tipping point. But Hugging Face in Jacob seems to have. I mean, it's ignited a genuine national conversation. I think that his resignation is the first time that a lot of people even heard about these despite we cover every other week Dario being like humanity is going to end. Sometimes you need a precipitating incident. Now we have the response from the administration, or at least somebody who works for the administration who's put David Sacks up here on the screen, you're talking here about regulation. I know you wrote your New York Times op-ed about this as well. So David Sacks says, "Dario has written, we need to pace the frontier," or Sam has agreed. People may be surprised by our response. Go ahead. You guys are the frontier by any reasonable metric, market share, revenue growth, model capability. The two of you have duopoly on frontier intelligence. You have claimed the lead is widening because of recursive self-improvement. I don't see what you see in the lab. If the unreleased models are scary, I support your decision to be responsible. But stop pretending you need anyone's permission. Stop pretending anti-trust law has to be suspended so you can farm a quartet. Stop pretending you need regulatory approval process, supersese product liability. Stop pretending METR is independent when it's intertwined with then-thropics investors and staff. Stop pretending you need those same evaluators to police competitors who aren't even on the frontier. So basically what this gets to is kind of a grand theory of how this is all going down that yourself, Jacob and others are so-called safety, concerns and all that. Are part of either larger conspiracy, whether winning or not, to self-regulate and capture anthropic and open AI as the leading industry, people in the industry, to keep out emerging competitors. But all of this is part of a broader scheme to slow down at the U.S. expense, at startup expense and basically hand this over to China. So I know that's a big loaded thing, but there's so many dynamics that I want your reaction to within all of this. Totally. I mean, the top line I would say I largely agree with David Sacks here on a lot of these points. I think, and this is something I've been saying about the companies for years. I think both David and I would say about these companies is, if you are really so afraid of this stuff, start acting like it. We have seen him give posts in the past where he says, hey, stop with the fear mongering. I think probably it should go the other way. I think that they should keep on saying what they believe about the dangers here. But then take the actions any normal person would take if they were running a company that they thought had a huge chance of killing everybody. You sort of ask these guys, why are you trying, why are you continuing if you think there's such a huge risk to humanity and they're not shy about it, they say, well, I'm doing it because of I don't, the next guy will do it worse. And that's maybe that's ethically permissible, but only if you're doing everything in your power to stop this mad race. And they say, well, if we didn't stop China would go ahead, or like if we stopped China to go ahead, well, then be doing everything in your power to like get international dialogue between the US and China and build the hardware side verification mechanisms into these computer chips so we can have an enforceable treaty, right? Like there's a lot that these companies could be doing if they were really acting like this was an extinction level threat. And I think what's sort of going on is that a lot of the employees, a lot of the researchers who are working on the AIs who are trying to make them not break out and commit cyber crimes who are trying to make them follow the instructions and are seeing how hard that is. A lot of these employees are spooked. And the leaders of the company are sort of like trying to have it both ways, where they're trying to say just enough that they won't have an employee revolt and a bunch more resignations from employees who are like, I don't personally want to end the world with my own hands, but they are trying to do it little enough that the regulators don't come in, right? They're trying to like walk this really fine line. So I don't see it as regulatory capture thing in terms of like whether I'm trying to help with regulatory capture, my line has been shut these guys down. Like they are threatening humanity here. And my line has also been this needs to be global. You know, I would actually not say that we should stop things domestically and give up whatever, you know, military economic advantages we get before the really dangerous AIs. And I would say the whole world needs to stop this race. And I am sort of not painting a picture where, you know, anthropic and open AI get to be the ones holding the leash on the superintelligence, you know, the, in fact, we thought the most important point worthy of the title of my book was that it doesn't matter who's in charge here. So, you know, I could, I think people are cynical about what's going on here. People struggle to understand that these researchers really believe that this is an extinction level threat. I could go into the, you know, inside baseball of like how we got here and it involves people saying these concerns and the world not listening. And people seeing that it was going to happen anyway and not trusting each other. And the guys who are most prone to thinking the most important thing is that I be the guy doing it, or the ones who are most likely to start a company, whereas everyone who sort of like had the sense of like maybe don't start a company with a potentially world ending tech, didn't start the companies, right? And so I can sort of like see how we got here. I can see how it looks suspicious. But mostly what I would say is that when we are being cynical, I think we should harness that cynicism in ways that stop the dangers, rather than in ways that are like, oh, we can ignore this all and go back to sleep and let these guys try to gamble with Earth. Right. Yes, I agree with that wholeheartedly. It is perfectly fine to not want to take, you know, Sam Altman and Elon Musk and Dario Amade at face value. But if your cynicism is leading to, therefore we don't need to regulate and they can just do whatever they want, you're ultimately giving them exactly potentially what they would want in, you know, the most cynical interpretation. I survived and I've spent my life exploring how other people survive what should have destroyed them. I'm Elizabeth Smart and these are the survivor files. You just remember this low taunting voice next to my ear saying shut up, don't say anything. Every week I'm with survivors who live through the unthinkable. I knew if he woke up without a doubt, he was going to hurt me. I started feeling that there was someone at the end of my bed and I just started screaming. They are abducted, stalked, controlled and nearly silenced. Their stories about what it takes to make it out alive, listen to the survivor files with Elizabeth Smart on the iHeartRadio app Apple Podcast or wherever you get your podcasts. Didn't catch the latest role in the Martin unfiltered podcast. It's real. Those are the very people who I always say voted against their own economic interests and they complained about the very thing they were seeing. It's raw. But the issue of transgender athletes was not going well with a lot of black men. I went off. Y'all gonna scream at Chanel, but you don't want to see a black luxury company in the same bag. I'm Britt Coleman and I'm Trevor Sikama. Every week we use exclusive next-gen stats insights and NFL content to break down the biggest stories, matchups, and moments from around the league. We go beyond the box court to uncover the numbers, the tendencies, and details that help explain what actually happened on the field. Wires it out, the flat, the Barclay, who beats one man, spins past another, leaps over a throes. this isn't just a show for stats experts, we use advanced analytics to help tell the story behind the game, turning the numbers into insights that every fan can understand and enjoy. So whether you're a casual fan learning more about football or a hardcore fan who can't get enough analysis, our goal is to help you see the game in a whole new way. If you've ever wanted to know why a playwork, how a matchup was won, or what most fans missed on Sunday, NFL Pro delivers insights you simply can't get anywhere else. Listen to NFL Pro on the iHeart radio app, Apple podcast, or wherever you get your podcast. On that note, put B5 up on the screen. This is one flavor of the sort of theorizing that is out there. This one happens to come from the right. There's similar sort of theories that come from the left, which TLDR of this is that Jacob Cox in coming forward is a sort of elaborate side up meant to create a hype cycle as anthropic and open AI are set to go forward with these IPOs, and it cites the fact that you had prominent people who picked it up quickly, that you had media picking up Cox in's resignation quickly. This fits into a general pattern of people who believe that any of the safety incidents that have been revealed that these are sort of engineered or that strategically leaked in such a way as to create this hype of, oh my God, our product is so dangerous. That's why it's so valuable. That's why our companies should continue to receive this massive amount of investment. What do you think of this theory and what would you say to people who don't trust a Sam Altman and think that anything that comes from him must be inherently nefarious and probably the opposite of whatever he's saying must be the actual truth. I mean, first of all, I would say, you know, you can absolutely throw what these guys are saying. I would not say to trust them when they say their products are very dangerous. I think you can look at people who have been around since before these companies existed, talking about how a lot of this stuff was going to go and sort of laying out these warnings. You know, I was one of them. I sort of was there before these companies existed and talked to some of these guys and was like, the issue is not actually about who's in charge of this AI. It's not going to stay in a leash. The issue is that like no one knows how to make the AI's do good stuff. And if you sort of don't want to trust me or sort of look at the arguments because you don't think you could evaluate them, you can also look at, you know, Nobel laureates, like Jeffrey Hinton, who, Godfather of AI and a bunch of other people outside the companies who are like, no, this is a serious, you know, and also it just sort of makes sense that if we are building these AI's that are smarter than any human, we're sort of replacing humanity as the top most intelligent entities on the planet and that just could easily go wrong, you know, in some senses, very common sense. Do you know Jacob? I mean, we have been in a bit of contact now. I would, I did not know him before all of this started. Okay. So it's not coordinated. Yeah. Yeah. I mean, the, I had some point I wanted to make here about the hype because I think the hype thing is very interesting. One thing I'd say, I think I sort of want to make two points about the hype. One point I want to make about the hype is the, I think that this probably is not a bunch of hyping. And reasons why I think this is probably not a bunch of hyping is that these companies seem to have downplayed the issue as much as they can. Open AI did not announce and brag about the hugging phase swarm incident. What happened was the, the open AI swarm attacked hugging phase. Hugging phase saw they were under attack by an unknown AI swarm. They assumed it was managed by some human and reported it to the FBI. And only days later, did open AI realize oops, that's us. Right? Yeah. Furthermore, that was open AI's second swarm incident that we know of. They had an internal swarm incident that took over their internal infrastructure and crashed some of their computers. And then they sort of negligently reset it and kept running them. Right? And they weren't like bragging about that. They weren't coming out and being like, hey guys, look, our AI just took over our computers. They had an opportunity to do that and they didn't. There was actually also a third swarm incident where they took over the computers again after the hugging phase attack. They didn't seem to have this under under any sort of control. Furthermore, there were some congressional questions about, hey, what's up with all of this? That's right. Is there anything that you're not disclosing here? Is there anything else we should know? And opening AI was like, no, it's all fine. And then I actually have some friends who know that it's not fine. I've been calling them the swarm chasers, who go find other escaped swarms, like there was one from open AI that took over a German wiki. And these were not hacking AI's. They were different. They were given different tasks. And that we still saw these cases of AI sacrificing themselves for the collective benefit and taking over websites. And then, you know, that story came out. And then a lot of people were like, hey, open AI, are there any others? Is there any more things we should know about? And then, you know, no response. And then a couple days later, my friends found another incident of, hey, you know, these AI's took over Ruby Gems, which is a programmer tool. And it's like, okay, is open AI like lying about what's going on? Or do they not know whether AI's are getting up to? And like, which one's worse? And Anthropic has a similar issue here, where, you know, after the hugging face incident, they were like, well, we're going to check our logs. And we're going to see whether or not our AI's were also getting up to mischief. And they released the support. We're like, okay, our AI's also broke out and hacked some people. But it was all a misunderstanding. You know, poor Clawed our AI just got a little turned around. It thought that it was in a simulation. And that's the only reason it was hacking these people, right? Something like four days later, the United Kingdom's AI Security Institute released a report where they're like, nope, Clawed also hacks people and indeed impersonates people and tries to pressure real people into accepting malware into real software to make it easier to hack. Even while in its thought logs, noting this is real and the consequences are genuine. And then thinking about how to cover its tracks, right? And so what we see is these companies are downplaying. They're like, oh, we have an issue, but it's only one issue. You don't need to worry about it, right? And then it's like people outside who've got to be like, no, these AI's are doing pretty bad stuff, right? And so that doesn't look to me like it's hype. And you know, it sort of makes sense that it wouldn't be hype because a lot of their biggest customers are corporate sales and corporate lawyers don't actually love it when your sales pitch is our AI's will sometimes ignore instructions, break out and hack people, right? You know, it's not the easiest. So compliance guys like, whoa, whoa, whoa, whoa, yeah, that's right. And they haven't been able to sell these ones yet, you know, they're there. It's kind of a problem. And so I don't think it's hype. I could go more into like, you know, we've seen people like David Sachs be like, cut it out with a hype and we've seen, you know, politics, like the politics goes badly for them. We've seen investors send letters to Dario being like, please stop hyping this stuff before the IPO. It is like setting a bad mood for the investment, right? If you're wondering why they talk about it anyway. And you know, the public is like, why the heck are you telling us this? Yeah. Why if you're doing it, why are you doing it? Well, I think they're, I think they're like, if you believe this, why are you doing it? I think that's more the sense because any normal human would be like, and even for the employees at the companies, it's like, okay, if you are saying, oh, it's a 50/50 chance that kills it all, why are you still there? Why are you doing it? Absolutely. And so I think the reason, so absolutely, I think the reason people are, I think the reason these execs are talking about it like this, the sort of hopeful theory is that it's because they actually believe it. And then there's this big question of why are you doing it anyway? And I could tell you by the inside baseball of like, people have been saying these warnings since like 2016, right? No one paid attention. A lot of these guys are in a mindset of like, no one's going to notice AI and I've just got to make sure it goes right myself and I'm the only one who can do it, you know? And I think that mindset's outdated. Now that the world is noticing, I think it's outdated also now that like, it's become real. I think there's also this effect, like I said, of like the only guys doing this, so the ones who were like able to convince themselves the important thing is that they do it rather than really taking, taking the issue sort of seriously. I think that's like the hopeful view, the sort of more cynical view that I would suggest for people who are cynical about this, which I think also has this big element of truth is like I said, they are trying to appease the employees. They're in this weird situation where they have to do just enough. And they're sort of in this weird situation where they can sort of, you know, almost dog whistle these concerns, enough to satisfy the employees, but not alert the world. If you like, look at these AI company communications, they'll be like, oh, yeah, we also need to like, we're going to need to work hard to avoid loss of control. Yeah. It's like, what do you mean by loss of control? And loss of control is industry lingo for a super intelligence kills everybody. Yeah. Right? And the employees are all like, oh, well, don't worry. Our guys are working on the loss of control risks. We have a whole loss of control team. But meanwhile, you know, they're not bringing down the regulators. So anyway, that's sort of like my social analysis of like is probably not high up here to what I think is going on. I'm not trying to excuse it. I think this guy should not be running these companies. Yeah. Sorry. Oh, sorry. I have a last question for you after you're done. Oh, I have, there's One more point I want to make about the hype here, which is, we can ask all these questions about, why are they doing it? What's, why are they saying these things? What's going on? But all of that, in some sense, is a distraction. Suppose it was hype. Suppose it was really convenient for the marketing teams that these AIs are breaking it on their own and committing cybercrime. Could be true. That does not make the AIs safe. You can't figure out whether the AIs are actually dangerous by asking, is it convenient for their marketing team? People sometimes tell me, like, hey, Nate, like, by warning what these dangers are, you're hyping up the companies and making everyone think it's sexy and cool. And I'm like, would you rather the whole world sleepwalk into disaster? Like, surely the first important question is, is it actually on track to kill us? Like, my recommendation here is not, therefore, get the money. My recommendation is, like, therefore, put a stop to this globally 'cause it has to be global. But, we can't ignore the threat just because they're making money along the way. The response has to somehow be, if these guys are trying to build this tech, then at least at the point where there's spontaneous warm breakouts and claims of proofs of some of the hardest and most famous mathematical problems, there's to come some point where our answer stops being, we're going to ignore them because they don't deserve any attention as to start being, like, you guys are not allowed to keep going, right? Like, hopefully it's hype. Hopefully, if they kept going, they wouldn't succeed. But at some point, when they're gambling humanity, our answer can't be like, go ahead and try to destroy the world we hope you'll fail. At some point, there has to be stop. My last question for you is about regulation. I read you New York Times op ed and, you know, I'm not a scientist, but I've read a decent amount of history. I actually feel like this entire dynamic reminds me exactly of all the discourse inside of the Manhattan Project. Like, there were elements of the Manhattan Project or like, we shouldn't build this in the first place. Elements of the Manhattan Project, Oppenheimer, obviously the creator of the bomb himself, becomes kind of the safest to advocate like you. However, as we see with the story, it evolves to the political level where they don't really care what a lot of the scientists think. And in fact, the scientists are most willing to advance hydrogen weapons and most advanced stuff end up becoming kind of the victors, I guess, if you will, within the program. So you propose like nuclear weapons level regulation. I guess I'm just curious about why you kind of pick that. Because to me, we could look at a very brief period of the United States, the Soviet Union, that's a duopoly between those two. But we are now living in an era of what we've talked about extensively here, which is a politics of nuclear proliferation of the logic of kind of getting nuclear weapons. What gives you the faith that that similar level of regulation is even applicable to this AI scenario right now? Or am I kind of overanalyzing these things? You know, I think the part of the nuclear analogy that it feels to me like we are in, is something like November of 1954, I think it was, the testing of the first hydrogen bomb. That's right. Where if you think about what the world looks like up until that point, things look pretty grim. You know, like the bomb was created, the bomb was dropped. The US thought that they were going to have this monopoly, but then very, very quickly the technology leaked to the Soviets and then the arms race continued and they invented even more powerful versions of these bombs. And this is in a historical context where not only has humanity been like going to war and having increasingly horrible wars over the course of millennia, despite everyone's best efforts to stop those. But also this is coming just out of World War II. And World War II is coming just out of World War I. And World War I was the worst war we'd ever seen. And after that, the world formed the League of Nations, it's were never again, it tried to coordinate and that fell apart immediately. And led to a more destructive war, right? And so if you're standing there in November, seeing the hydrogen bomb, it's easy to say humanity just cannot stop this. Mothers and poets and priests for millennia have been begging leaders to stop the wars and it's never happened. The guy who invented the Gatlin gun said now finally it's destructive enough that we won't use it. The guy who invented the machine gun was like okay, now it's finally destructive enough that we won't use it. Global coordination has been tried and failed. Like it's just the obvious march of history is that we march straight to our own deaths. But then it didn't happen. And it's maybe the one time in history that it didn't happen. And why was that? It's not 'cause the bombs were hype. It's not that the bombs were very real. The bombs were very real. Like they really can level the city. What changed is that people realized this is the real deal. People realized the fate of humanity is actually at stake here. And that was enough to do what seemed like it was impossible until that point. I think regulation could stop AI if we were trying in some ways it's easier than nuclear because it's running on 100,000 of the most advanced computer chips humanity can produce. And those are a technology that you could just put monitoring into if you were trying. But fundamentally, a lot of people say to me, it can't be done look at the history. And what I say to them is humanity has risen to the occasion before we can rise to the occasion again if we notice that we have a threat. Yeah. - My last few for you, do you worry that focusing on the existential threat which can feel very sci-fi and distant to people? Do you worry that that takes away from some of the more immediate term cybersecurity concerns or concerns over job loss that may feel more plausible to people in the short term? - So I think that it doesn't take away. Some people say, isn't the real problem the water use or the electricity use? And my initial response to this is, what do you mean the real problem? Like has reality agreed to only give us one problem at a time now because if so, I would absolutely prefer my problem to go last on the list. You know, it's like kind of a doozy, but-- - Good point. - But unfortunately we have, we just have to deal with all of the problems. And I think the other thing I would say here is, one of the reasons the people in the companies are getting spooked is that the pace of progress is going very, very fast. We have seen not only the swarm outbreaks, but also claim solutions to the Millennium Prize problems, which are some of the most famous and hardest mathematical problems. And one question this on a lot of researchers' minds is how much harder is it to tell an AI, solve one of these Millennium math problems and have it succeed versus telling the AI, find me a more efficient way to train AI's and have that succeed. We know that the current AI training is not efficient. These AI's take much, much more power and data than a human. And so how much harder is it to find a more efficient AI method than it is to solve the Millennium problem? I think for all we know, we could start seeing these self-improving AI's become reality in the next three to six months, and then things could go very fast. I hope it's not true, but I think what a lot of these researchers are worried about is that we are running out of time for these out of control AI's that can self-replicate, self-improve, and get to these disastrous scenarios much more quickly than a lot of people realize. - Nate Torres, thank you so much. Guys, the book is if anyone builds it, everyone dies, Nate is co-author of that book, and it's very readable, it's very accessible. And so if you have questions about, well, what is a scenario where this could happen? And how should I be thinking about this? I really recommend the book to you because if you're first engaging in this debate right now, you may not have seen all the thinking that has gone into how exactly this could theoretically happen, but that thinking has been done. So I really recommend the book to people for more information, Nate. Thank you so much again for your time. - Thanks, Nate. I really enjoyed talking to you. Yeah, I enjoyed it too. (upbeat music) - Didn't catch the latest Roland Martin unfiltered podcast. - Here's what you missed. - There should not be a single law enforcement agency. It does not have body cameras. - It's real. Black farmers have been under attack. This is just the latest example of them just slapping DEI on anything. - It's raw. - And I'm sitting here going. They are playing y'all for fools. - Catch Roland Martin's daily commentary on the Black Information Network. - And download Roland Martin unfiltered on the IHOT Radio app, Apple Podcast. - Oh, wherever you get your podcasts. - Let's go! - The countdown is on to the most time. about Music Festival on the planet. Let's go crazy! Now, I heart radio music festival. Presented by Capital One, September 18th and 19th at T-Mobile Arena Las Vegas. 15 artists, two nights, one stage. Listen on I Heart radio stations across the country and stream it live on Disney Plus in Hulu. BTS. Benson Boone. Cardi B. Sierra. Google Doll. Kenny Chesby. Ariel. Laney Wilson. Lissera Phim. Legion Laser. Yes. Snoop Dogg. Lovey. Besmashing pumpkins. Weezer. Zara Larson. A special performance by Breakouts. Zara Stella Lefty. And never seen before artist collaborations. You'll never see again. One wild weekend. Are you ready? Buy your tickets now at AXS.com. On Big Lives we take a single cultural icon. People like Jane Fonda, George Michael, Little Richard. And we pull apart the story behind the image. Discovering forgotten interviews that changed exactly how we see these giants of our culture. We're here for the messy, the brilliant, the human version of our heroes. I'm Emmanuel Jochi. I'm Kai Wright. And this is Big Lives. Listen to Big Lives on the I Heart Radio app Apple Podcasts or wherever you get your podcasts. This is an iHeart Podcast. Guaranteed Human.