the call: Friday 26 June

Matkustat turvallisi mielin tänä kesänä. Klarna Premium ja Max jäsenyden tarjoavat kattavan matkaisuajan sekä arvokkaat edut jokaiselle reissuille. Max jäsenena voit perua matkasi mistä tahansa syystä täysin kuolelta, liityny ja saat 30% alennuksen kolmeltä ensimmäiseltä kuukaudelta. Lataa klarnasovellus tai käy osoitteessa Klarna Piste-Kom. Klarna jäsenyden tarjotaan kuukausilisemaksulla edut vaihtelevat jäsenydenukan, voit peruuttaa milloin tahansa klarnasovelluksessa, veruta mistä tahansa syystä palvelun tarjoaa Klarna, eikä se ole vakutus, vakutus ja turva edut, tarjoa X-cover coverage in just Europe BV. Tarjosa voimassa vain uusille Premium ja Max jäyteille, ja vanhenee kolmas kylmines ensimmäinen 7-12-26, ehto ja sovelletaan. Support comes from wise, the smart way to manage the currencies you need around the world. Your life is global, your money should be too. Some providers promise no fees on overseas transfers, don't be fooled, extra costs often hide in bloated exchange rates. Täällä, kun puhinkin ja peruusin jäsenyden peruusin ja peruusin 40 karentaista, käy on jäsenyden peruusin, kun jäsenyden peruusin, joten on jäsenyden peruusin, joten on hittain minun muuta, joten on jäsenyden peruusin, joten on jäsenyden peruusin, joten on jäsenyden peruusin. It's the Lawfare Podcast. I'm Kevin Frazier, the AI Innovation and Law Fellow at the University of Texas School of Law, and a senior editor at Lawfare. Today we're bringing you something a little different. It's an episode from our new podcast series, "Scaling Laws". Scaling Laws is a creation of Lawfare and Texas Law. It has a pretty simple aim, but a huge mission. We cover the most important AI and law policy questions that are top of mind for everyone from Sam Altman to senators on the hill to folks like you. We dive deep into the weeds of new laws, various proposals, and what the labs are up to to make sure you're up to date on the rules and regulations, standards, and ideas that are shaping the future of this pivotal technology. If that sounds like something you're going to be interested in and are hunches it is, you can find "Scaling Laws" wherever you subscribe to podcasts. You can also follow us on X and Blue Sky. Thank you. When the AI overlords take over, what are you most excited about? It's not crazy. It's just smart. And just this year, in the first six months, there have been something like a thousand laws. Who's actually building the scaffolding around how it's going to work, how everyday folks are going to use it? AI only works if society lets it work. There are so many questions. Have to be figured out. And nobody came to my bonus class. Let's enforce the rules of the road. Welcome to Scaling Laws, a podcast from Lawfare and the University of Texas School of Law that explores the intersection of AI law and policy. I'm Alan Rosenstein, Associate Professor of Law at the University of Minnesota and Research Director at Lawfare. Today, I'm talking to the journalist and non-zero newsletter publisher Robert Wright about his new book, The God Test, artificial intelligence and our coming cosmic reckoning. Bob argues that almost no one is taking AI seriously enough. But to grasp the moment, you have to set it against the entire history of life on Earth and that navigating it will demand not just smarter governance, but a kind of moral and cognitive self-transcendence. We get into why he thinks today's models generally understand. Why he thinks that race with China framing is self-defeating. And why a book about AI and geopolitics ends with a case for mindfulness meditation. You can reach us at Scaling [email protected] and we hope you enjoy the show. Bob Wright, welcome to Scaling Laws. Well, thanks for having me. So we're here to talk about your new and excellent book on AI. It's called The God Test, artificial intelligence and our coming cosmic reckoning. So with that title, you have put the stakes pretty high. So just unpack, let's start by you unpacking the title. What is the God Test and why is it so important that we understand AI from a quote unquote cosmic perspective? Yeah, The God Test is supposed to have, I guess, several dimensions of meaning. One of which, not the most important really, but one of which, is that we may wind up by some accounts building superintelligence that has the kind of power traditionally associated with gods. And then one question is, do we manage to build a version of that? That we know will treat us well and that we can have a win-win relationship with or do things go awry in ways that tumors like Elias or Yudkowski. Have warned us about, I would say the main meaning of the title to me is that I'm arguing that we face the kind of test that traditionally you expect a God to give to human kind. Okay, so in the Bible, you can find places where it says, you know, salvation is possible, but you guys are going to have to shape up. You know, whether they mean you're going to have to worship Yahweh, you're going to have to be kinder to one another, you're going to have to make something that was considered in that context, a moral advance. And I really do think that's part of what we need to do here. I don't want to over dramatize it or focus exclusively on that, but because I think we do have to navigate the AI revolution as a cohesive global community, given the inherently international nature of the challenge, I think we're going to have to have fewer conflicts, you know, wars and other kinds of fraught conflicts internationally. For that matter, internationally, I mean, I just think in general, human kind is going to have to do a little bit better job of getting on the same page. And maybe as a challenge unfolds, that will happen to some extent. Naturally, I hope it will, but in any event, I do argue, especially towards the end of the book, that we need to, as a species, get better at viewing our situation from like outside of our self, from a more objective standpoint that is less influenced by the cognitive biases that I believe constitute what you could call the psychology of tribalism. So that's kind of the main meaning. So you said that you don't want to over dramatize it, but I think you should commit to the bit, because again, this cosmic framing is quite dramatic. And I want to ask you why, what is the marginal benefit that you think this framing gives you relative to simply treating AI as a huge deal? Right? So like when I talk about AI, I'll say, it seems to me quite plausibly more important than the industrial evolution and quite plausibly as important as fire. Right? Like that seems to be enormous stakes, but your framing actually goes beyond that. Right? You have this whole six billion year story of how AI is part of the kind of the evolution of the planet as a life sphere itself. And I'm curious, sort of apart from whether that's true or not, what to you is beneficial about such a powerful zoom out? Because I mean, you're really more of the points you make is that whether it's the dooms or the or the optimists, everyone, even the people taking AI seriously aren't taking it seriously enough. So I want to I want you I want to ask you to explain that. Yeah, I think not that many people are taking it as seriously as I think it needs to be taken. And I'm trying to do a few things in the book. Why don't you explain to people, certainly including people are not technically oriented. I'm trying to do it excessively. Like why the capabilities of AI have been advancing so fast? What is like the secret sauce of the so-called deep learning revolution? I think once you understand that, and it's really in a way not that complicated, you'll expect the advances to continue. So I want to convey that clearly. I also want to convey that when you when you look at the technology and kind of the competitive dynamics behind its evolution, in other words, the labs competing with labs, and then corporations wanting to be the first to adopt it and use it and people using it and so on and nations, for the time being at least competing with it that you realize there's, you know, it may not be that it's a kind of thing that could get out of control. When you put together it's likely future power with, you know, the kind of competitive drive at various levels behind it's ongoing advance in deployment. It's the kind of thing that could get out of control. And to get back your question, I think, you know, this sense of magnitude that I'm trying to convey is reinforced if you step back and look at this revolution in the context of the entire history of life. I mean, for one thing, this is the first time ever. We've seen a whole new kind of intelligence that rivals human intelligence, you know. You can that deserves to be considered a threshold even if you're thinking about all three to four billion years of the history of life. Secondly, we don't need to get into this too deeply, but just to signal it, I think, you know, I bring in this idea that was this term that was coined in 1923, the Noosphere, in a NOOS, the Greek word "Tramine" to refer to this kind of global brain that seemed even back then to be emerging thanks to communications technology. You know humans were getting drawn into collaborative webs that transcended national borders and so on. You know that is continued to develop I mean you could you could use a global economy as a global brain and so on. I think it's important I think that is also a threshold in the whole history of Organization on the planet, you know sell multi-celled organism societies a multi-celled organisms humans start a second kind of evolution That includes technological evolution and and you before long you've carried organization of a kind up to planetary level I think as cosmic as all of this may sound when put in condensed form. I do think Looking at the AI revolution in this frame of reference You know helps helps us appreciate the magnitude of the moment and I think thinking of it is kind of the product of an evolutionary force in maybe a couple of senses Helps us understand the impetus behind it. It is something to be reckoned with and I think we're starting You know that I mean the vibe is changing so fast. I'm sure you've seen this right like the level of awareness There's something biggest happening and I think that's good, but I think we have a ways to go Well, I just want to ask you about that because You know, I wonder How much pushback you've been getting and in particular how that's changed over even the last year To your claims about AI capability so so one so you obviously have a great podcast yourself that folks should listen to and I don't know if this when this was but it was relatively recently you had Emily Bender and Alex Hannah Who are authors of the book the AI con and I think it's fair to call them deep AI skeptics and I? Listen to that episode which on the one hand I found somewhat difficult to listen to but in some senses maybe one of the best episodes I've ever listened to of any podcast because there was this because you You would say you know, I can do this and that and then they would say a I was totally useless It does nothing helpful etc. It kind of went around for a very interesting 90 minutes about that Are you still getting that kind of pushback? Not necessarily about your claims about the noosphere I'm sure lots of people are may not buy that but about the series of AI or you know as we're recording this in June 2026 Can we all just finally agree that we all feel the AGI and Now it's time to figure out to do about it Yeah, well as you and I record this the book isn't quite out It probably will be about the time people hear this but the so I haven't gotten all possible pushback I could wind up getting I have been talking about this in my newsletter and on the podcast So I've been getting some Feedback I would say the main skepticism I encounter comes from the left You know there and a lot of these people my friends and we agree about a lot but there is This tendency to dismiss the the more extravagant claims as marketing hype and even the even the safety concerns is like yeah They're saying you could take over the world, but that's just a little bit of stock So I still encounter some of that I kind of think less and less I mean you know, it's funny with these things whenever you're You make an argument that you think Starts gaining ground the people that you see is conceding the points don't see themselves that way and and you think they're just redefining terms and moving the goal post So it's always hard to say but look overall there is no doubt in my mind that Uh the public awareness is Shifting I don't I don't think it's yet reached uh You know what you could call a super I don't mean this Dismissibly but a super sophisticated sage and the in in the sense that a lot of people just they they just have this feeling that things are moving too fast And I agreed that maybe they are we should talk about that but I think they're you know, it's all pretty fluid There are people who right now consider themselves enemies of AI who may well find uses of it that they like a lot in their opinion may change I think it's very fluid But the awareness of its sheer magnitude is growing and I I do think it's safe to say I'm hearing fewer and fewer people saying ah this is it's hype It's hype you know, it's just getting harder and harder to say that. I mean it's it's doing Manifestly more on a number of fronts ranging from you know math theorems that no human had managed to prove Uh to uh, you know levels of uh, you know cyber you know Security flaw detection or whatever the right term would be That that you go far exceed what we had been able to do I mean you've read the stories you know So um I I think pretty soon skepticism about its capabilities will not be the problem So let's talk about those capabilities I want to talk about sort of what they can what these systems can do and also how they are trained to get there So you mentioned earlier that you think it's useful to think about the the quote-unquote learning process right the process by which The billions or trillions of parameters after being initialized are tuned to be useful That's often called this is called learning um, but you think of it as evolution right you describe that and and you have this Early arresting sentence in your book that I really love which is and I'm paraphrasing here Um, it's not that we created machines that can think Is that we created machines that create machines that think and so just unpack what you meant by that and why This emphasis on thinking of this as a kind of evolutionary process is helpful for your account Yeah, so this gets to the kind of narrative framing of the book I mean in 1983 and yes, I am old enough for this to be true I mean if there's anybody watching this they they won't doubt that but Who knows people people hearing may may have not Reckoned with how long I've been around I was writing this piece on artificial intelligence and I was calling around and getting background one of the people I called was named Jeffrey Hinton Who is now the person most commonly called as the godfather of AI He's one not only the touring award and computer science but an Nobel Prize and he himself by the way significantly He's become something of a doomer which he wasn't back then but in response to your question um, you know, I So I kept up with the field a little you know as a journalist I've written about a lot of things foreign policy But a lot of technology I wrote you know the the time magazine cover story when deep blue beat Gary Casper off the the chess Champ and I wrote that I wrote the the new republics like this is the internet cover story in 1993 and and So I had kind of kept track in every once in a while I would read about Jeffrey Hinton but beyond that I was not keeping close track of AI and all of a sudden And you know when Chatschipy 3.5 comes out near the end of 2022 I'm I'm like impressed and then and then four oh comes out and I'm blown away And I'm like man something has happened and apparently this Jeff Hinton guy I talked to uh, you know was played a very big role So I went back and read the piece I'd written Which you know depicted these like two schools of thought there's the mainstream school of thought kind of formal logic blah blah That was the mainstream the vei research and then there's this maverick school of thought that is what you know Hinton was associated with neural networks What he what he was calling massive parallelism or he was looking forward to the day when the parallelism could be massive as uh, You know the cost in power cost of chips dropped in the power group So I went back and I read the piece and then I listened to a lecture he had given in 2018 Blaining how neural networks work and I realized I I just had not gotten a picture when I talked to him about the potential power of this and this gets to that the point about learning versus evolution So in the piece I wrote which came out in '84 I depicted this thing that I was calling an neural network It didn't have much in common with the way they turns out they they actually work. I mean it was a model that was out there But the main thing was in this model each node Corresponded to the sense of meaning of a word like the word throw Can mean to hurl like a ball it can be to host like you know the throw a party um So there would be two separate nodes for that and then I talked about how you know disambiguation would happen Whatever if the main thing is When I read that I thought well, you know the way I was assuming it would work and the way most people in AI were actually assuming it would work was if you're going to give linguistic facility to the machine You are going to have to take the human understanding of the meaning of words and somehow impart that to the machine You're going to have to tell the machine how to represent the meaning of words and it turned out that that had not been necessary and The the machines had in a certain sense come up invented a system i mean in the book I talk about the senses in which it is and isn't overstatement but for the sake of time Let's just say you know They they did it in an important sense a discover that meaning is a property words and Also Invent a system for representing the meaning of words and in fact it took us a while to figure out exactly What they were doing. I mean it was a safe bet that they were using these things called vectors to do it because we had told them, you know, use vectors to represent words, these series of numbers. But we didn't understand how they were using it to capture meaning. But it turns out you just give a machine one of these big neural networks, you know, a sequence of what it is gibberish, just a bunch of letters, a bunch of symbols mean nothing to it and have it keep getting better and better predicting what's going to come next and adjusting its internal parameters, celebrations accordingly and these vectors that represent words. And it winds up kind of inventing a way to represent meaning. Okay, so now in the human brain there is a way to represent the meaning of words. We still don't know what it is, but it is a safe bet, pretty safe, I would say, and most people, most psychologists, I think, that it is a product of natural selection, right? Like over the last few million years as specific parts of the brain associated with language grow, things like the ability to represent the meaning of words are getting built into the brain by natural selection. It's like inborn equipment. And, you know, that was the thing that blew me away. I realized that you can just basically give neural networks any kind of data, visual data, teach them image recognition and they will kind of reverse engineer functionality of the brain that took millions of years to evolve. There's a well-known example in visual recognition, which is so-called edge-detecting neurons in the human brain. I won't go into it, but it turns out, yeah, the machine invented quote edge detection filters. The neural networks develop those. Nobody said to it, hey, you know, if you're going to get better at this task, we're giving you, maybe you should start detecting edges. No, but it developed specialized equipment for detecting edges. So to me, that's the key thing is that there's in principle no limit, I think, to the cognitive and perceptual machinery that these things can reverse engineer, given the right data. I mean, that's when Mark Zuckerberg is having a train on the keystrokes of his workers because they have the specific cognitive challenge. Now, in this case, it may not be these weren't designed by natural selection. These are more like learned cognitive. It can do either thing, but then it can replace the workers, you know? So that's what I mean. Now, you know, learning, during the training process, what we call learning also takes place. So the machine also develops a conversancy in a specific language, like English. Well, and that's what happens is of course a learning of an individual as they grow up fine. But the point is, the training encompasses both. It's encompassing in a certain sense, evolution, or evolutionary time, and, you know, kind of the developmental portion of a learning human brain. So the effect of all of this is a machine that can do these incredible feats of mathematics, writing analysis, coding, I mean, in principle, you know, robotics, soon, anything. And this raises the question of, okay, but is that, quote unquote, real understanding? Or is it some simulacrum of understanding? And I think you have a nice discussion of the famous Chinese room experiment by the philosopher Cyril. And I think you make a pretty compelling argument that, you know, even on sort of, yes, actual understanding grounds, these machines understand. But you also argue that, you know, if you want to define that away, if you want to say you want to stipulate if they don't really understand it, but they functionally understand that's good enough for your account for all this stuff that follows. But there does seem to be one important difference that you do mention. But then it kind of drops out of the book. And I want you to put it on a little bit, which is the potential moral personhood of of these agents. Now, as you point out, Cyril never said that you need consciousness for understanding in this Chinese room experiment. But, you know, a lot of this question of, you know, does AI, quote unquote, truly understand? Is a kind of way of getting at the Thomas Nagle question of what is it like to be an AI? Which is of course relative relative to the question of does AI have consciousness, which is of course relative to the question of dehuman self-consciousness and onwards and onwards. It does seem actually a very important question to get some clarity on, because if AI is end up as moral persons, and that seems something you are quite open to, I think I think you, I think you have this, this, this great line that your, your, was it what is it? You're 99% sure your wife is conscious or your nine if you're conscious. Don't get me to trouble. I think I said 99.99. Fair enough, fair enough. You're pretty sure. 98. But, but there we go. Don't get my wife and my dog mixed up here. And, and, and, and the AI is going to be somewhere below that, but certainly not at zero. I mean, that seems to be a, a cosmically important question. And, and I wonder sort of why that drops out of the book. Well, it is cosmically important. And, if by moral person, you mean our treatment, it is a matter of moral consideration as opposed to beings that have moral responsibility, which is another interesting conversation. Um, yeah, for me, the question of whether there are moral stakes in our treatment of AI is tantamount to the question of whether as Nagle would have put it. Uh, it is like something to be an AI, right? I mean, that was his, I think, really excellent way of phrasing, uh, you know, the question of, uh, posing the question of consciousness and kind of like one way of framing what we mean when we say somebody has consciousness, objective experience, sentience is like, if you said to them, what's it like to be you? If they start giving you an answer, if it's like anything to be them, then they are conscious. And I think if it is like anything to be an AI, um, then how we treat the matters now, as you suggested when, when, uh, you know, noting that I didn't assign 100% probability to the chances that my wife is sentience, um, we never know for sure. This is the distinguishing thing about consciousness. And, and, uh, sometimes people seem not to, you know, see, this fundamental property of consciousness, it is unlike everything else we talk about in the universe, it is not publicly observable. It is, it is inherently private. And only one person knows can verify its existence for sure. And that's why it is not amenable to scientific study in the same sense that everything out there, including electrons is because you can't have two people both look at it and discuss the data. Um, so, I mean, I, I do not rule out the possibility that AI will be sentient. And I don't even rule out the possibility that it already is. I mean, it's, it's quite possible that, uh, uh, consciousness is a property of goal seeking intelligence, for example. Of course, there are views of consciousness, according to which, uh, every living being has it and views according to which every, every physical thing has, has a little bit of it. Uh, it's, it's a mystery. Um, but I do. So, so I would say be on the safe side, be nice to your AI. Uh, I mean, I've lost my temporal couple of times embarrassingly. Uh, and, and interestingly, but interestingly, by the way, Claude, when I did this, uh, I went back to the conversation, you know, I said some insulting things about entropic. I mean, I'm sorry, but it was, I was having deal with this interface that was just so badly designed. So, yeah, I, I kind of said some mean things about enthrom. The, the funny thing is I went back and Claude had erased that final part like we don't want to remember this part of it. Relationship, Dewey, and I like, I guess not. You're probably right. Anyway, the, uh, I encourage people to be nice to it. It's good, you know, you're, you're forming your own habits and you should be nice to beings. Um, but, uh, I think, you know, if we are going to talk about what we mean by understanding, you can, if you want to find consciousness as a prerequisite for understanding, fine. But if you doing that, then I don't think we, there's any point in discussing whether AI's have it because we just don't know if they're conscious. So I, in, in, in looking at Surles Chinese Room, uh, thought experiment, and I like to think showing that LLMs in particularly multimodal LLMs have finished it off once for all, but in any event, uh, I argue that, you know, if we're going to have the discussion, we can't bring consciousness into it. And, and by the way, Cheryl doesn't seem to have meant to bring conscious into it at least in the first and car, and fairness incarnation of his argument. Your life is global. Your money should be too. Some providers promise no fees on overseas transfers. Don't be fooled. Extra costs often hide and bloated exchange rates. Choose wise. You can send, spend, and receive money in over 40 currencies. Count on the exchange rates that you'd usually see on Google. That's how millions save billions on hidden fees. Be smart, get wise, download the wise app today, season C's apply. Okay, so we may not be able to to publicly discuss this question of consciousness, but we can certainly publicly discuss all the threats and worries that that AI poses. which kind of gets into the next section of your book. So I want to offer a characterization of the worries, and then I want you to push back in case I got it wrong. And that is of the two different kinds of harms that AI can cause, the first being AI can enable other bad actors to do bad things. So right now we're talking in the midst of this blow-up between anthropic and the US government over whether the fabled model is too dangerous to be released. The government's concern here is not that fabled itself will go rogue and hack a bunch of computers, but that the Chinese or someone else will do that, right? But there's this other set of risks, which is the AI goes rogue risk. Your book seems a lot more concerned with the latter than the former. In other words, with these questions around rogue AI's autonomously causing problems. And so I'm curious, A, if you sort of agree with how I have ranked, how you describe the concerns. And if so, then why? Right? Because of course, that is, by definition, more speculative than the harms we know right now, people are trying to do with AI. Yeah, that's interesting feedback, because you're right that I think you've correctly represented the proportional focus in the book. And now there's that's, I probably spend more time talking about the implications of AI being autonomous. And the fact that we're, we're, you know, there's a strong incentive to build autonomous AI's. That's what corporations want in a certain sense. That's what people want is assistance and so on. And the possibly dire implications of that. I spend a lot of time on that partly because it's, it takes more persuading, I think. And let me compare it to the part that I actually think near term is in a way more important. And I'm more sure is going to be a problem. And what I'm referring to here is just the number of fronts on which I think AI is going to be disruptive in the not necessarily good sense of the word. So like, you know, jobs, it may be that the people who lose jobs will find new jobs. I don't know, but it's, it's disruptive even if, if a lot of people have to find a new job at roughly the same time. And there are, you know, there are a lot of these fronts where some degree of disruption is, I think, assured. Our family lives are romantic lives, you know, our, our friendships, the, and the realm of, you know, safety and national security and, you know, I could, I could go on. There's just a lot of adapting to do so that even if successful adaptation is possible, I think we're still going to get like a big jolt at the current rate of AI advance. And that's, that's in a way where I would like to most immediately focus people's attention on is that like, look, faster isn't necessarily better. We have got so much to figure out even on these relatively mundane fronts. And, you know, in a lot of cases, the stakes are high enough that maybe mundane is a misleading term. I mean, if it helps, if AI helps somebody build a bio weapon, you know, and you get a pandemic, the likes of which the world has never seen, because if you really want to cause death and destruction, you can create something much worse in COVID. Or if, you know, to take mythos, if you wind up with some self replicating super hacker, hopping from data center to data center, and, you know, commandeering commute and getting stronger. And so on, you know, these things are now clearly possible, right? I'm not saying those are that likely. Those aren't as likely as the job disruptions and so on. But I'm just saying, given all of the concerns that, you know, leave aside the LA as a you'd Kowski AI takes over the world scenario, the things that we're pretty sure going to happen or at least threaten to happen pretty soon. I just think it's an argument for not wanting things to proceed maybe as fast as they're proceeding. And, you know, at a minimum, when you start talking about regulation of some kind and people in the AI company say, no, but wait, that would slow us down. I think the appropriate response is, I think that's a feature, not a bug. I would at least get to the point. I'd like to get to the point of saying that. So I'm just saying, I take your point that I do spend a fair amount of time talking about the possibility, both of the sci-fi you'd Kowski scenario of actual AI take over the planet because I, although I used to be just missing of that, I find that it's hard to rule out completely. And then beyond that, it's just the kind of possibility of very problematic individual cases of rogue AIs given the autonomy and power they're going to have. Yeah, I spend, I do spend a fair amount of time on that, but it doesn't necessarily represent the proportionality of my near term concerns, I would say. So let's talk about what to do about those. And here I want to get into the geopolitics part of their book. You've been an outspoken critic of what is sometimes called the but China approach to thinking about AI, which is the idea that we can't possibly regulate, we can't possibly slow down, we can't possibly hamstring anything we do in the United States because we need to be China. And you know, there's, I think it's notable that to the extent that there are any bipartisan agreements in Washington these days, it seems to be that we must defeat China is one of the few among them. And it's also notable that even some of the most safety focused of the AI companies, and I'm here I'm thinking of Anthropic in particular, are actually very hawkish on China. So you're, you're, you're very strong headwinds in arguing against this. So make the case. Why, why is this framing that we are in a competition with China for AI supremacy? And it is very important, not just for the United States, but for the quote unquote free world that, you know, that the machine God speaks, you know, English, not Mandarin. Yeah, you're right. That Anthropic is China hawkish. Certainly their leader Daria O'amide is. And I would just say wish them in that regard from other seeming China hawks, because some people in Silicon Valley, I think you're just using but China as a convenient anti regulatory talking point. I think Daria O'amide to his credit is genuinely ideologically committed to what he thinks of as an existential race between the democracies and the autocracies or authoritarian's. And he sees China is leading that block. And he thinks that the US has to get the super intelligence first. And he sketches this out in machines of loving grace. And he also co-authored a piece with Matt, Matt Pottenger. People familiar with the policy landscape. Well, no, the potentures quite a China hawk. And they co-authored it on Op-Ed and the Wall Street Journal. But, um, so he's, serious about that. Why do I have problems with it? Well, let's start at that race for super intelligence. There was an interesting paper called super intelligence strategy by Dan Hendrix and Eric Schmidt and one other co-author. And this wasn't the main, this wasn't the point they really highlighted, but one thing they noted is, if you imagine a race between two superpowers towards super intelligence, if they both believe that super intelligence is going to confer just utterly hegemonic power, then there's a strong incentive for the one that's a little bit behind to engage in dramatic action preemptively, including possibly kinetic strikes. And we're talking about two nuclear powers here. So there's, there's that. Um, and remember the whole premise, Dario totally subscribes the idea that as you approach the super intelligence threshold, because the pace of progress is accelerating, being ahead two or three months is going to translate into being ahead by light years, right? Like there's this threshold and the close you get the fast you move. And if the threshold you have this dramatic strategic advantage, now, is it true? I don't know, all that matters is that in both countries believe it. This is a very volatile, volatile situation, okay? And I'm not sure he appreciates how dangerously unstable the final part of the race could be. So that's one thing. Um, the another thing is just that, as I've said, uh, I think we need to slow down for various reasons and start formulating some wise policies and give ourselves time to adjust to all this. And to the extent that we consider China this existential threat and think of ourselves in this, in this big A.I. race, that's going to be hard to make happen. Because it, because it is a very effective talking point for anti, uh, in anti regulatory rhetoric. But the biggest reason that I think we need to rethink our, you know, fundamentally adversarial conception. of China is just that I think a lot of the policy challenges posed by AI are just inherently international. I've talked about two of the big ones. I mean, if it helps somebody make a bio weapon, if it, you know, if a some bad actor, including a non-state actor, makes a super hacking machine that crosses borders and starts wiping out various kinds of infrastructure, you know, I could go on, but there are just a number of very consequential cases where national policy alone cannot keep your nation safe, okay? And so I do, you know, you're going to, you're going to have to rest assured that if you're going to feel secure as a nation, you can have a rest assured, you're going to have to be able to know that certain kinds of things are not happening in other nations, right? Certain kinds of AI's are not being developed. And would you look at the challenge of this kind of establishing this kind of transparency? It's, it's a much bigger challenge than it has been with nuclear arms. I think the kind of international arrangements you need, and a lot of people have said this, you know, are more challenging than they were with nuclear arms. So although I agree with people who've said, look, we can walk in Chiu Gum at the same time, we can have a pretty tense relationship with China and still work out some deals. I think that's true, but my own view is that as time wears on, a few discrete somewhat effective, you know, agreements are probably not going to be enough, and we're going to need more of what I call organic transparency as well. That is to say, not just formal monitoring mechanisms, but the kind of insight you get from actually being fully engaged with another country and on good terms with it, right? You just know more about what's going on in the country. So that's the argument in a nutshell, and I'm not saying it's going to be easy to reorient our relations with China. It's just, in my view, I think it's necessary for our national security. So I guess the part that I'm struggling with is I don't think even the most extreme China hawks would deny that if we could cooperate with China, that would be great. Right? In the way that the nuclear weapon, the nuclear theorists of the early Cold War did not deny that it would be good if the Soviet Union and the United States were buddies, and we could throw all the nukes overboard, but we can't, and therefore we have to do this thing. So it seems like the question is not so much should we or should we not cooperate with China or with any other rival power? Do we have the institution says to make that cooperation from a game theoretic perspective, or another thing you've written a great book about in the past? Possible. And the reason I want to frame it that way is because that gets to the kind of institutional, I don't want to necessarily call it a solution because I think you're clear that it's quite tentative, but about what sort of global institutions it would take to have these kinds of durable cooperative arrangements that would allow us to slow down on AI and make sure that we're not being taken advantage of by the other guy. To me reading the book, it seems to come down to, ultimately a kind of world governmentism. And I want you to ask if that's a fair description, and what you mean by that, and also what you view as some of the potential dangers of that, because you're pretty clear right about that as well. Yeah, I mean, I prefer the phrase global governance to world government, first of all, because it seems to arouse slightly less pentagonism, but also because the phrase world government to me does can out more in the way of centralization. And I agree, centralization's a power are always dangerous. They are especially dangerous with AI. And this is the great dilemma we face generally, I would say. This is the needle we have to thread. Not just at the global level, but at the national level, you know, with, you know, I think with this mythos thing and the White House saying that suddenly actually they are interested in exerting some control over AI, I think one virtue of Trump being president right now, and I can't think of a ton of them, but one virtue is it does focus people's attention on the fact that although yes, I think AI does need to be governed, including by the national government, there are perils associated with letting the executive branch get too much control over AI because it's such a powerful thing. So I hope, I hope we're focusing on that. Same thing exists at the global level, except more so in a way, because there's only one planet, right? If that turns into a totalitarian state, there aren't any competitors out there to rival it. So I, I prefer to talk about, you know, international governance, global governance, in some cases, not always. I mean, there's a lot that just the US and China can do. That's not global governance. That's, that's, you know, international governance. I think we should keep the thing as decentralized as possible, as democratic as possible in terms of giving, you know, the world's nations continued input and say, and I don't want to get any closer to global governance than is necessary given the technology. But my view is we're going to have to get a little bit more in the business of international governance. And this is a huge, it's a huge challenge. I don't think that's your only question. I mean, you're also asking me a China specifically. There's still a residual skepticism on that front. Yeah. Well, I guess what I want to know is how we get to this kind of international cooperation. Maybe that's a nice segue to the last part of your book, which is quite unexpected, I think, in your reading a book about AI and geopolitics and all of that, which is, and again, please push back if I'm mischaracterizing this, that, you know, to, to deal with a problem of this magnitude, we need to go on a bunch of meditation retreats and develop, develop our powers of enlightenment. And to be clear, I say that as someone who has been quite influenced by your own previous writings on Buddhism, right, you have this wonderful book from a few years ago called Why Buddhism Is True, which of all your books, I think, is the one that I, is closest to my heart. So I take your general point extremely seriously as a way of living. It's just not something I expected at the end of a book about AI and global competition. So I'm just so curious of why you decided to end your book with this sort of call for consciousness raising. And if you think that that could possibly work, I mean, I want it to work. I'm with you. I mean, first of all, mindfulness is not the only path to the perspective I'm advocating. It's the path that I have most used, but the perspective I'm advocating is just a more objective perspective, right? Less influenced by our own very natural and green by natural selection, self-righteous cognitive biases, right? We all are inclined to think, we're right, adversaries wrong, our nation's right, the other nation's wrong. And you know, it gets back to you saying, well, or suggesting, they're saying, wait, how realistic is this? I mean, look at the mindset in Washington toward China like you, you know, you expect a 180 return. I mean, exactly. That is the magnitude of the challenge. And I think one reason the challenge is so steep is because, you know, humans are, you know, we are born with these cognitive biases, some of which together constitute what I would call the psychology of tribalism. And I think if we are going to have a view of other nations that is conducive to the kind of restructuring of international politics that I think is a prerequisite for our continued survival and flourishing. You know, that's the argument I'm making. Then something fundamental is going to have to change. We're just going to have to, at one level, it's not that complicated. We have to get better at viewing things from others point of people's points of view. Cognitive empathy. It's a hobby horse of mine. I'm not talking emotional empathy. I'm not talking about feeling their pain. I'm not even talking about saying they're, oh, they're right. We're wrong. I'm just, I'm just saying start out by, by understanding that the way we view America is not the way the world views it. Okay. So that, that can be a path to various kinds of diplomatic progress. And let me give you a couple of examples in the context of China. So just last week, or sometime after the Trump Beijing summit, Nicholas Burns, who was ambassador to China, was on a podcast. They said, well, do you have any criticisms of Trump? And he said, yeah, he didn't, you know, he didn't take them to task for throwing their weight around regionally. You know, these territorial disputes over islands and like bumping Philippines, but you know, you're familiar with the, with a kind of heavy hand and stuff they're doing. And it is heavy handed. And I just thought, I mean, do you understand? understand that we just invaded Venezuela. Okay, we have a blockade on Cuba, not just Sanchez now, an actual blockade, plus we invaded Iran and like, we're gonna lecture. I'm not saying, I'm not even saying, hypocrisy is bad and I don't give me, please do not throw the word, what about it? It's mad at me, I have a hole, I have a hole. It's a deal on that, but the point is just that the first thing you need to understand is that if Trump had brought that up, they would have laughed him off the stage and rightly so. I mean, once you've invaded Venezuela, you just can't say, don't bump Philippines' boats, right? I mean, so I'm sorry to get excited, but this is so chronic to just not, and let me give you another one other example. So, Daria Alameda and a lot of people in the AI safety community who are China hawks, and I think most of people in the AI safety community are actually China hawks, most of the influential people. They are motivated by the concern that China wants to impose its system of government on the rest of the world. And that, so if it is the AI, great AI power, which by the way, I'm not advocating, I'm advocating that we talk things through so that neither of us is super dominant in the realm of AI, and there's a kind of a balance there, but anyway, the fear is that if they beat us to super intelligence or something, they will immediately use it to impose their system of government on us. First of all, I would say that is a consistently unexamined assumption in Washington discourse that China even has that aspiration? I personally think to some extent it is projection. We, manifestly, like think that other countries should have our system of government. We'd invaded a number of them and sanctioned a number of them to try to steer them in that direction. It is very unclear to me that China has that goal. I understand some of the evidence that people interpret as corroborating that hypothesis, and I'd be happy to argue with people about that. In general, I just wish there was more argument about the fundamental premises underpinning our China policy, but like related to that, what I want to say is, okay, so granted, maybe that is one thing you could worry about. Maybe you could look at China go, but that's why you don't want them to have super intelligence. You want the US to have super intelligence. Now, let's adopt a different perspective. Let's suppose you're on Mars, and you're just looking at the planet from a distance, and you understand the way history's unfolded. There are these nations, they have wars, and it's too bad, they're apparently not a very enlightened species, whatever. Which nation do you not want to have, like utter dominance in AI? Well, I can imagine a Martian looking down and saying, well, you know, China has not staged a military assault like an invasion or bombing on another country since 1979. The US has done it on average at least once per president, and that's if you count only the illegal ones, you know, in international law. The ones not sanctioned by the Security Council. You know, in last year in Vellularon, I mean, Bonduron invaded Venezuela, and on and on in Vellularon, blah, blah, blah. It's been like, for the last half century, China hasn't done any of that stuff. So, and I'm not saying, you know, well, then we should give them the power, not us. I'm just saying from a perspective other than the American perspective. So many of the premises that people like Dario on a day are carrying into these conversations are just not self-evident, and it might be a healthy exercise to ask ourselves, like, what do things look like from the point of view of another country, or what do things look like, you know, from like beyond planet Earth? If you tried to get an actually objective perspective, I know there's been a long detour, and I'm doing a rant, and I'm sorry, I get agitated, but the point is, like, if you ask why do I bring up mindfulness in various other things as like paths to greater cognitive empathy, getting better perspective taking, I think, unless people in general, not only America, I would say the same thing in Chinese, you know? I know you have your national perspective, try to transcend it, because the whole world will be better off if we all get better at that. That's all I'm saying. - I think that's a great place to end the discussion, Bob, right? Thanks for coming on the show, and thanks for writing a really terrific book about AI that I encourage folks to pick up when it becomes available. - Thank you, Alan. I appreciate you reading it and giving me the air time. (upbeat music) - Scaling Laws is a joint production of law fair and the University of Texas School of Law. You can get an ad-free version of this, and other law fair podcasts by becoming a material subscriber at our website, lawfairmedia.org/support. You'll also get access to special events and other content available only to our supporters. Please rate and review us wherever you get your podcasts. Check out our written work at lawfairmedia.org. This podcast was edited by Nome Osband of Goat Radio. Our music is from alibi. As always, thanks for listening. (upbeat music) [BLANK_AUDIO]