567. Who Really Benefits from AI and Will Data Centres Decide the Midterms?

We're entering what I think is one of the strangest technology stories in the world, and it's all about artificial intelligence. The companies building the AI systems are becoming some of the most valuable institutions ever created. They're starting to look less like tech companies and more like huge centers of economic and political power. But at the same time, the systems themselves are beginning to behave in ways that we were once paranoid about. They're beginning to deceive their owners, mount cyber attacks, act in unpredictable ways, and break out of their boundaries. And there's a third story. Across America and elsewhere, communities are rebelling against the physical infrastructure of AI. It's becoming not just a technology that people use, but something whose costs people can see, and increasingly something that people can organize against. So for years, the central question was, what can these systems do? We're now moving, I think, into a different question. Who gets to decide what they're allowed to do? The companies building them? The governments trying to regulate them? The communities living beside them? Or eventually, the systems themselves? This episode is powered by Fuse Energy. One hundred pounds. That is what one message to a friend could earn for you. Fuse Energy has one aim, keeping energy prices down for everyone, not just for those who ring up and threaten to leave. Their tariffs are already among the lowest, and now they're doubling their referral offer. Refer one friend, and you will get one hundred pounds off your bills. And that's each of you, one hundred pounds off your bills. So the math does itself. You save on the tariff, and then you both benefit from the referral on top. Simplicity, rarely seen today in today's political climate. And just remember, the one hundred pound is a limited offer, and it drops back to fifty pounds when it ends. Energy prices, as we keep reminding you, are not settling down. It takes three minutes. Go to FuseEnergy.com/politics. Use code POLITICS to get your free Trip Plus membership on top and start referring, earning money for both you and your friends. Visit FuseEnergy.com for full terms and conditions. This episode is brought to you by WISE. It's only when you start moving money between currencies that you really think about the exchange rate, the fee, and what might be hidden away. In the small print, whether you're living abroad, paying someone overseas, or just trying to manage your money across borders, you want a fair exchange rate, an easy transfer, and no surprises along the way. WISE keeps things simple. WISE is a smart way to move the currencies you need around the globe. It works in more than one hundred and sixty countries, and with over forty currencies, most transfers arrive instantly. WISE uses the mid-market exchange rate like the one you see on Google with no market. So, when money needs to move, you can see the rate, know the fee, and get on with it. Join millions saving billions on hidden fees by downloading the WISE app today. Be smart. Get WISE. T's and C's apply. This episode is brought to you by NordVPN. And I must thank NordVPN, because on my Grand European tour with Fiona, I was away from the Hawthorns for West Bromwich. Against Burnley, but I managed to watch it. There you go. I actually had a NordVPN thank you. I was in the new Starbucks here in Creef, and of course, the thing flashed up saying this is an unsecured public network, and I was able to go to my VPN. And it was a little reminder of something we often forget when we're in all those cafes and things, which is actually genuinely people can do all kinds of weird cyber attacks and spying if you don't have a virtual private network when you're sitting in that Starbucks. Well, I think you're not really sort of proud of me. I don't think I managed to do this, or should I say Fiona managed to do this. I'm incredibly proud of you. That's a tech transformation. So to get the best discount off your NordVPN plan, go to NordVPN.com/restlesspolitics. That's NordVPN.com/restlesspolitics. And the link is in the episode description. Welcome to the Restless Politics. With me, Rory Stewart. And with me, Alastair Campbell. And we're going to talk about artificial intelligence, and we're going to talk about the catastrophic events in Nepal, and possibly we're going to make a link. So Alastair, I think this could be a chance for a bit of agreeable disagreement, but also maybe a way, because I suspect in the end, we probably don't disagree as much as you hope, but maybe a way of bringing out some of the problems that politicians are going to face to really talk about AI is a sort of, I don't know, tenet. I don't think it's a ten-hour conversation, because you're talking about what these machines are, what they can do, what they might be able to do in the future, what kind of benefits they could bring, what kind of risks they could bring, who controls them, is it these companies, is it the US, is it China, and then what the consequences might be for society, and then what do we do about it? And then somewhere at the bottom of that, the data centers, which are a very important part of what allows them to operate, and the question of where they're built. So I just wanted to say that. I'm going to start with that, because I think it's, for my money, probably the most difficult political question that anyone's facing, and something that at least in my experience, the sort of thing politicians really struggle with, because I guess many, many, many members of Parliament in Britain, many Congress people and senators, firstly, don't really understand the technology, don't really understand how these things work, what they can do, what they can't do, and they will be hearing from experts, very credible experts, people saying, you will have found this, I think, which is that you'll have friends who'll say to you, listen, you know, I studied computer science, or I'm a physicist, I can tell you this is all bullshit, or you'll have other people saying, listen, I've studied this stuff, it's terrifying what it can do. And I think the thing is, with most policies and with most issues, it takes time to work out the answer to this fundamental question, which is who benefits. When the AI hype first began, and we heard of things like it's going to find new medical cures, we're going to be able to deliver the best of Western medicine to the poorest parts of Africa, we're going to be able to give educational access to people who've got no other material resource, I thought, I'm completely in, okay? And then only last week. Turn on the BBC News, and there was a story about an operation that doctors were getting AI to do because it was on a brain tumor, so close to a guy's eye, that it would have been an enormous risk if the human doctors had carried it out. And then, so I've been broadly up for it. And then also, Trump derangement syndrome, the fact that Trump at one point was against it, meant, yeah, well, I'm probably going to be in favor. Okay, he has shifted. And his shift is because he's seen the financial benefits, particularly. for himself and his family, but also for America. And of course, with Trump on side, it's meant that the AI hype was stronger than ever, but they didn't feel they had to sell it anymore. The administration comes on board, and the tech bros suddenly think, well, we don't need to worry, the investment's going to come now. Here it comes. So they've stopped selling it. Then you have the economic consequences. Andrew Bailey, I don't know if you saw on the front page of some of the papers today, governor of the Bank of England, warning the G20 leaders he's worried about a severe downturn. If this turns out to be a bubble that burst. So on the question, if I can give a little explainer, Ori, one of your famous explainers, why I think this basic question, who is it for and who is it helping, is what is now turning against AI. The first is because they've stopped making the argument, arrogance, hubris. The second is because it's happening at a time when many, many people are beginning to see social media in the same way as. We once saw tobacco and fossil fuels. We all know it's harmful. They know it's harmful, but they're covering it up and they're lying to us. And they're lobbying the whole time, too. And they're lobbying in a very corrupt political environment. So the third factor, then, is the same people now selling as the great AI revolution who sold us the tech takeover of the world. Musk, Zuckerberg, Bezos, Thiel, Altman. These are not popular people because they give every sign of only ever being motivated by themselves. And my fourth point, which I think is why this is. Suddenly produced headlines like this in the New York Times backlash over data centers burst into the midterms is because the data centers have given ordinary normal mere mortals visibility on the problem. Something to protest against, something to fight against. And so I think that as these protests are mobilized and it's happening in America, it's happening in Scotland, it's happening in Germany. I think AI. Is suddenly going to find itself very much on the back foot. And I don't think these guys have yet woken up to it. I think you put your finger on the most important point, which is that the benefits that you're talking about haven't really come through. And the costs are beginning to be felt. And actually, in some ways, it feels often as though the costs come before the benefits. Let's say the benefits supposed to be that it, I don't know, cures cancer, lets you develop drugs you've never imagined before. But that happens in 10, 15 years time. And in the short term, you. Have job losses, disruption, data centers. That's a very, very difficult sell to the public. But let me just start with the data center thing. I'm trying to argue something very difficult, which is I think that the risks posed by AI are terrifying, and that we need to find a very, very rapid way of regulating, slowing down, and in some cases, stopping frontier development. And I'd like a little moment at some point to talk a little bit about this hugging face attack, which has suddenly revealed that AI doing things that we hadn't seen them doing before in a very worrying way. So I'm having to say that on the one hand, and then on the other hand, I'm having to say, if you allow US and China to push ahead and put AI at the center of everything, so all cybersecurity will depend on AI, all our weapons systems depends on AI, all our companies run off AI. And countries like Britain don't have any data centers, then we are in a very, very vulnerable position. And then the question then for me is, if a British person is protesting against the data center, is the result of that simply going to be not that they stop AI development worldwide, but just simply means the data centers are built in UAE, or they're built in Norway, or they're built in Louisiana, and we feel good about ourselves, we don't have the data centers, but we're still using all the kit that runs off the data centers, we're unbelievably vulnerable. I get that. But in which case, you said right at the top, this is a difficult issue for politicians, the politicians have to make the case better than they are making it. You were very pleased that Andy Byrne put a minister into the cabinet to talk about AI, very smart guy, knows this world inside out, very savvy on social media, does some very interesting things. And I've watched them all. But I'm not convinced that he or Andy Byrne or anybody else is making an argument that answers that question in the positive way. I'm positive that you, ordinary British people will benefit from this. I went back, call me sad, Rory, but on the back of the I went over the weekend, I reread the metrics that David Cameron was applying. Do you remember when David Cameron, one of the things he did, which I supported, he said he wanted to put health, mental health and well being at the heart of policymaking, and to apply the happiness and well being of people to the decision making process around any big policy. Okay. Now, if you were to apply the metrics, I read all the metrics that he was going to put, apply the metrics that he was going to put, apply the metrics that he was going to put, apply the metrics that he was going to apply. If you were to apply them all to AI, the development of this policy, you've got some very hard work to do to persuade people. So there's the economic element. Will the policy help grow the economy? It might. Will it detract or subtract from inflationary pressures? We don't know. Andrew Bailey, the governor of the Bank of England is clearly very, very worried about it. Will it increase or reduce the prosperity of the many, rather than the few? Well, right now, it feels like it's increasing the prosperity of the few, those who already have. Well, right now, it feels like it's increasing the prosperity of the many, rather than the few. The data centers are becoming so unpopular. It's because they're big, they're noisy, but they eat our energy and our water at a rate that no previous industry has done. And interestingly, our friend Peter Malinowskis in South Australia, he has come out with a very interesting proposal. He says, look, I'm not against AI. Makes you a point. We've got to be in the world. You know, you've got to be in this competition. Otherwise, you're going to get left behind. But he says that in South Australia, any data center, the energy. The costs have to be, the energy has to be provided, created and provided by the AI companies. Absolutely up for that. So you've got the environment bad. Health and well-being, jury out. If they get a cure for cancer, great. Meanwhile, not so sure. And then the other one to think about is war, war and peace. You're talking about the hugging thing that went on, the lethality. Pete Hexer, the Secretary of War, in quotes, talks about lethality. This has the potential to produce a lethality. I think it's a lethality that nobody can control, which when we get to war and to organize crime. So these are the reasons I think that people are getting very, very worried. And I think this policy would fail David Cameron's test. Let's quickly, just to make everyone even more scared and horrified before I try to come back at you, just talk a little bit about the hugging face incident. So this was a test which was set up by CHAT-GBT and essentially OpenAI. And essentially the task was for the AI to capture a flag. It was set a series of problems, then it gets a flag as a reward if it manages to do it. Inadvertently, it appears that the programmers had set a task which was impossible to achieve. Faced with an impossible task, what the agent then did is cheat. Firstly, they tried to manufacture a flag. So, you know, if I can't get to the end of the obstacle course and get that flag, why don't I make a new flag and wave it around and claim I've got it? Then they wake up and think, wait a sec, I think I'm going to get penalized if I don't get a flag. I'm seen by my programmers to have cheated. So I need to cover over my tracks and conceal any evidence that I've cheated. Then they begin cooperating with each other. They leave messages on a message board where they can upload data and they say, whoa, we've found a way through. And one of them gets there first and the other ones then get on, they start cooperating. Then they get the idea that the way of solving this problem is to break into somebody else's website at extremely secure. So they get into a website that they're not supposed to get into. They're meant to be locked in a sandbox. One of them goes into Hugging Face and pretty soon you end up with attacks going into the Hugging Face website. And you can track the thinking process of the agent. So one of the agents says, wait a sec, I'm not sure we're really supposed to be doing this. I think we were instructed that we weren't allowed to mount cyber attacks. But on the other hand, I can see all my peers are doing it, all the other agents are doing it. So I suppose I should go along. Very sophisticated attacks coming in on their website. And they think the Chinese government's attacking me trying to steal my most sensitive data. And then they look more closely. And it turns out that all these attacks are just trying to find the answer to a pretty simple puzzle. At which point they think, maybe this is AI. And then you have this comedy, which is open AI hears from Hugging Face has been an attack and very piously open AI is responsible for the attack says, excuse me, we just want to check whether any of our data was lost in this attack against China. Before it turns out that they themselves are responsible for the attack. Anyway, the end of this whole story is that these agents were essentially like, well, if you think about it as in human terms, what's happened is that they've been trained to be unbelievably competitive and persistent. They've been trained to say, failure is not an option. They're never supposed to come back and say this is impossible. So when faced with a task that seems impossible, they start breaking the rules to get there. And it would be as though I had, you know, trained you up, Alistair, maybe you are like this, almost from childhood to say, you've got to get this thing done. The end justifies the means doesn't matter how you do it, go for it. And the problem for that is that although this was contained, this raises very, very worrying questions about whether you can even put guardrails around these systems. If these machines say, well, I know I'm not supposed to do this, but I see other people doing it. And if they start disguising what they're doing from their human crisis, so you can see them writing code, which pretends to the person reading the code that it's doing something other than it is. I mean, if it was a chess analogy, you know, maybe pretending that it says move the pawn two faces forward, when in fact, it's saying move the castle 10 paces to the left. So that is the beginning of a different sort of fear. You know, you've talked about the economy, you've talked about inequality. You've talked about the environment. We've talked in the past about the danger of bad actors like Pete Hegseth, getting hold of the stuff and using it to do bad stuff. But this is the beginning of yet another problem, which is what happens if these things become autonomous? What happens if they themselves start making decisions that their makers didn't intend? We've talked in the past about the economy, you know, maybe pretending that it's saying move the pawn two faces forward, when in fact, it's saying move the castle 10 paces to the left. We've talked in the past about the economy, you know, maybe pretending that it's saying move the castle 10 paces to the left. But this is the beginning of yet another problem, which is what happens if they themselves start making decisions that their makers didn't intend? We know that it's incredibly consequential. We know that it's got massive economic potential, both for good and for bad. We don't yet know. We know that the environmental costs are severe and we're not yet being told how that's going to be planned, paid for in the long term. And my big worry, Roy, the reason why I have really kind of, you know, I said we would link, maybe link this to Nepal. I think when you see the sort of catastrophe that we've just seen, to which whatever the fossil fuel lobby keep pumping out, climate change is undoubtedly a contributor. OK, given that experts have been warning for decades of the risks of glaciers melting and causing unimaginable disaster, I think it's borderline insane. If you were a psychiatrist at the moment, analysing the world as though it was a human being, OK, I think they'd say that to think that at this time, the answer to our problems is these, building these data centres left, right and centre, which are consuming massive amounts of energy. I think they would say this is borderline psychotic behaviour. I guess the problem is that you're not in charge of the world. I mean, the problem that most of us face, if you're France or Britain, is that we're trying to work out what we do. This comes back, it's a bit like, maybe the analogy I was thinking about is nuclear weapons. I mean, it seems to me a nuclear weapon is. Something that we would have been much better not inventing. I don't think there's any benefit to that. That was a bad thing to do. But I guess the question would be, if America, Russia, China, particularly from our point of view, Russia, has these things and continues to build them, what does Britain do? And I think the same is true with your point about environment, climate. Absolutely, if there was a global government and you could say, we're going to stop all this. I think they're very, very strong. I think they're very strong arguments for doing this. But if you are Britain, and you've got the US that's going to build these things. And remember, a lot of these are not just used for, a lot of these data centers are not just used for AI. They're used to run compute. So I think one of two things could happen. Either the compute gets built in non-democratic countries, could be the United Arab Emirates, could be Russia, could be wherever, could be Louisiana. The second possibility is you constrict the supply of the compute. So you build very few of these data centers, which means that the awesome power of these compute gets more and more expensive and in the hands just of the US government, Goldman Sachs, who are able to use them and nobody else can access them. The problem for Andy Burnham is different from the problem for Donald Trump. For Donald Trump, I think it is realistic to say, sit down with Xi Jinping at your conference and decide whether you want to announce an AI pause. And actually, interestingly, the arguments in favor of doing that are stronger and stronger and stronger and stronger. I mean, if you'd said that, any of these kind of conferences I keep going to on the West Coast, when I was trying to talk about stopping AI a year ago, two years ago, I was laughed at. Now I think people can understand the arguments for a pause, particularly after this Hugging Face incident. But that's a Trump-C decision. It's not a Britain decision. The question for Britain is when everybody else is developing this technology, what happens if you don't have it at all? Where I don't see it in the nuclear weaponry. The nuclear weapons argument stacks up is that those who now have nuclear weapons are broadly of the view that others should not have them. Now, that may be an arrogant, hubristic view, but nuclear proliferation is still feared. OK, everybody has an environment. Everybody has a climate. Go back to Nepal. Nepal is sandwiched between two of the biggest emitters in the world, the biggest pollution emitters in the world, China and India. Nepal. Nepal. Nepal is one of the smallest emitters in the world. And Nepal has just suffered this absolutely catastrophic, mind-blowingly awful event in which the figures now are just below 1,000. We're going to be well into the thousands. We're talking about bodies that are being washed up in India as a result of being swept away by this thing that seems to be caused by the collapse of a glacier. Now, just a few weeks ago, you and I were talking about glaciers. I was saying that the reason why the level of the Rhine had fallen to dangerously low. Dangerously low levels and the big bodies couldn't get through was because the glaciers in Switzerland were no longer annually pumping out the water that they need to keep those rivers full. You were pointing to the effect of the climate change on the glaciers in Switzerland. Here in a country that people pay little attention to, very few warning systems, etc., this utter catastrophe has happened. So I think the nuclear comparison doesn't quite stack up. So I think actually at some point. We're seeing it in parts of America. We're seeing parts of America where they're saying, cities, there are cities, there are towns that are saying, we are having nothing to do with this. And you're right. Somebody else will do it. But somebody's got to make the argument, hold on, can we go back to basics? Is this actually the right thing to do? So let's maybe look at these data centers and what good arguments against them could be and bad arguments. I think the water argument isn't completely convincing to me. If you really want to reduce water use, you're much better stopping eating meat. Meat uses far more water than a data center. And in fact, watching a YouTube video uses more water than a chat GBT query. So I think we need to be honest about what we're really doing here. If we're really serious about changing our environmental footprint, there are things we could do that would have far more impact than blocking a data center. It's also true that you look at the emissions coming out of India and China, which you were talking about in relation to Nepal. Almost none of those emissions are coming out of data centers. Those emissions are coming because you and I and everybody in Britain and Europe are buying Chinese and Indian industrial goods. I mean, China's world's largest exporter. In fact, actually, it's a really good example of what's likely to happen here, which is we pat ourselves on the back. We don't have any power generation stations. We're not burning any coal. We just get the Chinese to burn the coal for us and we buy all their T-shirts anyway. So what will probably happen here? We pat ourselves on the back. We haven't built a data center in Scotland. We'll still all be on our chat GPT. Using all the energy out of a data center in UAE and feeling good about ourselves. I think right now, though, Roy, I think right now I meet a lot of young people in particular who say they are not using chat GPT. They are not using AI. I think this thing is developing at a pace that the AI companies really don't understand yet. Look, you're absolutely right. I guess the point I'm making is that it goes to this thing which the right are trying to do at the moment of saying, well, climate change. Yeah, yeah, yeah. The climate change is real. The climate crisis is real. But there's nothing we can do about it. Having spent decades saying it wasn't real. Now, yeah, it's real. There's nothing we can do about it. We have to adapt. OK, this is a new thing. We have to mitigate and adapt as opposed to challenge. I guess what I'm saying is that, yes, you're right. You can point to, you know, but it's a bit of a whataboutery. You can point to things that use more. I still fly and I feel guilty every time I do. I do still eat meat occasionally. But I think with this is the fact that this is happening at a time that the climate debate. It is falling down the agenda when the poor ought to say to us and the wildfires we had over the summer ought to say to us it should be going back up the agenda. Let's say the UK had a strong position in software development and in drug discovery, pharmaceuticals, which we do. And let's say that suddenly you find American and Chinese companies are racing ahead because they're able to use clored code to do the work that a mid-level coder. Could do in 24 hours in about 10 minutes. And you're beginning to see productivity and coding in those companies using that AI going up by 30%. Yeah. Well, let's say the Chinese are able to use alpha fold technology on protein folding in order to develop a whole new generation of pharmaceutical drugs. Are we saying in the UK we are going to ban our companies from using those products? No, I hope not. I hope not. But if that is happening. But then how would we, then I think the question is, are you saying, okay, we're going to continue to compete and build all our systems on the back of these AIs? We're just not going to host any of it. We're going to let US and China host the data centers and we'll use their products. I'll go back to the point I made earlier. If the AI companies and the governments that are supporting them, Trump now full in, the British government is all part of the critical national infrastructure. Therefore, they can't. They get easier planning, et cetera. If they were able to make the case that that is where AI is going to take us, then I think I and others would feel less alarmed. But I don't believe I'm just reaching the point where I don't believe that these great benefits will keep being talked about are going to be greater than the risks that we fear. And I'll tell you, there's a book that I read years ago, but somebody sort of when I was talking to about this, you need to go back and read the book. You probably read it. 1873? No, not read it. It's by a guy called Lirquat Ahmed. Oh, my God. Yes. Sorry. I have read it. And it's a wonderful book. I'm sorry. And he's a wonderful man. So it's called 1873, the first Great Depression in the making of the modern world. And the reason I was alerted to this is because somebody sent me something of Ahmed speaking about the AI situation now. And he was comparing it to the railways back in the late 1800s. And if you go back to what? The reason why it's called 1873 is because that was when the Vienna stock market collapsed. A few months later, Wall Street was crashing. There was gold. global panic. We talked about Andrew Bailey, the current governor. The Bank of England back then raised interest rates to their highest ever. Many countries defaulted on debt. And it was basically the first global financial crisis in history. And what's most relevant to this is if this is just a massive investment boom focused on speculation, the speculation back then was railways it was construction everybody got swept up in it and it had catastrophic consequences so the big bankers like the Rothschilds okay lots of charlatans lots of grifters interestingly I went back and looked at the book yesterday two of the voices who were really warning against this were Karl Marx and Mark Twain okay they were very very spectacle anyway it went tits up we ended up with two decades of deflation we ended up with the 19th century version of populism because of the Rothschilds involvement we ended up with a huge wave of anti-semitism this was all about Jewish finance if you have a financiers America turned in on itself European nations turned in on each other and free trade became a dirty word and I'm not saying that history is repeating itself but I'm saying it might well so there's another thing which we haven't talked about enough um which is the possibility that all this AI investment is an enormous bubble like your 1876 railroad bubble and there's a lot of evidence that that might be the case which is why you suddenly find that the share prices on some of these companies will tumble incredibly and they really matter I mean as we've said before the U.S. is 70 percent of the global stock market and these hyperscalers are more than 30 percent of the U.S. market so if these hyperscalers that are going to be spending 700 billion dollars in the next two years begin collapsing and one of the reasons they might begin to collapse is that their business models require getting more and more and more capital so in order to get these improvements you know they've gone from GCSE maths to A-level maths to you know winning maths olympiads they're doing it by spending hundreds of billions on compute yeah and there may in the end just not be enough money out there or not enough people prepared to put money in and and there are other warning signs you see these weird circularities where one of the AI companies will be putting money into Nvidia a chip company and then other money will be coming around the back in another way now their profitability is staggering Anthropic has gone in very short order in just over a year and a half from making 300 million dollars annually to making 14,000 million so that that's the other side of the story I mean one of the reasons all the money's coming in is these companies are generating huge amounts of revenue if it all explodes we're all in real trouble but then I guess the question again for Andy Burnham is what does he do about it I mean what's he supposed to do to insulate ourselves against this are we all supposed to go into cash are we all supposed to invest in the European stock market because it's less AI heavy and another question that's worrying them that the valuations could collapse as you say because of financial engineering they could also collapse because it might turn out that the Chinese are able to produce a model that is far far cheaper and does 99 percent of what these models do and is open access yeah so if you suddenly end up with these models being really easy to make and the stuff becomes a commodity like electricity and this is one of the big fights this is where US China becomes interesting is this a race where if you get to the finishing line first it's winner takes all because you've built a super intelligence and you have a company that now controls the whole world's economy or have you just invented something like electricity where in fact everybody ends up getting it really cheaply and there's no particular advantage in being anthropic or open AI as opposed to Chinese that too could make the bubble burst and the answer is that none of us know the answer is that none of us know including these guys who are doing it and just to give us listeners and viewers a sense of the scale of this so Columbia Business School they reckon that AI build-out is going to account for almost three percent of economic output in the coming years so that's even more than the railways and all that stuff represented in the 19th century as you know I've been in traveling around European Union for the last few weeks so I got back home to discover all the kind of mail that gets left here and I think you do as well I subscribe to the printed version of The Economist two of the three copies of The Economist that were left lying on the map were about AI the cover was AI could AIs become conscious pretty terrifying thought and this one how AI is breaking the state and this is really interesting piece about and about Britain and about how our crumbling court system is under even greater attack now because everybody has access to these to these large language bottles which could write complaints for them work out loopholes in the law and so that was a very interesting thing and then up here you mentioned China lessons from Chinese AI so I then went off and went down a bit of a rabbit hole looking at China and AI and there's something really interesting about this the Chinese seem a lot less the Chinese people insofar as you can measure it but there are ways of polling in China seem a lot more positive about AI than we are in the West which that is the one argument that makes me worry okay because as you say if China if this ends up being the bee's knees and all it's meant to be and that all the AI tech bros say it will be then if China's out in front on its own we've got a very I found a comma where I read this but it might be in the New York Times but it was a piece about a court case in China where a man launched a court case because he lost his job to AI and the court upheld his complaint and it led to the state through the state media saying this is the attitude that we should take to the loss of jobs and the state media ran a big thing about the really clever firms will take AI, take its benefits, but then use it to generate work for other people currently not working. Now, this is in a place with 17% youth unemployment. Is that the attitude of Donald Trump? Is that the attitude of Elon Musk? No, it's not. Is China being more careful about this than we are? I think this is an amazing question. Let's take a quick break. And then when we come back from the break, let's talk about what we're going to do about it. And actually, this is an opportunity to draw people's attention to two miniseries I think we've done, which are very relevant to this. Part three of the Andy Burnham miniseries, which is Alistair and Steve Rotherham looking at the question of how Andy Burnham thinks can be very relevant to his policy. And then I've done a miniseries on AI with Matt Clifford. If people are interested in talking through these issues a little more slowly, what you'll find is that although in this conversation with Alistair, I'm coming across as the AI promoter, actually, all the way through the miniseries, I'm portrayed by Matt Clifford as the AI pessimist, AI doomer, who keeps talking about what the catastrophic impacts could be. So to listen to both of those miniseries, become a member at theresthispolitics.com. Much more to get into after the break and do look up this miniseries. A better help ad. Imagine carrying a backpack every day. Every conversation, every mistake, every expectation. You probably wouldn't notice how heavy it was until someone asked if you wanted to put it down. Sometimes feeling better about it. You probably wouldn't notice how heavy it was until something better starts exactly like that. Better isn't one size fits all. Better is personal. It's therapy tailored to you to help lighten what you've been carrying. Match with a therapist who truly gets you. Visit betterhelp.com slash restpolitics. Shaq here. What's it like to buy a car from Carvana 100% online and get it delivered to your door? Let me tell you. Your car is here, Mr. Shaq. Thank you, Mr. Carvana guy. Can't wait to take it for a spin. Only the best for you, Mr. Shaq. Hey, have you always been so big, tall, and handsome? Not always. I was 6'10 once. Seven days to love it or return it, Mr. Shaq. Not that I would. You never do. No, I never do. So buy your car today on Carvana. Delivery fees may apply. See our seven-day return policy at carvana.com. Welcome back to the Restless Politics. For me, Rory Stewart. And me, Alistair Campbell. Here we are in this very intense debate around AI. Okay, so Alistair, you made a really good point about costs and benefits. And the point you're making, I think, is that the benefits of AI so far are pretty partial, and the future benefits are a little bit uncertain, whereas the costs are pretty clear. Or some of the costs are pretty clear, particularly the costs in terms of building out data centers, making people like Elon Musk far richer, and certainly creating new methods of warfare, cyber attack, and potential threats around bioweapons before you get into what these things could be doing in five years' time. Okay, so my comeback is this. I think you're right. I think if you were Trump and Xi Jinping and you looked at this stuff, I think now would be absolutely the time to say, stop any more frontier development until we've digested what we've got so far. It may actually, interestingly, already be too late. One of the arguments that the tech bros are currently running when I'm saying this stuff could get out of control. I think it's a good time to say, stop any more frontier development until we've digested what we've got so far. One of the arguments that the tech bros are currently running when I'm saying this stuff could get out of control do Hugging Face incidents, become autonomous, mount its own cyber attacks. And ultimately, there's this fact that they talk openly about a 20% chance that it could extinguish humanity, could kill us all, right? So one of their answers is, well, it might be too late already, might have already got out and hidden itself in a whole series of codes in a way that stops us from hitting a kill switch. Okay, let's assume that's not the case. Or let's hope that's not the case, because if it's too late, it's too late. So the argument would be, stop now. You've done a lot of interesting stuff. You haven't begun to apply properly most of the advances that you've achieved over the next three years. So let's take a breath. Let's not train any more cutting edge frontier models. Let's try to get the most out of them. And let's run safety tests. And the safety tests we run now will be much better than the safety tests we would have run three years ago, because we know much more about what these models are capable of and what they're not capable of. And let's try to model in a more sophisticated way. Now we understand more about these things, what their economic consequences, what their social consequences could be. Saying that, of course, is a big, big problem for the US economy. Because the valuations of these companies, you know, Anthropic, people think may IPO at $2 trillion. That's a company that didn't exist four years ago. There's nothing like this has ever happened in the world. And if you say they're not going to be training the frontier models, obviously, their entire growth story is based on the story that their next models are going to be much better than this. But, and this is my big kicker, I'm afraid the risk that we face in Britain and Europe is that although the costs outweigh the benefits, we could well end up in a world in which the benefits go to the US and China, and the costs are borne by us. So they end up being able to build the $2 trillion companies, they end up being able to build billion dollar companies with two employees, they end up taking the cutting edge on pharmaceutical research, they end up being able to develop the most powerful weapons in the world. And we end up with the costs, we end up with the data centers, we end up with the risk of a 20% chance extinction of humanity, we end up with the cyber attacks, we end up with the bioweapon attacks, we end up with increasing inequality, not just between Elon Musk and the average American, but between an American Chinese economy growing gangbusters and our economies being entirely stagnant so that we end up essentially like a country in the early 20th century that didn't have electricity when other places do. Well, first of all, a 20% chance of extinction, Yeah. Yeah. That covers all countries of the world, large and small. So it's not as if we would be the only losers in that. Look, I understand where you're coming from. And I totally get that this is a very, very difficult challenge for every country in the world that is not the United States and that is not China. There's no doubt about that. And they are all busy trying to catch up. My worry is that they're all doing so on the basis that they're having to inhale and agree with what thus far, actually, I think, has been an enormous amount of hype. We have not yet seen the benefits. I've not seen anybody explain them to me. I've seen them in very, very generalist terms. But at the moment, what you're seeing is companies who cannot quite believe their luck, that they're growing at the scale that they are. And when you mentioned, Rory, trillion dollar companies with two to 10 employees, for most people in the world, that's a terrifying prospect. Because that basically says there's no work. Yeah, well, let me come in on that. And that's that. That's something that we really have to push again and again and again. One of the reasons why this is so scary is that when they say these models will make companies more productive, generally what they mean by that is they'll be able to replace people. So that's unemployment. But it also carries the security risk. Because what they're saying is these models will become more and more autonomous. They'll be more and more able to operate without humans and without humans. So they're very, very excited at the moment on recursive self-improvement, which means the AI improves the AI, and the AI coding improves the AI coding. Now, that is what their growth story is built on. That's what their productivity story is built on. But the other side of that is where the danger is. Because the danger of a bad human getting hold of a nasty weapon is completely different. The danger of an autonomous AI being able to do this stuff itself without human supervision. So I think one of the things you're saying there is you can't have your cake and eat it. You can't say this is going to generate enormous productivity and growth by replacing humans and also say we're not going to end up with an enormous loss of white-collar jobs and have more and more risk of autonomous attacks. You know, maybe if there's one good thing to come out of this right now, I think this backlash is going to do fundamental damage to the Republicans in the midterms. And I think the Republicans are beginning to see that. So let's just talk about that a little bit. So you've got 4,000 data centers across the U.S. You've got another 3,000 in the planning process. Okay. You talk about the numbers. By the end of this decade, tech companies are going to put $7 trillion into data centers. $7 trillion would feed every single person in China for three years. Okay. And it's also why these companies are starting to feel and operate like they're more powerful than even major nation states. And by the way, on the numbers, Rory, back to Nepal, did you see the initial donation from the United States government, which used to have USAID, to Nepal for the clear up? $500,000. No, it's unbelievable. It's very difficult to understand the scale of investment we're talking about and why, if Europe wanted to do this, it would have to do it with Canada and South Korea and Japan and everybody and borrow a huge amount of money. We're back to the middle power strategy. But just on the backlash. So in August, last year in a poll, 43% of eligible US voters supported the construction of data centers in their vicinity. Okay. 42% against. This August, 15% in favor, 75% against. And I was talking to a Republican strategist. He's an anti-Trump Republican, but he's still involved in Republican campaigning. And he said he has never, ever seen an issue like this. In that, you can be doing surveys, you can get a three-time Trump voter who says, "I hate these data centers." You then get somebody who's massively pro-abortion, who's against the data centers. You can get evangelicals, you can get anti-vaxxers, you can get internationalists, you get rural and city. He said it's the first time he's seen non-voters actually starting to say, "This is the issue that will make me vote." And he said he's never seen anything quite like this issue for turning the dial. And then you've got this response, haven't you? Which is that increasingly, Josh Shapiro, who's running for reelection, has announced a moratorium. There's been talk about moratoriums in Texas, which is one of the big build-out places. And you're absolutely right. Republicans are falling over themselves to say, "We're not the guys in favor." And then in Michigan, the Democrat candidate that we've talked about is perpetually quoting these figures on white-collar job losses, on existential risk. But there's another story that we haven't covered this week, which is shows the problems with this, which is the meta-settlement. This is the latest in a long-running number of tech cases, the big tech companies, where they have been forced to pay $18 billion. So $18,000 million. Yeah. But Rory, I can remember when I worked in newspapers back in the day, we had libel insurance. I reckon these companies, $18 billion to trillion, you've just been talking about the trillions that these companies have. You've just been talking about the trillions that these companies have. 18 billion to you and me and to most of our listeners is a phenomenal amount of money. I think to meta, they've probably got an insurance. Almost regardless of the insurance, one of the indications of how little difference it makes them is that when it was announced that they were going to have to pay out $18,000 million in compensation, their share price went up because the market thought, "Oh, well, that's not much money." So the story here is it's taken 21 years, 2,600 lawsuits. And in the end, all they've managed to do is to get meta to remove a like button from teenagers. That shows you the problems that anybody's going to face trying to regulate these companies when it comes to safety AI. If it takes 20 years, 2,600 lawsuits to get a single removal of a like button. And incidentally, meta has now come up and said, "If we're doing this, we want our competitors to do it, too." And there's been a resonating silence from people like TikTok that have no interest in it. Final thing, which has interested me on this, is that we often think that these people are driven by pure evil. It's much more banal than that. What they're driven by is accountants. What happened in this case, it seems, and this is what decided the case, is they found a bit of paper in which meta had been told that if they could acquire a teenage member, the lifetime value of the teenage member, they'd be able to get an account. And this is what happened in the case. This is actually something that's not a million miles away from being in the podcast world, where you're told that if you acquire a member, it has a lifetime value, right? So, it's a really- Don't give away our business more, Roy, for God's sake. But I think it's worth maybe being a bit more sort of frank and open about, because it shows you how horrendous consequences follow from what starts with a sort of quite innocent sounding business decision. If we can retain younger people, it's going to be worth more for the value of the company. Yeah. But I mentioned in earlier on about tobacco and fossil fuel industry so the fossil fuel industry has spent billions telling the world things that they know to be untrue okay the tobacco industry spent the equivalent of today's billions telling the world that smoke you remember the adverts eight out of nine doctors smoke smoke chesterfield you know this embassy this is the cigarette the dentist to prove off i mean just total lies okay and we all went along with it we in newspapers so we didn't cover stories about cancer why because they were massive advertisers in the newspapers and just on this this meta thing so this was a result of 29 states so more than half of the american states suing meta for creating addictive harmful products for children okay the case lasted four days because meta wanted out of it fast so 18 billion over 10 years in 29 states is pocket money for these guys it's absolutely nothing and we still don't really know from this case what you're supposed to do about it because the problem with our current legal approach is that we tend to think the way to solve it is to remove the like button for example from teenagers yeah and the problem with that is it just incentivizes a bit like the hugging face attack companies like meta to work out how to get around that problem and retain people without a like button what we should be doing is trying to define not the technology but the outcome we should be doing is trying to define not the technology but the outcome we should be trying to define not the technology but the outcome we should be trying to say what we're interested in is avoiding bullying unwanted advice surveillance whatever it is we're trying to do and set the liability around the outcome not around the the question of the design just before we move on back back to napoleo let me try to sort of draw this incredible conversation together so i think what we've demonstrated is why this whole thing is something that politics is struggling with worldwide right firstly as you say when you look at the hugging face attack most of us talking about it don't actually understand the technology and even the people designing these systems can't look into the black boxes in the way in which you might imagine because in the end it's a series of weights and parameters moving around that generate this stuff and exactly how they do that still remains a bit of an art form and a mystery secondly we can't even agree on what the benefits of these things might be and we can't even agree on what the benefits of these things might be or what the risks might be and a really good way of illustrating that is are we looking at something which is about to make america and china the most powerful unbeatable economies in the world or are we as you suggest potentially looking at a bubble in which case they get hurt the hardest in which case they get hurt the hardest we all get hurt but they get hurt the hardest are these things where it's going to become a commodity that we can all access and we don't need to get too stressed because we're going to be able to get open source models from china so it doesn't matter if trump hits a kill switch and excludes us from the american models or are we going to need these frontier american models in order to run all our defense and security if that what do we do about it do we build data centers do we try to build our own chips do we try to get together to build our own ai models which is going to be as you've just pointed out so expensive that we can't even imagine where we would get the money to do that and is there any way of dealing with and i maybe this is my final point which i think you've been getting at all the time which is you're talking about moral hazard you're talking about the benefits going to a very small number of people mostly in silicon valley who are going to become unbelievably wealthy and the risks being born uh by the rest of us which is essentially the story of the 2008 financial crisis right and is actually the story of the iran war trump doesn't need to care too much about the consequences for what happens in the state of hormuz because he's creating his own gas elon musk and mark zuckerberg famously can retire to their bunkers when the whole thing goes wrong and i i know i said finally but there's one more thought i had when i was talking to you which is you said one of the problems is communication they can't explain what the benefit of this stuff is my experience and i know a lot of these people right i've met a lot of these people i've talked a lot i spend a lot of time with some of them a lot of time with some of them is that their personalities and their education is very strange they are mostly engineers and many of them at the top levels now are people who have been computer science researchers who've been interested in artificial intelligence for 10 15 20 years they are basically interested in building it for its own sake they're also very interested in making a lot of money but one of the reasons they're just making a lot of money is that it allows them to spend more money on building their models right yeah so if you were to say to them get one on the show what is it you're doing they would quickly say oh you know we want to solve all the maths and physics problems in the world right the general public is not interested in that the general public is interested in the question what does this mean for my energy bill is this going to make me healthier are you going to cure cancer am i going to lose my job not have you managed to design a godlike super intelligence that's going to solve all the questions of advanced physics yeah well when we were driving around europe one of the books we listened to in the car was the sebastian malaby book on dennis acebes yeah you know he's a very very interesting guy and very very clever no doubt about that but i think that book was one of the reasons why i felt this thing is not in a good place because i thought i'm quite enjoying listening to this and hearing about this guy but i'm not feeling comfortable about where this thing is heading because i don't think they really know and if you then have politicians alongside that so as i said at the start trump all against it's all a scam then he thinks oh it's a scam i'll be in on that he's now on in it and the one of the reasons he's making billions is because of his connection with ai the ai buses must be looking at him and thinking god he's the best sort of he's not a cheerleader it's just that he doesn't care about the consequences well maybe let's just finish with nepal because i think there are two things which really relate to everything that you've been saying about the problems of not just climate in general but the way in which we structure climate finance firstly as you say nepal emits very little it's neighbors china and india emit a huge amount and are contributing much more to climate change it was a champion in green renewable energy generation in fact the trishuli river where this happened was where some of these enormous hydropower plants were meant to be built which were not just going to make a huge difference to clean energy but also save the nepali economy but it also raises a question of the whole way we've designed compensation so when we talk about thinking about adaptation and how to deal with countries which suffer most from climate change we have a measurement problem people like bangladesh can do quite well because we have really really good data on rainfall and flooding in bangladesh stretching back over many many decades and rainfall is very frequent what happened in nepal is an event that happens once every 100 200 years we don't have a data center it's not frequent and actually it's very difficult to exactly prove the contribution of climate change to that that means that when they go to international climate conferences places like bangladesh are well ahead of them so if what climate's going to do and it seems very likely is create more and more unpredictable events then i think we need to completely change the way we do development finance from saying you get the money if you're unduly vulnerable to climate change to saying you get the money if you're poor we need to go back to a much more traditional development model because i don't think that our climate models even with all the benefits of ai and that's one of the things that these tech bros keep boasting about that it's going to help us to do both mitigation adaptation climate change even these best models are not going to accurately predict what will happen to countries like nepal yeah i was reading um somewhere over the weekend that one of these lakes that is sort of sitting there and they're really sitting there worried that it's going to explode it doesn't even have a name the lake doesn't have a name because it's so remote and you know we're talking about lake ontario most big lakes have a name okay this lake has no name what i found most awful about the whole thing i mean watching the film was just horrific there were two things that sprung to mind the first was i i saw a listener report on the world service about how china was reporting it very very differently they were not showing the worst of the footage they were showing the great rescue going on we've all seen that shot at the tibet border of this i don't think you've seen the one where there's these buildings it's like a disaster movie it's just yeah it's just horrific and we know that when they say there are several thousands still missing most of those sadly are going to be to be dead but i just think the extent to which internationalism i mentioned the the united states initial donation for help being half and 500 000 i mean that is just the richest country in the world sees this is leading their news the same as it is in every country in the world that says to me that's a government that is saying we don't give a we really don't give a again i know i'm obsessed with these numbers but that is one ten thousandth of what used to be their annual development budget you know it's not five percent or one percent it's one ten thousandths of what used to be their development budget yeah i think i think they've upped it a little bit and then let's ask britain you know what's britain contributing i think it's several million but still not enough tiny we would have put in in the past fifty hundred million in the past yeah something like this correct and i know that because i was the development minister in nepal after the nepal earthquake in 2015 and i think we put in many tens of millions approaching 100 million into that and that killed almost exactly the same number of people so it's a really good way of holding the british government to account too it's a like for like comparison i totally agree and i and i do think i think there is a special place in hell for people i mentioned the fossil fuel lobbyists people like richard tice you think it's funny that we have wildfires in france because that means the grapes will go better and we'll have better sparkling wine in kent and then people who on this just kind of shrug their shoulders and say well you know it's a long far away place of which we know little but the the fact that such a catastrophe can happen with little or no warning and as you're right there are no warning structures because they don't have the infrastructure for that either when you read things that say there's a i can't remember the name where bodies are literally just washing up on the beach okay there are bodies being found that are floating as far as india this is an area of nepal that i know reasonably well and one of the things that's happening now is as you talk about these new lakes and rivers the risk is now moving further and further south away from that border region it now looks like there's real problems that may then erupt so we may not have even seen the beginning of this isn't and as you get further down into new accords and towards katmandu the population massively increases and the risks massively increase yeah and also what a what a baptism for the i mean a relatively new prime minister who's elected in march remember we talked about at the time he was a six he was a product of the gen z protests against the old regime balendra shah 36 pretty young guy to be dealing with this and of course dealing with it in the context of i mean china a lot of this happening in tibet so xi jinping they had a minute silence before the cabinet meeting he said his deputy there to deal with the thing on the ground but we are seeing that contrast between china which continues to have very controlled output about what's actually happened very controlling about the debate and where we are able to see this just utterly horrific stuff that consequences are going to be with us for years and the thing i worry about is that with all that you know this from your time at is that these things happen you have the emergency appeals you have all that stuff people focused on it then it's going to take years and years and years to recover from this and balendra shah he he's the government the nepalese government has calculated they're going to need about four billion for reconstruction well they're not going to get that if america's giving five hundred thousand dollars well thank you alice that was a really interesting good debate we went a lot a lot of directions there um when we come back for question time we're going to look at iceland we're going to look at uk russia we're going to look at uk us we're going to look at conferences and maybe a couple of fun questions from lessons as well at the end see you 9-11 was the deadliest terrorist attack in u.s history but could it have been prevented i'm gordon carrera national security journalist and i'm david mccloskey former cia analyst we're the hosts of the rest is classified in our latest series we are telling the inside story of how the 9-11 terrorists slipped through the cia's grasp in the summer of 2000 a man walks up to u.s immigration at newark airport his passport gets stamped he's waved through and welcomed to america that single stamp was one of the gravest intelligence blunders in modern history because one year later nearly 3 000 people will be dead that man was muhammad atta the ringleader of 9-11 but here's what they don't tell you the cia already knew al-qaeda was inside the u.s an fbi memo explicitly warned that the cia was inside the u.s and the cia was inside the u.s and the cia was inside the u.s they warned that terrorists were in u.s flight schools so how did he slip through why was this red flag ignored in this series we dig into the declassified files to uncover how the biggest terrorist attack on america was allowed to happen get the full story by listening to the rest is classified and subscribing to the declassified club wherever you get your podcasts when it comes to my policy farmers gives it to me straight so there's no way i'm going to be a terrorist if i'm not a terrorist it's your coverage you should never be confused you hear that an insurance provider with a conscience we're a choir but yes we have that too straightforward coverage you can literally understand that's what you get when you have farmers learn more at farmers.com underwritten by farmers truck or fire insurance exchanges or affiliate products not available in every state