The backlash against AI

Hello and welcome to the Rachman review. I'm Gideon Rachman, Chief Foreign Affairs commentator of the Financial Times. This week's podcast is about artificial intelligence, AI and international politics. My guest is Sebastian Malibi, author of a new book, The Infinity Machine, which is a biography of Demis Hasebus, a key figure in the AI revolution. The potential good and bad of AI seems to be limitless. So should the world's biggest governments put their rivalries aside and agree to regulate AI together? Today we stand on this edge of another technological transformation, one that will be larger, faster and more consequential than what came before. It will touch every profession. When Eric Schmidt, the former CEO of Google, gave a commencement address at the University of Arizona recently, he got more than he bargained for. I know what many of you are feeling about that. I can hear you. That response to Schmidt captured the widespread fear that AI is out of control, and that the dangers and threats it presents to humanity are greater than any potential benefits. So is it time for governments to do more to control this technology? That was the issue I discussed with Sebastian Malibi, but we started our conversation by talking about the booing of Eric Schmidt. It doesn't show that the backlash against AI is growing. Yeah, and it's growing amongst college graduates who are at the sharp end of Leibn market disruption. One of the obvious things that AI does is that it carries out a lot of cognitive tasks. And if you don't have a lot of professional experience to differentiate yourself from the AI, then you're the most at risk. So new graduates are on the receiving end and they're well aware of that. And also, I think, beyond just whether they get a job or not, there is the immense frustration of all the people feel about the application process itself. You fill in with the help of LinkedIn, a thousand applications, two clicks, and you're done. The CV is uploaded automatically, and the letter is written for you. And then on the receiving end, the company screening the applications has to screen it with AI because it's a voluminous. And so we have this AI sort of arms race in the job market, which is no fun for other side. And is there evidence yet of job destruction, or is it still mainly fear? I think it's mainly fear at this point. You know, there's certainianic dots and it's difficult to disentangle whether a company like Meta that risks a lead off 10% of its workforce citing AI as the reason. Is it the reason or is it excuse? Is the question? And I think that's been true across a bunch of hyper-file headlines. When people do careful disaggregations of the data, they often find that any slowdown in hiring an optically segment seems to have started before chatty PT came out. And so it makes it doubtful that a continuation of the trend can be attributed to AI. But I'm not terribly invested in that debate in the detail because I think that whatever happened thus far in the data, I'm pretty sure that AI over the next year or two is really going to start hurting the labor market. You know, Demis Haseeb is the boss of Google DeepMind very well and lots of figures in the AI industry. And one of the things that struck me when I occasionally come across them is that they don't dismiss these fears. They certainly talk of the potential benefits of AI, but they're very conscious of the potential downsides on this job's thing. I don't think they seem to have any particular answers themselves. And they're also worried about deskilling, about the fact that humans will simply stop learning because the AI can do it more quickly and efficiently than they can. Yeah, absolutely. I mean, in my book on Demis and DeepMind, there's this scene where in 2015, Demis has co-founded Mustafa Suleiman presents slides to the top brass of Google, which is just bought DeepMind, showing the pitchforks coming. There's a clip from the Simpsons with these characters waving their weapons at the capitalist who are presumably cowering behind the castle gates. And you know, the point of the slide was to say, listen, the whole world is going to hate the AI labs and Google included if we don't come up with some prescriptions for preventing jobs apocalypse. This is now 11 years ago. And there are to be fair to the labs, some documents that they've put out talking about things like wage insurance so that if somebody loses their job because of AI, they get some insurance money to make the fall in income a bit more tolerable. There are other ideas like there are Trump accounts in the US where kids get savings accounts with some equity exposure in them. And there are ideas around putting AI stocks into those accounts in order to make people feel they have a stake in the upside from AI. So there are ideas kicking around, but they haven't really been implemented yet and therefore people are pretty frightened. And I suppose that the other reason the pitchforks might be coming for the AI bosses is that they themselves are becoming fabulously rich as a result of this technology which may be taking opportunities away from human beings. And therefore AI seems to be likely to increase in the quality quite sharply at a time when we're already worried about inequality. Yeah, I mean, absolutely. There was a eerie moment in the Elon Musk versus Sam Altman trial, which was going on until about a week ago. And it was revealed that Greg Brockman, a co-founder of OpenAI, remember OpenAI started as a so-called non-profit. His stake in the company is 30 billion. It's a pretty amazing number. He wasn't even the boss of the company. He was just a co-founder. And so these astronomical fortunes are being generated at the same time as people are appearing for their livelihoods. And talking about these fears about what AI will do to people. I mean, again, I remember speaking to an AI boss who said he personally was worried about the education of his children because why would they bother to learn a language, why would they bother to learn to computer code when they can just turn to the machine and can do all of that for them. And there does seem to be some evidence that actually test scores may be going down as people begin to use AI. So do you think as it becomes more intelligent, we're going to become stupider? I mean, I have a friend who teaches economics at Harvard and he insists on having his students do tests with pen and paper, no AI or other computing allowed and they're kind of under supervision as they do this to force them not to use AI. Even though of course this friend of mine agrees that AI is incredibly useful, but it's only useful if you do the hard work of educating yourself first. And I think the key thing is we need as humans to retain the sort of sense that hard work going through the difficulty of learning things is worthwhile and it pays off and that's how you really improve yourself. And then if you can keep hold of that ethic, then you're allowed to go off and get easy answers from AI that accelerate how many things you can do and so on. I mean, I feel personally that just because I now rely completely on my iPhone to find my way around a city or drive without losing my way from A to B, it doesn't mean I stop thinking. I've just offloaded one mental task. I've kept several which I engage in myself very energetically. So I think so long as we keep the second, we're probably okay. Except if I think back to my sort of teenage self, you know, I'm sure I would have taken on board the argument that you know, you should learn all this stuff, but you get home and you can either spend three hours battling with a maths problem that you can't really do or you can ask Chatchy P.T. to do it for you. I mean, everybody's going to do the later on, right? Yeah, that's why you need the classroom tests where no computers are allowed. Absolutely. Because I think particularly children won't go through this hard process of learning. I mean, another example here is I was talking to a 25 year old who's a friend of my son who says that he now vibed codes. And at the office, he gets an almost credit for knocking up some code which creates a new website and does something useful for the company. But in his view, any idiot could do this because you just have to talk to the computer in the right way. The AI system creates the website for you. He has a masses of computer science. He says he doesn't use it at all. But he's actually wrong in the sense that you need the masses and computer science to even know how to formulate the question about creating code to create a website. So the point is he did some hard work. He then uses AI. He has the benefit of both. And he gets deserved QDoS from his colleagues who didn't do the hard work of a masses and computer science and don't feel they can create the website. And what does Demis Cecebus he write about? Deep down think about all this and why does he keep going despite being a apparently man, you know, as a highly intelligence and B, has a social conscience? What's his attitude to it all? I mean, that's precisely the question I wanted to grapple with when I wrote about him. How did you process the fact that you've got your hands on the 21st century version of the nuclear material? And in addition to it being a potential robot apocalypse or terrorists getting hold of it and doing bad stuff and all these other risks, there's also this job prospect. And I think Demis consoles himself with the idea that once you go through a painful transition, there'll be milk and honey in the distance where people will have more leisure, they'll pursue their hobbies, their interests. There was a science fiction series called The Culture Series by Ian Banks that portrays a future in which AI is happily co-existed with humans, humans can hop on a rocket whenever they feel like inhabiting a different planet. They can pursue all these interesting hobbies and play games and all that stuff. That appeals to Demis personally. He says, "I would sample all the planets." And so I think he has this vision that in the end, humanity will be augmented, but the transition is going to be painful and I don't think he's really gotten answer for that. I suppose there's also the whole question of human meaning, which again, I think he and others have thought about. I mean, how satisfied are people going to be just traveling or playing chess worse than a computer? You know, Demis would say, and I think he has a point that if you look at chess as an example, deep blue, the IBM computer system defeated Gary Kasperov in the late 1990s. So it's been more than a quarter of a century since humans could win a chess against machines, but it hasn't stopped chess as a human sport from growing in popularity. More people watch it, more people play it partly because of the advent of computer systems where humans can have fun playing against machines, all playing against each other in a machine-enabled platform. And so you and I come out of similar educational trajectories. Maybe you studied French, like I did. Maybe you read Camille and Sartre and we were told we were 16 or 17 that look, life is pointless. There's no real objective. We're just all inventing reason to get up in the morning. Maybe that's right. If you're not religious, perhaps that's what you believe. But we're extremely good at inventing things. People have different passions and obsessions, whether it's their friends, whether it's pottery, whether it's watching football on television, you know, it could be anything. But I do actually believe that humans have an intrinsic ability to find purpose. If you think about how the labour market has changed between now and the pre-industrial revolution, I mean, it's completely unrecognizable way that people live today. And yet they've been able to generate purpose and you know, canes is letter to his grandchildren, the famous thing about how will you cope with leisure in the future. Well, you know, people invent new things to do and they seem to be happy at mostly. Yeah. And I guess one of the lessons, I mean, you mentioned how labour markets have adjusted in the past, is that when agriculture first became mechanized, everyone said, "My goodness, what are people going to do if they can't work on the land?" And yet the economy created other occupations. And yet I have heard some AI bosses say this time it's different. They're not that confident that all this labour will simply be displaced to some other activity that a computer can't do better. Yeah. I mean, I think one lens through which to view this is the China shock in the early 2000s. And you're until that time, every single serious economic study of labour displacement by trade showed that jobs were created fast enough that unemployment didn't really go up and wages were not negatively affected. And then came this moment when the shock was big enough that there was an effect and two million additional people lost their jobs over the kind of baseline. And that was spread over a period of 12 years from 1999 to 2011. This is in the US. This is in the US. So this was a shock that was certainly enough to change the politics around trade and to create a big anti-globalisation backlash. In some sense, it's a warning because two million is not that much in a very big economy like the US when it's spread over 12 years. And yet a real shock, but not a big shock, could create a big psychological and political consequence. And so I think with AI, that tells us that both, there can be shocks which are big enough fast enough to create proper damage to people's lives. And also that the political consequence would probably be disproportionate. It's a multiple of the actual effect. In fact, I think some people are even talking about, well, could we reverse this? Could we say, "Okay, actually, we don't want this AI? Could we turn it off? What would Demis have said about that?" I mean, his personal motivation, let me just describe that for a moment. We spoke for more than 30 years of this project and I got to know him extremely well and he sort of trusted me. And so we're sitting in this park and people at the next table having a regular conversation about their friend who went to hospital and is now better and so forth. And I'm with this character who is saying, "Look, I read scientific papers when I work at night and it's 2 a.m. and I see reality screaming at me saying, 'I'm here to be discovered and I have to discover it. I have to do this before I die.' And if I understand the fabric of reality more deeply, I understand something which presumably an intelligent being has created and I'm getting closer to what I might call God. So he's not conventionally religious but the language he uses to describe his scientific compulsion to discover how the world works is spiritual. And so that's his reason for doing it. And so what he would say if somebody said, "Look, why don't we just stop AI?" He would sort of view that as a negation of the human mission. We would be living in cave still if we didn't embrace technology and move forward even though at each step it can be frightening and risky. Now, I'm not saying that's an answer that would satisfy all of society but that would be his answer. Would he also say or others say you say, "It's also impossible, it's too late, you couldn't do it." I mean, as an AI, we'll have replicated itself, we'll have found other power sources. No, we're not at the point yet where AI can autonomously replicate itself and find other power sources although there are those who believe that maybe in 18 months from now we will be there. But we're not there now. I think where we are though is that there is a terrifying multi-country race with labs determined to try and go as fast as possible. And therefore if America stopped, it wouldn't stop China from going forward. And in a larger sense for humanity, I don't think you can shut AI off. So let's talk about China. You've been there quite recently and the way we've been talking about the debates is very much the debate as it's taking place in the West. What's the debate in China like? Are they grappling with the same questions or are they coming to different answers? Yeah, that's a really interesting question. I went to China in March because China does everything faster and so that includes publishing books and they got my book out first even there to translate it. So I was going around four different cities talking to AI leaders both in industry and in academia. And the thing that struck me was that they raised AI safety spontaneously without me having to provoke them into it. And that's not actually how the China debate tends to be characterized by people in Washington DC where I spent 18 years and I have a lot of friends there and I talk to them. And their view is that China doesn't have a muscle memory of the Cuban missile crisis. They don't see technology as accidentally dangerous. They think that big catastrophes are things like the cultural revolution or the great famine. So their political catastrophes, which their history has talked about. And look, I'm not claiming that just because I had 25 discussions over eight days and people talked about safety. It doesn't mean that encompasses the thinking of the Communist Party. It doesn't exclude the reality that both in the US and in China, there are people who want to race and then there are people who are more concerned about safety. There's a spectrum of course of opinion. But it does suggest to me that giving up on talking to China about safety is a mistake. The door is at least a crack open. And yet as you say, you know Washington very well and it seems to me that the race dynamic and particularly the military applications of AI make it highly unlikely that the US, even aside what they're actually thinking in Beijing, would agree to regulate AI in common with China. Yeah, I mean, the way I see it is that during the Cold War, we have a good model of how things could play out with AI. And in the nuclear race, bilaterally between the Soviet Union and the US, essentially there was a race. And the way we avoided catastrophe was just parity, mutually assured destruction and so forth. And yeah, sure, there were arms control deals which put a cap on the nature of the arms race and perhaps reduce the cost of betting. That's great. But basically that dynamic depended upon parity and mutual deterrence. But then separately during the Cold War, there was a different category of concern, which was the proliferation of these weapons to other actors. And that's where we had first the IAEA agreement in 1956 and then the non-proliferation treaty in 1968. And those agreements were relatively successful in preventing nukes from going loose and finding their way into the hands of rogues for quite a while. Of course, it leaked, of course, it's breaking down over time, but it was not bad. And I think for AI, the bit that's missing in the conversation today is that the Biden administration in 2022 addressed the risk of AI falling into the hands of the Chinese. At least that's what they thought they were doing by controlling semiconductor exports to China. But they didn't have a strategy for non-proliferation of the models to criminals, and terrorists, and rogues. And I think that's kind of back to front, because I think in the absence of a policy to stop China from getting it, we would just deter each other. Whereas in the absence of a policy to prevent AI from falling into the hands of criminals, there will be devastating cyber attacks and possibly bio-weapons. And do you think even if the US and China said, "Okay, we're going to get together and try to control the proliferation of AI?" I mean, is it a technology that's even comparable to nuclear technology where we know we look at the efforts to prevent Iran? getting a bomb that it's a complex technology to master only governments do it. This seems to be a technology that's already out there that has numerous applications. It's not just making a bomb. It could be all sorts of different things. It's been worked on by private companies. So even if the US and China decided jointly together, we want to keep this tight. Could they even do it? I think there's a lesson in the last month from Anthropics Frontier Model called Mythos, which is very, very good at cyber attacks, finding vulnerabilities in cyber systems and exploiting them. And what we saw there was that Anthropics said, "Okay, this is going to be dangerous for cyber security." And immediately, the Trump administration, which had been completely stand-offish unless they fair about AI, took control of the question of who would get Mythos and have basically forbidden it to go to anybody beyond the first 40 companies or so that were named as sort of responsible internet companies that would get it first so that they could harden their systems. So the government has stepped in and seized control of the distribution. I mean, there's a joke actually about it used to be the means of production that Marxist worried about. Now it's the means of prediction because AI is basically a predictive system. And the means of prediction have been seized by the government that wants the commanding heights. Thank you very much. And so I think we shouldn't underestimate the ability of governments to wait in and make their preferences known, even though, of course, private sector actors are building this stuff. Ultimately, these actors have subject to American coercion with the exception of the Chinese tech stack, which is obviously separate. But if you are ASMR, the Dutch lithography company that is vital for making silicon chips, you may be Dutch. The lens builders that you get your lenses from may be German. But they are Johnny who are going to do what the American government say because they sell into the US market. They have operations in America and they are subject to American coercion. So what I'm saying is that the problem of getting control of this stuff is less complete than you're suggesting because the US government could just tomorrow announce a set of coercive rules that would close down unauthorized AI distribution at least for frontier models for the more powerful models. And that would apply to everywhere outside China and we run the experiment with Mythos. But the models that are already out there, as far as I know, the AI companies, the anthropic school, the mines, etc are already concerned that they might be enough to enable somebody to create a bioweapon with the models that they already have. Well, certainly the models out there already have given rise to an increase in the number of cyber attacks during 2025. Bioweapons, we haven't seen yet, but no doubt they are in view in the next 12 months. But when we say the models are already out there, we have to remember that to use the models, you need access to a very big computing cluster. These very high-tech clouds of enormous great servers with cooling systems and all that. And the US government knows where these are, many of them in the US and all of them use American technology. And by caressing that choke point, it could prevent bad guys from using the bottles in a bad way. There was a big cyber attack in Mexico a few months ago. And the attack involves models made both by OpenAI and by anthropic. And when the two labs realized what was going on, they just hit the kill switch and stop them. And that's another point. The really good models which are not open-weight, the ones which are proprietary, people access them through a portal that is controlled by the creator of the model. And the portal can be closed on a customer who misbehaves. So up and down this stack, there are more off switches than one might suppose. But you suggest that basically only off switches and the stack is in the United States. And the reason I raise it is it's alive debate in Europe right now because people are suddenly aware that United States run by Donald Trump might not necessarily always be friendly to you. I did debate earlier this year with Arthur Mensch, who's the head of Missrella, French AI company. And he said, look, Europe needs to develop its own AI, it needs to develop its own cloud because if we don't do that, we're handing the keys to our society and our economy to the United States. How feasible is that? Or did it get too late already? Well, Missrella is developing pretty good models. They're not the frontier, but they're not bad. And you could certainly build some compute clusters in Europe and indeed there are companies right now doing that. But I think the autonomy, the strategic autonomy that Arthur Mensch is talking about is illusory because the clusters in Europe are going to use American chips and American technology. And so they're going to be subject to some kind of US oversight. Having said all that, I do think though that there is of course a very important point that Europe legitimately wants what they call sovereign AI. And I think it would approve the US industry to devise a mechanism. And I think I'm going to come back to the Cold War here because I'm sorry to go on about these analogies, but they really are useful. What you need is to square the circle whereby Europe, let's say it's a German chemical company. And the chemical company is connected to the German military and its national security importance. And it doesn't want there to be an American kill switch, even if it's where the private lab, because the private lab in California is subject to aggressive efforts by the White House. So they want their own sovereign rights and they want the kill switch if there is one to be in Germany, maybe with the national German AI regulator, which if it doesn't exist yet, ought to. And so the deal becomes American builders of AI supply the technology to other countries. On condition that they have a serious national regulator, which will use a kill switch to prevent terrorists and criminals from doing bad stuff. And then it can legitimately supply for civilian purposes AI however they like. So it's like the nuclear regime. You can have civilian nuclear power, but it comes with some rules and conditions and some oversight. And I think that's the future we need because both points are important. The sovereignty claims of Europe matter and the desire to prevent criminals from destroying cyber space and building bio weapons. That also matters. Just to finish on the kind of point we began with, this growing question of is there going to be a backlash? Because the kinds of measures you describe rely on, they're being basically a very rational debate in which people stay calm and kind of work their way through all these questions. But it seems to me that given the fears around AI and around jobs, plus the rather sinister image of many of the AI people, the chances of a backlash are growing. And when I say sinister, I mean, if you think Elon Musk is not the most popular guy in the world, he's accused of backing the far right in Europe, Peter Teal, the guy who's funded so many of these most important Silicon Valley companies also deeply controversial. So, Dennis Hisebaz, who you write about, I think, can legitimately be portrayed as somebody with fairly liberal instincts, but are a lot of the other people behind AI not particularly liberal. They're becoming sort of super villains aren't they? I mean, if AI is associated with Peter Teal and Elon Musk, does that not increase the chances of a backlash? Yeah, but I think the backlash is not a bad thing if it leads to regulation over some of these risks. I'm worried about two things. I'm worried about governments not doing enough to control the risk of terrorism and criminal activity with this technology, which I think is really, really serious. And this mythos model shows that this is not theoretical. It's actually a threat now. That's the first thing. And to some extent, the more there is a backlash against the lamps, the more governments should feel emboldened to go after the lamps and impose sensible rules. Why would we have a food and drug administration in the United States, which tests medicines before they go into people's bodies to make sure they're safe? But then we don't test AI models before they go into society to make sure they're safe. It makes no sense. So it's a some extent that backlash is a good thing. But beyond that, I don't want to shut down scientific progress with AI. I don't want the upsides of AI to be stifled. And I agree with you because the villain factor in some of these characters and Silicon Valley is quite high. There could be an intelligent backlash. But during the Biden period, there was collaboration between the White House and the labs and all of the labs, according to Ben Buchanan, who ran the policy at the time in the White House. And he said this on the record, whenever he talked to the labs about how the government might try to intervene and supervise them more, the lab said, "Yeah, absolutely. You need to do it now because it's coming soon." And they were actually welcoming of that. And final question. Where are we in this process? I mean, a lot of what you say implies that it's past time for us to start regulating not quite too late, but we should have gone on with it faster. But as far as I know, people like Haseba say, we've only just scratched the surface of what AI is going to do to society, the economy, science, and so on. So is the pace of change that already seems quite bewildering about to accelerate? Yeah, I mean, it is. At the end of my book, I talk about his vision for using artificial intelligence to build large hydro and colliders in space so that he could understand the fabric of reality and see where the quantum effects exist below the plank level. I mean, this is mind-blowing scientific ambition, which he wants to engage in. I mean, he's got one Nobel Prize already, he'd like a couple more, and he's not tired of pursuing it. So this is going to get very exciting. If you think the last three years were a bit wild, it's going to be even more astonishing in the next three to five years. You talk to people at Anthropic right now, and they say that by 2028, a year and a half away, there could be computers, the AI systems that are so powerful that they code the next version of themselves, and then that version turns around and codes another version so that you get this vertical takeoff in the power of AI systems. Sometimes that's known as the singularity or recursive self-improvement. So yeah, there's a sense that this is really, really, really taking off, and we ain't seen nothing yet. That was Sebastian Manabe, author of the Infinity Machine and Senior Fellow at the Council on Foreign Relations, ending this edition of the Rackman Review. Thanks for listening, and please join me again next week.