Demis Hassabis says society is “not ready” for the AI future

AI is full of possibilities. Enterprises run on realities. That's why Pega combines the world's most powerful AI with the governance, transparency, and control enterprises demand. So you can move faster, adapt faster, and scale AI with confidence. All the power of AI, none of the uncertainty. Learn more at pega.com slash AI. A lot of the cyber issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering. The human's very much still in the loop. And the next phase, which I think carries the most risk, is closed RSI, closed loop, without the human reviewer process. And that will obviously be faster, but I think is also a very dangerous way to get to full AGI. I do think there are some things that generally all the leading labs do agree on, like the idea of third party, third party auditors or evaluators coming in. Obviously, there's a question about which third parties. And I think we need a healthy kind of ecosystem of that. You could have good startups and companies doing that. There's enough revenue there for them, given the amount of investment going into AI as a whole. This is Masters of Scale. I'm Jeff Berman, your host. In today's episode, our very own Reid Hoffman sits down with the incredible Demis Esavis. Demis is the co-founder and chair of Google DeepMind, and chief scientist of Alphabet. Reid and Demis discuss the race to AGI, the future of AI-fueled scientific discovery, the need for more cooperation on safety efforts, and much, much more. I hope you enjoy it. Demis, thank you for joining us. Great to be here. Thanks for having me. We've known each other for, I think, well over 10 years now, maybe approaching 12 or something like that. It was an honor when you asked me to be on the AI ethics board for Google for our various funny meetings. Yes. So let's start, because one of the things we've been trying to do at the summit is to be envisioning the world as we'd want to make it. The theme has been agency. And, you know, as you have been a leader, you've been a new, well before most people were thinking about AI, that one of the key things for thinking about AI for human benefit is medicine and medical. And so say a little bit about how you envision this, how this helps, you know, people say, oh, if I could be getting that, I understand why you AI guys are actually trying to build something that creates a longer human experience, a more quality of experience, call it anti-X risk. Yeah, for sure. I mean, if we take a step back from, you know, the moment is like big picture and you know this knowing us from quite early in our deep mind journey, that the whole point of me working on AI for my whole career was to use it as a kind of like the ultimate tool to advance science and medicine and our understanding of the world around us, but also improving human health. I've always said that the number one thing we should use these amazing tools that we've now built and we were, you know, dreaming about 20 years ago is to improve the human condition and improve human health. And I've tried to do that, certainly with my own work, my own more personal projects, obviously things like AlphaFold, but also the spin-out drug discovery company, Isomorphic. I've tried to push it really hard in that direction, as I know you have too more recently. And I think it's exactly the sorts of things we need to be showing the world that AI can do. I think, you know, what better thing than to, you know, use AI to help us cure terrible diseases. So some of the protein folding was this kind of intersection of the idea of like, we have really good data, we could use scale learning because the whole learning, the cognitive revolution from learning, starting with even Pong, but the, you know, and self-play and all the rest is like, okay, that's, this would be a really good way of doing it. How does AlphaFold lead to medicines? Because one of the things is a question I get frequently is like, yeah, you're talking about that, but is it true? Is it going to happen anytime soon? You know, so I'm passing along the question I get to you. Yes, sure. Well, look, biology is the most complex systems in the universe, right? So that's why it's so hard because of the emergent behavior, the interactions, and proteins are the building blocks of life. That's how I've always seen them. And the protein folding problem, which is what we tried to solve with AlphaFold, was considered to be like a 50-year grand challenge in biology. It's sometimes described to me as like Fermat's last theorem, but for biology. And it's critical to understand the structure of a protein from its genetic sequence, its amino acid sequence, to understand its function. And- That's really important to understand basic biological mechanisms, but also to understand what goes wrong in disease and what things to target. You know, if you imagine you're designing a drug, then you need to know what the 3D surface of the protein looks like so you know which part of that protein to bind to. And so it was always a critical, but not obviously only still a small part of the entire drug discovery process. But it's an absolutely critical part of that. And we were able to, you know, with AlphaFold, accelerate it. By orders of magnitude, by building AlphaFold. And interestingly, at the time, and if you think about it, we finished AlphaFold 2 back in 2020, which is, you know, six plus years ago now. And it's still probably the best example of, concrete example of AI applied to a really hard scientific problem that was thought to be sort of uncrackable. And so that just shows you how far ahead of its time it was. And now, you know, we're really proud of the impact it's made. Like 4 million researchers around the world use it, pretty much every biology. And medical research in the world, almost every drug in future will have made use of AlphaFold, all the structures, but it's not enough. We want to expand it to biochemistry and also predicting things like toxicity or acne properties, all of these other things that you need to know about before you can have a drug that's safe to use and that's efficient in whatever it is you're trying to tackle. So speaking of accelerations, let's go to one of the hot topics of the day, recursive self-improvement. What is, what's your kind of theory of the game? And I say this 'cause you, you know, are a master games person. So I use that as kind of deliberate. And if you don't know what I'm referring to, either look at the documentary or look it up on Wikipedia. So like RSI, opportunities, concerns. Yeah. Well, look, RSI, it's obviously an extremely useful technique to make progress. It mostly works best in what I call verifiable domains, what's called verifiable domains. So math is one, which is why we're seeing so much advanced there. Coding's another. But originally it was games. So actually one of the first sort of famous recursive self-improvement systems was one we invented quite a lot of years ago now called AlphaZero, built off the back of AlphaGo, which was the kind of, you know, famous program that beat the world champion at Go back in 20, you know, 10 years ago now, 2016, and kind of was the beginning, I think, of the modern AI era. And then we built a second version that, was in some ways more consequential, AlphaZero, that didn't need to use any human data anymore. And it just basically played against itself, started at random. And then after, you know, around a dozen to two dozen different epochs, you know, it played like a million games against itself. It would then train a new version of itself that was slightly better than the old version. And then that would play a million games against itself and so on. And for example, and then it could play any game, not just Go. It could play any two-player, perfect. information game. And for me, it was especially visceral as a chess player because I remember watching it play chess and within, you know, 12 hours, basically, or less than 12 hours, it would start from random play in the morning to, you know, you'd have your coffee break at 11:00 AM and I could still just about beat it. Then by lunchtime, it was grandmaster level. And then by tea time, it was better than any chess playing entity. - Tea time is dinner time for the US folks. - Yeah, computer time or computer, you know, your kind of handcrafted chess computer is better than all of them. So we sort of saw the power of that recursive self-improvement. And obviously the thing that all these domains have in common is you can verify the answer. Is it a correct answer? Is it an optimal answer? Is the equation correct? It does the code run? Did you win the game? So those domains are, and the important thing about verifiable domain is you can verify the answer. So you know when you've hit the right point, but also that means you can create almost infinite synthetic data because you can just use the current best versions you have to set new problems or make new moves and then just sort of verify whether the synthetic data was correct. The problem with synthetic data a lot of the time is you can't verify that it's right from the right distribution. But in these cases you can. So we've seen the enormous progress with maths and with coding and coding's obviously become enormously valuable and everyone is having to do that now. I do worry about it because in the limit, so there's different forms of RSI. Right now we're in the early days of RSI I would say, where what's happening is, and you're probably all experiencing this as well with your teams is, you know, it's making a human engineer maybe three, four, five times more productive. Okay, so, but the human's very much still in the loop and reviewing code and architecting things and writing code still, but it's all just enhanced. The next phase, which I think-- think carries the most risk is closed RSI, closed loop, where you basically manage to somehow figure out how to do it without the human reviewer process in the whole procedure. And that will obviously be faster, but I think is also a very dangerous way to get to full AGI, because we're likely for two reasons. It's sometimes used to be referred to 20 years ago as like an intelligence explosion, because you don't quite know, you know, how do you control, keep control of such a system, such a process. And then the end result will likely be quite alien. It's going to be like an alien intelligence. There's no reason you to think it would be sort of in the similar kind of mind space that humans are in. So that will make it very hard for us to understand it and maybe for it to understand us. So I think it's, you know, we should be very cautious as we get closer towards that closed loop form of RSI. And I think all the AI researchers sort of know that, but there is a race to the, there's this sort of intense, obviously competitive race to improve your coding systems. So presume that the incentives to be in a race continue, and you know, you and I, some people really know this, you know, have tried to figure out if there's various ways you can get global collaboration to have less. You've dedicated a huge amount of energy to that, so thank you. It's a hard game, as you know. But presume that the competitive pressure for continuing to do RSI and coding continues. In that environment, what are the best ways that we can potentially navigate that? Because we have to see kind of where RSI is going, what it's pointed at, what are the kinds of things that the, you know, the leading labs such as your own and the others, what should they be doing to try to navigate this environment? Well, what we're doing, and I think other leading labs are trying to do, is you've got to, so there's more prosaic things like make sure cyber defense is accelerating faster than cyber attack, and making sure you give access to trusted cyber defenders first on maybe slightly fine-tuned versions of your general models that are fine-tuned for cyber defense. And so you keep the defenders ahead of the attackers. It's a pretty tricky problem to do in general. Also, building much more hardened sandboxes. I think a lot of the cyber kind of issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering. I think better, you need really amazing monitoring systems. You know, this shouldn't be possible for agents to be running for weeks without anyone realizing. You know, these kinds of things that we've seen some of our competitors and others, maybe, you know, it's just the beginning of this happening. So I think everyone will get rapidly better, but they have to as well, because otherwise something bad is going to happen inadvertently. Some kind of incident, you know, I sometimes call it like an oil spill is going to happen on the current trajectory if we're not, if we don't get, collectively, if everyone doesn't pay a lot more attention and put a lot more resources on these types of systems. Eventually, you kind of want formal guarantees, maybe some bounds around what these systems can do and how, you know, robust, test how robust, like, the things like the guardrails are. This is actually something I encourage a lot of top mathematicians that I know well who are maybe, you know, pretty sad about the state of pure maths, but their skills would be incredible if they pivoted to these types of domains, more like complexity theory, more like theory, game theory. How do you, and of course, we're not going to have to start dealing with swarms of these agents, potentially collaborating in some way. How does one keep track of all of that? Should there be ways that the labs are encouraged, required, permitted to collaborate on this? And if so, what should some of those ways be? Because it obviously gets stronger if there is sharing of intelligence, methodology, you know, mutual accountability. Yeah, I mean, there's been lots of proposals, and I've been working on these things a lot, you know, you saw my essay over the summer, maybe some of you read it about potential for like an SRO industry run standards body with some kind of government oversight over it. I think something like that is going to be needed. And then ideally, it becomes international, you know, it's US led. And then obviously, the state of geopolitics today makes it and our international institutions are very, very weak, as we all know, just at the time when we need them to be strongest in a way. So that's a bit unfortunate, but we've got to start somewhere. But British, a bit unfortunate is holy, just to translate, translate another translation. Yes, like tea time and dinner time. Yes, exactly. A bit unfortunate. And, and so we're trying to, you know, that various of us are trying to do things about that. And, and that essay did have an impact. And, you know, in the White House and in the administration and with our colleagues, and eventually, you know, I think some of that energy became the accord that you the famous letter everyone signed, which is, which is better than nothing. I don't think it goes far enough, but it's a start. And at least it acknowledges from all of the main players, the issues that are there and that they need to be addressed doesn't say too much about what to do about them. But you know, that has to come next. And at least it gives air cover, I think, for the teams to work together on, you know, what sorts of benchmarks, what are safe best practices on safety? What are you seeing in terms of analysis of what things could happen with these types of systems. And I do think there are some things that generally all the leading labs do agree on, like the idea of third party security. And I think, you know, I think there are some things that generally all the leading labs do agree on, like the idea of third party auditors or evaluators coming in, obviously, there's a question about which third parties and I think we need a healthy kind of ecosystem of that, that actually, you know, you could have good startups and companies doing that, that there's enough revenue there for them. And they should be given the amount of investment going into AI as a whole bit like there are auditors, you know, in finance in the finance industry or in any of these industries. So I could see that happening. And some of the big players are moving into this like Accenture, we're working in our other leaders are working with I think they're a little bit ahead of the game, because they've acquired, you know, they've acquired acquired, you know, a good startup, good friend of ours, Mark Warner startups, they have some expertise internally. But I think we need more of that. And then other things like, you know, the leading labs may be red teaming each other before general release of these systems. We all know the disappointment that comes when a grand gesture falls flat, something that seemed impressive in the moment, but didn't deliver the results you expected. Marketing budgets too often fall into that trap, aiming for big impressions, big reach, but without the revenue to back it. LinkedIn has a name for it. Bullspend. LinkedIn ads generate the highest ROAS of all major ad networks. They target by company, industry, job title, and more. Spend $250 on your first campaign on LinkedIn ads and get a $250 credit for the next one. Just go to linkedin.com slash scale. That's linkedin.com slash scale. Terms and conditions apply. Learn more at creativeplanning.com slash masters of scale. Banking services provided by Fifth Third Bank, National Association member FDIC. If you want to learn more, go to our website at www.fifththirdbank.com. realize is you also come to this from a depth of expertise in brain science and you know one of the things is not just a oh did the chat bot say it was conscious or not is not a way to make a decision on this kind of thing what do you think some of the ways to think about this is and also how we should be deliberately trying to shape it yeah well look i think there's quite a few things to say about this first of all we haven't despite thousands of years of thinking about this there isn't a agreed upon definition of consciousness so that's one one issue i think and that's one issue with labeling things conscious especially the hard problem of consciousness you know is it really feeling this internally like the way we we seem to you know behaviorally i don't know about you but yes exactly the behaviorally well but i used to argue with daniel denner about this that There's an additional thing is we're not just behaving in ways like we'd imagine conscious beings behave, but we also share the same substrate, where we have the same squishy brains. And so then it's most parsimonious to think you're feeling what I'm feeling. But of course, with an AI, that's not the case. You don't have the second, the substrate equivalence. So I think it's gonna be very hard to conclude that. I think there are dangers. Look, there's two reasons. why i think we should be focusing on building intelligent tools today not conscious entities i think for one thing is a safety argument and technical safety and risk which is that you know there's a direct trade-off between creating something let's say with self-volition right or self-awareness even prior to consciousness these are the prerequisites right to a conscious entity that it will and that an instruction following or sometimes called corrigibility so this idea of a system actually following instructions and if it's going wrong you telling it you know you as the user or the human user telling it like stop doing that right or switch yourself off or whatever it is that you're going to you know change course so if you are training your system to be able to reject commands based on its own internal workings then you're setting up a situation where if you've got your constitution wrong or your rules wrong and this is what asimov was always warning about with these three laws you know that was a form of a very simple form of a constitution and his books are all warnings that that's you're never going to get that right in every corner case in every edge case and i think his books show that very well and yet we're sort of i think a bit hubristically thinking that that some people thinking that you know sometimes called the misalignment problem is sort of under control just because today's systems you know the good news is they seem more malleable than we might have imagined 15 20 years ago which is great news in terms of their steerability but that doesn't mean we've solved the misalignment problem in total as we get closer to a gi and super intelligence and so on and i just think it's dangerous to think one has so that's the first thing there's a technical problem especially if you combine it with something like an rsi yes exactly you know the reason why we're going in this direction right so rsi with self-volition is you know sort of doubly dangerous so that's one thing second thing is a societal more of a societal issue which is that i don't think we are ready yet as a society we haven't had the debate with a wider society philosophers theologians artists humanities not just technologists as to you know and from many different cultures like do we want to cross that second rubric on are we ready to have the discussion about do these systems have moral standing do we need to worry about what are we doing with them in this box you know are they slaves of some sort you know all these questions come up that are complex questions and i don't think we're equipped to even begin to start to think about that and i think that's a really powerful way of dealing with those questions so my view is we should cross the first rubric on on its own which is build these intelligent tools and then actually use these intelligent tools to do more neuroscience to go back i'd go back and do more neuroscience to actually understand what the neural correlates of consciousness really are finally understand that yes and then you know that point take five ten years and then cross the second rubric on well to have a discussion about should we cross the second rubric on and then if so how right of the conscious entity type and how would you because you know you engaged in hours of these discussions how would you answer the folks who say look it's the only way we get the intelligent correlate to be powerful enough is it it has as or the ephemeral accidental yes it actually has this cognitive state and so right so we should talk about this a lot we discussed this you know even 10 plus years ago and i used to discuss it in my phd back 20 plus years ago of look i felt and i still feel intelligence and consciousness are double dissociable which means that you can have an intelligent system that's very smart on iq and so on kind of like the systems we have today which i don't i personally don't feel are at all conscious yet right from my own you know everyone probably has their own view but i don't feel yet and then you can have conscious entities that are not that smart for example my pet dog it's lovely but he's and he's pretty conscious i think he dreams and everything but he's not that smart right yeah and that's provable so i think you can have double dissociation here you know many of the higher animals dolphins monkeys and so on elephants i think most of us would view that they have some you know their conscious beings but and they exhibit all of that that sort of behavior and of course of course they're also made of carbons and and the same neurons and types of things we're made of so i feel like it's pretty obvious they have some of those qualities but they're not necessarily that smart so it feels like you can dissociate these things and i think to the extent that you can i feel like we should you know we're coming up close to the end of our session and the end of you know the summit sad but what should we think about in terms of the whole notion of ai sovereignty right because obviously one of the things that is kind of going around in a lot of countries right now is oh is this really important and you know one of our earliest conversations was with you trying to solve this problem so i wanted to share some thinking and obviously it's updated for whatever that was 13 14 years ago but yeah well look i think first of all first thing to say is it looks to me like there's going to be you know half a dozen six to eight if you include chinese companies company you know offerings at the frontier or near the frontier and it feels like there's going to be leapfrogging and maybe something's like three to six months ahead but it's going to be relatively close so there's a lot of options if depends if you're talking about an enterprise or a country right so i think as an enterprise you have a lot of options to arbitrage between these different things some of them are better at certain things you know multimodal understanding versus coding or it's very hard to be better on all on all vectors of importance at once so i think that's going to happen and then open source which also has its risks because how do you as we've discussed for many many years how do you restrict bad actors using that whilst enabling of course as many good actors scientists and others to use these fantastic tools so that's an unresolved problem but open sources you know maybe only lagging six months to a year behind obviously we need kind of western open source solutions but i think at this point if we're talking about countries as well we're going to have to look at the data as well all companies that need their data to be private or they want to fine tune further on their own you know sensitive data again be that enterprise data or sovereign data i think that's that's going to be one solution there's also looking at what are the downstream things that can be done with these tools you know big companies and big vectors sectors that can benefit from these tools like healthcare that you can just use this you know almost like commodities and then build the value on top of it rather than you know i wouldn't recommend that you recommend every country for example trying to build a frontier model that just doesn't make sense to me so we're gonna they're gonna have to find some other way of shortcutting that yes well and also collaborating and working out arrangements with multinational companies like your own i think one of the reasons i wanted to kind of end on that question is that part of the theme of of the summit is agency and agency is not just a question of individual agency like how do you take agency but it's also a collective how do we take agency and i think part of the thing that's actually important in the age is that we take agency not just as any one country but as countries together and it's part of how we have a we are in this together and we are building for the benefit of humanity and i think that's the kind of arc of we start with sarah fryer and thank you for joining us because ending and demis has been with such intensity that i was so grateful that you set up the agenda for me thinking about how to help in this from our very first you know discussion at king's cross so let's give demis a hand thanks again to demis for joining us at the twenty twenty six masters of scale summit this conversation was recorded live on stage in san francisco at the amazing presidio theater i'm jeff berman thanks for listening every business hits the same wall the moment your business outgrows the systems that got you here spreadsheets everywhere five different logins data scattered across entities that don't even talk to each other if your finance team spends more time finding data than using it that's when you need the intuit erp intuit enterprise suite is the ai native erp solution that's powerful painless and proven built for businesses scaling fast without the complexity that usually comes with it from the makers of quickbooks learn more at intuit dot com slash e r p hey listeners bob here if you listen to rapid response on masters of scale you may be missing half the show because every friday we release a second rapid response exclusively in the rapid response feed the guests and topics are just as compelling and timely from to nass's administrator to the lessons from the devil where's prada it takes about ten seconds to find just search rapid response wherever you listen to podcasts and hit follow to make sure you never miss an episode i hope to see you there humans will never be more intelligent than a i there's gonna be two types of companies those are great at a i and those that went out of business because they weren't how do we build a future that is human centered i'm ronald how you be and on my podcast pioneers of a i we answered that question and so many more as an a i scientist entrepreneur and investor i know what it takes to build a i that works for everyone every week i sit down with the pioneers shaping our future and we take you behind the scenes of the a i that's transforming our lives find pioneers of a i wherever you tune in masters of scale is a wait what original our Executive producer is Eve Trow. Senior supervising producer is Trisha Bobita. Associate producer is Masha Makatanina. Video editor is MG Frederick. Senior talent executive is Stephanie Stern. Mixing and mastering by the audio boys themselves, Aaron Bessanelli and Brian Pugh, with original music by the remarkable Ryan Holiday. Visit mastersofscale.com to find the transcript for this episode and to subscribe to our newsletter. And be sure to subscribe to our YouTube channel.