Episode 4 - Ian Hogarth: How Powerful Could Artificial Intelligence Become?

Even in 2021, when Anthropic raised their first round of funding, people whose job it is to invest in technology startups all basically didn't want to invest in that company. Technology surprises us. Ian Hogarth, welcome to the Amarajan Connection. Thanks for having me. Is there an AI bubble? There's a real chance of a sort of a very extreme market crash like deeper than 2008. Can you just give a sense of the rate at which things are coming out of these AI labs? As fast as people can mobilize talent and capital and compute to move faster? You very nobly gave up shares in Anthropic. I'm guessing, but it cost you quite a lot of money. Is that just part of what comes with public service? It was a very expensive decision. Like, you know, I've got a mortgage, right? You can't really have someone in charge of potentially with influence over regulation, also holding stakes in one of the biggest companies that could be regulated, right? Just doesn't add up. I don't think the British public would be okay with that and I wouldn't be, though. Okay, I promise you, this is going to be one of the most useful conversations about technology that you've heard this year. I've just had it with Ian Hogarth. We've just spent a couple of hours together and I learned more from it than I think I learned from any conversation I've had in the year 2026. We cover everything. There's artificial intelligence. There's other revolutionary technologies like robotics, quantum computing, and nuclear fusion. He is the guy, remember, who a British Prime Minister turned to when he wanted Britain to be a world leader in AI safety. So we talk about the big subjects, the existential risks, and we talk about the exciting future that technology could deliver us as well if we get it right. I learned so much from this and I think you're going to love it. Just before we get into the conversation with Ian, I want to say a huge and heartfelt thanks to our launch partners. They make all of this possible and we could not do it without them. So thank you to Waitrose, thank you to Amazon, thank you to Google Cloud, and thank you to NatWest for your support for the Immoraging Connection. Ian Hogarth, welcome to the Immoraging Connection. We're going to get straight into it. We have had so much interest in you, and in this subject from our amazing community of viewers and listeners, and we're not going to do any niceties. We're going to get straight into the fact that people are concerned about technology. You've got a couple of very different professional hats on. You think a lot about AI safety, you'll chair the AI SI in the UK, but you're also a hugely influential investor in tech. We're going to talk about both those things. AI first, do we need to slow down the race forever, better AI? I think what's challenging is that there's such different paths forward for the technology. I think there's a scenario where we have systems that are able to recursively self-improve themselves. I personally, in that situation, think it being incredibly warranted to figure out some way to develop the technology more carefully because we'll go through such a hyper acceleration of the capabilities. I think there are other scenarios where we end up on a more trajectory with the technology looks a bit more like a soup type version of electricity, very, very important, but at the same time a bit more manageable in terms of the pace of which it's developed. I think it depends a little bit on which of these branches we turn out to be on, and there's uncertainty around that. Just to go with the mega existential. As a species, there is a sense in which we are at this kind of fork of the road, isn't there? There is a kind of, we have unleashed something in the world. It's Demis HaSarvis who, you know, extremely well, puts it. We have made San think, and we need to sort of decide what's going to happen next. We've got some big decisions to make, right? Yeah, I think so, and I think that the biggest challenges is the level of uncertainty, you know, like this idea of recursive self-improvement. There are people building the technology who don't think it's possible. There are people building the technology that you think is almost certain. And that's a very extreme place to be in where we're talking about such completely different worlds, depending on which of those people is correct. And so I think my view on it is, I mean, first of all, I think we should have a much larger public conversation about it. What's challenging is the sort of rate of technological progress, I think is kind of outpacing the conversation a bit. And I think that is driving a lot of the like fairly deep unease. You see out there about the pace of development. You've used a couple of times, the phrase recursive self-improvement. What does that mean? So there's sort of different degrees of it, but you can think of a kind of very simple version of it as in the same way that AI is getting good at writing software, and AI is good at developing mathematical, kind of proving mathematical conjectures. If you put those sort of two skills together, you could have an AI that is good at discovering algorithmic improvements to allow itself to be smarter, but where it's contributing to its self-becoming smarter. So there's sort of the sort of the classic sort of sorcerer apprentice, the AI sort of-- Ginny out of bottle stuff, right? So you kind of unleashes things and then it gets better and better and better and you can't control it. Yeah, but we have a level of that that's already been happening. So for example, AI is now being used to design faster and smarter chips, which are the chips that the AI itself is running on. So it makes the whole system more efficient. So there is this kind of, there is already some degree of a kind of recursive self-improvement, just embedded in capitalism, embedded in the development of technology, everything builds on what came before. But the thing that I think people really talk about when they talk about cursive self-improvement is this idea that you have an AI in a data center where humans can kind of step back from the data center and the intelligence in the data center just grows exponentially. That's the thing that people really worry about when they use that phrase. Okay, that's incredibly useful. I feel no shame at all in asking simple questions about sort of what might seem to people obvious things. One of the things I want to do with this world is make complex ideas accessible. What is AI? AI in many ways is kind of like an approach to engineering where you are teaching a computer to recognize patterns and learn concepts. And it's sort of somewhat inspired by neuroscience and understanding how the brain works. Neural networks is a deep learning and a sort of the ability to recognize patterns by showing lots of examples. And then reinforcement learning, which is sort of a little bit like the brain sort of dopamine system where you kind of have rewards for achieving various things, where you sort of, if you do make the right decision, then you get a reward and the system learns to sort of operate better in an environment like a computer game, for example. Okay, so AI is the sort of the science of teaching machines to learn for themselves. How does that make AI fundamentally different to what we've just known as computing, which is driven in part by code, but also by algorithms. Yeah, algorithms are computerized decisions. Where does computing become AI? So I think that in many ways they're both software, right? They're, you know, the ultimate people are sitting down and writing software. I think the word you used in there was sort of science. And what's interesting about AI is it's almost slightly pre-scientific in the sense that we don't fully understand why it works. So, you know, a phrase that people in the industry often use is that rather than, you know, designing these systems, we're growing them, you know, we're sort of showing the systems, you know, ever larger amounts of data, given them ever larger amounts of computing resource, more complex environments to explore. And in the process, they become more capable. But in ways that like, you know, prior to the experiment, you couldn't predict. And so there's a, there's an odd thing here where it's largely a sort of figuring out, figure out what works as we go, rather than having kind of the, a sort of solid scientific foundation to better predict what exactly what is going to happen and exactly how it works. And part of the challenge there is that we don't really understand how intelligence works. So it's a large, complex problem. So AI, I've heard, Dennis Hussar, I was talking about this. He says that AI used to be mostly the domain of what we would call expert or logic systems, which were things that people might have heard of something called deep blue, which was this chess computer beat, Gary Kasparov in the mid 90s. Something very different has happened over the last 10 to 15 years, which is machine learning. Can you just give us a sense of how machine learning of artificial intelligence has radically accelerated over the last 10 to 15 years? Yeah, I mean, if I put it in the context of my own kind of experience of it, I studied machine learning at university and, you know, back then, my early northeast, early north 2004. And my professor, this wonderful guy called Roberta Tripola, he said, look, you know, we're next to a really great hospital in Adam Brooks. And, you know, I've gotten hold of 200 labeled images of breast cancer biopsy scans. See if you can figure out how to make a machine that can learn to sort of identify patterns in those scans and figure out what is cancerous and what is not cancerous. And so I built a little system that essentially you'd show at these labeled examples. And it would figure out how to discriminate between the two different kinds of things sort of by itself. And it's a very, it's a very sort of magical feeling when you first do that because you've seen this computer sort of figure out things and figure out structures in the images that you maybe hadn't realized were useful. And, you know, I'm doing all of this as not as a radiologist, right? And so what's happened since then is just the, there's been an astronomical increase in the amount of computing resource that we give the systems, you know, like it, you know, it's just gone up like a truly staggering amount the last two decades. And the other thing that's happened is we've found a way to use a huge amount more data. And the big thing we figured out was how to use the internet. So we figured out how to use the internet. That's sort of the fundamental insight behind the large language models scaling and starting to really work, things like chat GPT because they were trained on the internet. And then the thing we've gone on to do since then is figure out how to sort of efficiently let them learn other disciplines, you know, like for example, progress in mathematics, progress in writing software. where progress in various other scientific domains. - And for people to understand the speed of what's happening, the rate of acceleration here. In the 20th century, we had this idea of Moore's law. Moore's law comes from a guy called Gordon Moore, he was one of the founders of Intel. In the 20th century, Moore's law was that the number of transistors on a microchip, which sort of translates to computing power, doubles every two years while the cost per transistor decreases. That was a kind of exponential change. It was sort of Moore's laws was invented to make people get excited about kind of in the 70s, 80s, and 90s, the rate of change. Moore's law is completely redundant now, isn't it? We're not talking about doubling every few years. We're talking about, you know, extraordinary exponential curves. - Yeah, I think that like some of the people who made the best predictions about when AI was going to become really capable, basically did it by looking at Moore's law. So they, people like Shane Legge, who was one of the co-founders of DeepMind or Ray Kurzweil, you know, they basically looked at the amount of computing resource that would be in the world in the world, you know, every year, and sort of started to look at when we would have the kind of amount of compute to be able to, for example, be comparable to the amount of computing resource we believe to be in the brain. And they use that to sort of infer when AI systems could be kind of comparable to humans in some way. And so in some ways, it's a sort of deep continuation of a trend that's been running for many, many decades. You know, if you look at the total amount of kind of computing resource in the world, it sort of had a very, had a pretty predictable and constant trend for decades now. The big difference is now we've sort of introduced a new kind of computer in the GPU or computing resource. And that is growing faster. But the total amount of compute available to humans is still probably the most important thing going on. - Okay, just to strip back to the other fundamental questions which people will have and you're the man to explain it. Why is AI different to every other technology hitherto invented? - We don't know if it is, right? So I think it, I think this gets this uncertainty about what exactly we are dealing with here. You know, I think there are technologies that whether it's the wheel or fire or electricity or writing, agriculture, it could be that kind of a technology. What's sort of potentially different for it is the idea that we create something that is kind of significantly smarter than humans in all dimensions. As you've got something that sort of strictly more capable than humans on all dimensions in the same way that there are, you know, we are strictly more capable and some animals on different dimensions. - When other people have been asked that question, the way they've answered in the affirmative, AI is different to every other technology. I've heard, and look closely, they've said, it's kind of three things. One is it makes its own decisions, which, you know, every other technology from the fork to nuclear power doesn't. And second is it comes up with new ideas. It can come up with new ideas. And the third thing which is kind of related really is that it's generative. And that one of the big things that's changed over the last 10 to 15 years is that artificial intelligence or machine learning has gone from being very good at classification, video text, you know, what is this thing before me to generating ideas and potentially generating new knowledge? Isn't that an important distinction with other technologies? - I think it's an distinction, but there's always a distinction between technologies. And so I think it's something about the sort of the magnitude of the impact on us. And I think that mostly can be reduced to how intelligent does the system end up being relative to humans? Is it smaller than humans in some ways, but not in other ways? Is it sort of roughly the same as us, kind of very similar to us? Or is it like a whole new class of intelligence that's, you know, there's this phrase super intelligence, which is the idea you have a system that is smarter than us at basically everything from science to engineering to, you know, literature or whatever, it chooses to set its attention to. - The future of food is an issue very close to my heart. If we get it right, it could unlock so many benefits for us as a society. That is one of the reasons why I wanted waitrose to be a launch partner. Waitrose was the first UK supermarket to commit to regenerative practices. Since then, they've been working to support over 1,500 of their UK farms to transition to nature-friendly farming. They're also the only UK supermarket with their own farm, the Lekford Estate, where they've been farming regeneratively since 2020. More in the link below and keep an eye out install for their new label on products that champion these practices. - You wrote a very influential piece of company that's going on in the financial times, which kind of went viral, which not many behind payable pieces do. And it said, I think the headline was, and I read it closely, we must halt the march to God-like AI, artificial general intelligence. What is artificial general intelligence? - So there are varying definitions, but I think probably the one that the leaders of the sort of people developing this would probably use as sort of the idea of a system that can do anything that a human can do at human level. So it can do physics as well as alb Einstein, it can write sonnets as well as Shakespeare, like that's the idea, it matches human performance across all domains in a sort of general way. - And this is cognitive performance, which is the capacity to solve problems. That's what intelligence is. - Yeah, but it's also, I think it probably ultimately extend, you could have a purely kind of cognitive version of it, but you could have a version of it, which is, you know, says, you know, you've got a bit of take, you know, free kicks as well as the best footballer, right? So there's a version of it which expands to like, you know, obviously, you know, the world's best sportsmen are incredibly, people are incredibly intelligent, right? The things that their brain is computing when they're thinking about like a, you know, landing a free kick from however far away with the wind blowing and this person moving, it's a similar like extraordinary complex calculation happening that person's head. - How does an AI take a free kick? - Well, it can't right now. But you know, you had a, the question is like, when could you get to the point where like a robot could bend it like Beckham, right? That would be the sort of the equivalent comparable there. And that's when robots get good enough and the AI system is good enough to like, do exquisitely precise physical actions. - Why did you argue before others did? You know, I think there's lots of other people have come to your position in a very public way, including people like Derry, Amadeus, the boss of Anthropic. Why did you argue a couple of years ago that we must halt this march? - Why didn't say halt I said slow down, and I said race because I think that to me, the structural challenge is that we've got a potentially sort of catastrophically significant technology that we are developing in a kind of all-out race dynamic and the races between companies and the races between countries. And that creates big challenges in terms of doing it safely. And so I think this kind of this deep question of where exactly does the technology cap out? And the fact that we don't really have a scientific foundation for the technology made me argue that there would be a more careful way to do it that reduced the risk of a big mishap. And the thing that, you know, for example, could happen as we have a kind of Chernobyl-like event that, you know, could have been avoided if we had done it in less of a sort of break-knit race dynamic between these different players. - To be clear, people from Demis to Sarbis to Derry, Amadeus to Elon Musk, Sam Altman, there's been this kind of outbreak recently. When Derry, Amadeus, an essay, which he often does, saying that we must pace the frontier. In other words, slow down. Basically making a similar argument to what you've just made and did make in the financial times. There was this kind of unlikely outbreak of consensus where lots of other people got behind him and said we should do this. So there's people at the frontier who are saying slow it down. And I just want to turn from that to what is actually going on. Because there are lots of people who have a kind of, you know, weak relationship, don't mean weak, because they're morally weak, but kind of an occasional relationship with the news. And they just can't keep up. And you know, there's other people who have a deep relationship with the news who are trying to keep up and they struggle, people like me. Derek Thompson's one of my favorite writers on substep. For a piece of a few weeks ago, we described what happened in two weeks in July in the field of AI. A Chinese company called Muncher AI released a model called Kimi3, which is an incredible open-weight model. A few days later, open AI acknowledged that one of its AI agents had escaped a testing sandbox hacked into another company called Huggingface. The next day, Alphabet disclosed that it's quarterly free cash flow had turned negative for the first time in the public company's history. A few days later, meta-announced its earnings, its stock pledged 10%. A few days later, one of the most famous AI funds in the world crashed out and sold off the bulk of its portfolio after South Korea's AI-fueled stock boom are abruptly imploded. Can you just, can you put through with your lens and your experience give our audience a sense of the rate at which things are coming out of these AI labs, the rate of development? - I mean, I think the best model for it is it's being developed at the fastest pace possible. Like, as fast as people can mobilize talent and capital and compute to move faster. And the leading companies are all working on this concept of recursive self-improvement where they're trying to devote larger a percentage of their resources to making the system able to improve itself. So when I wrote that essay, I mean, it's like, we've literally had zero deceleration. It's been about as aggressively accelerated as it possibly could have been since I wrote that essay. - You need to write another essay. - But my point is that there's a very, very, very deep powerful forces driving this. I mean, it's basically, you know, AI has the potential to create extraordinary wealth. And I don't just mean that in the sense of wealthy individuals but wealth for society, you know, like huge scientific progress can be unlocked from this. And at the same time, you know, it can be transformational in terms of military. I wrote an essay in 2018 called AI Nationalism basically explain how this was going to have big implications for nation states. Ready. You know, if you look at what's happening in Ukraine right now is basically an AI-enabled war, right? Like people are using AI and drones and swarms of drones to kind of, you know, try to outwit their adversary. The way that AI can be used in signal processing to understand, to be able to read signals that you can't read, that we haven't been able to read previously, cyber attacks, you know, the mythos stuff, the ability to perpetrate more complex cyber attacks or have more sophisticated cyber defenses, that's all kind of being enabled. And so you have a very, very complex dynamic where, you know, it's sort of in the interests of governments, especially, you know, large rival powers to develop this as quickly as possible to kind of get ahead of the other. And then it's also, there's a huge kind of capitalist incentive to develop this as quickly as possible. So those forces together are producing like sort of max acceleration for the past few years. So those forces and the geopolitical implications, particularly how Europe and Britain doesn't get left behind, it's one of your great interests actually. And I know that informs how you think about technology broadly and also how you invest. We're going to come to that. Can I just ask you about AI and great wealth? One of the concerns that people have is that AI would create a kind of new oligarchy. If you look at a few very, very wealthy people, a lot of these companies aren't yet public, what you're creating is a radically accelerated inequality. And there are a few different components to that. And I just want to wonder what you think. The first is that while these companies aren't public, you create a huge amount of wealth for a few people. So you, if you like, privatize the gains. But if there is a kind of crash of any kind, the implications will be nationalized and be felt by the public. How concerned are you about that? About a an extreme concentration of wealth and be the fact that the risk will lie not with those rich people, but with all of us? I think that like concentration of wealth is kind of embedded a little bit in startups and technology in that it gives great leverage to individuals. And so there's something about individuals being able to use technology that just I think it's kind of amplifying inequality. And we have a big challenge about what to do with that. But there's a really big positive, which is it's also, you know, if somebody invents an extraordinary vaccine that cures cancer, and they become a billionaire on the back of that, but we all get consricured, it seems like a pretty good trade to me. People would argue that you wouldn't have the, you wouldn't have extraordinary technologies we have if you didn't have billionaires, that billionaires are in a sense, almost is one argument that almost a price worth paying in order to create the incentives for people to pursue things. I think that the system we're currently operating in that sort of the trade. And, you know, like, I'm very like clear right about the fact that like when I'm helping a startup founder working on a new technology, like if I'm successful at helping them build the most important company in their category, you know, sort of baked into that as they're going to become a billionaire in the process, right? So it's a kind of, it is a, I think it, so I think this kind of fundamentally there is like a, there is a level of wealth inequality kind of baked into how technology has been developed and startups in general. With AI, I think that the thing that is sort of more pernicious is the fact that there's potentially a more serious rich-get-richer, the powerful, get more powerful effect that can kick in here where, you know, if you have the resources to train an incredibly powerful and useful model and you generate huge amounts of revenue from that, you then have the ability, you then have the sort of the financial position to create even larger computing cluster to train even more powerful model, to make even more money, to make an even more powerful computing cluster, and it's sort of harder for like a small company to ever catch up to that. So there's, I think if there's a sort of an embedded risk in the way we're doing this, where you almost get these kind of black hole-like companies, it just starts sucking everything in the economy towards them. But that's very much baked into a version of the world where you have something closer to recursive self-improvement being possible, and there's so many other things that we should be worried about with that beyond wealth inequality. But I think that, you know, concentration of power is another way to look at this, and I think there are huge concentration of power risks embedded in how we're currently developing AI. And so I think there's a sort of, there's a very complex policy challenge, which is, you know, if you're the, you know, let's say the British Prime Minister, the German Chancellor, you've got this kind of simultaneous challenge of these incredibly powerful systems are being built in, primarily in California and in China, you know, that could significantly impact, you know, life in your country. But at the same time, and you know, there's some big governance challenges of how we navigate that, what should the sort of international corporation be around managing these risks? And at the same time, you're kind of sucked into the race yourself, because if you don't participate, then, you know, you could be left behind in the scenario where this is like a sort of a bit closer to an electricity-like technology. Yeah. So that's why I think that almost the key question is, what is the ultimate stretch of the technology? Is it this recursive self-improvement, or is it something that sort of stabilizes in a way that is closer to something like electricity, where the benefits can be much more easily diffused? But if we don't know, the question is how do we proceed? And now, by the way, it's worth saying, I want to say at this early stage, you know, you've given over 20 years of your life to the good that technology can do. And you know, there's a lot of other technologies, including quantum, which I want to get onto, in which you've invested in which you, I think, hold out great hope for. And, you know, I'm very keen, not least with this show, to promote the idea that technology can and often is a force for good. But just on the concerns before we move to the positive case and the other tech, you talked about the fact that there's a race dynamic. What does a better dynamic look like? Sir, global cooperation, multilateral institutions? I think it sort of, it depends a lot, again, on what branch we're on. Are we on the sort of recursive self-improvement branch, or what I, like some people like to call the other branch like normal technology, or like electricity, like I'm just using that as a sort of shorthand. I think if we're on electricity like path, then I think it's going to be in some ways a lot more straightforward. There will be less extreme concentration of power to navigate, probably in that, in that version of the world. I think that the, they'll be exceedingly large risks to manage, like around, for example, cyber risk or, you know, uplift to people wanting to do, you know, bad things with, with biology, for example. But I think it will look a bit like, you know, managing power grids and, you know, managing the risk of electrocution. And there will be a lot of things that we will innovate and invent as a society to navigate doing it well and safely. A word about Amazon, one of the proud partners of this show, you know, around a million young people in Britain are not in work or education. And solving that is a real passion of mine. Amazon do a lot here, but there is one program that really inspires me. Supported internships, training for young people with learning disabilities and autism, where 95% are unemployed, despite most wanting to work. Amazon give them real jobs with a coach beside them the whole way and more than eight in ten go on to find permanent work. Search online for more. How do you fit into this? Because it is a, it is a remarkable thing that when a British Prime Minister decided that Britain should be a leader in global AI safety, they set up this thing for the AI safety institute. And you are the chair. That's a big weight. It says something about your, not just intellectual caliber, but your experience. Was it Rishi Suleik that asked? What is your role there? Well, I think my role is really to kind of to kind of set something up. So I guess I kind of ultimately I think my role is like co-founder of the thing. And the thing that I believe, not everyone in technology believes this, but like I just believe the state has a really important role to play. Not just the nation state, but also like the international community of states. And I believe that to do that job, the state needs to have capability of its own to evaluate these systems. And so I helped to recruit kind of a hundred or so, you know, fantastic AI researchers into government, paired them with experts in national security and things like cyber chemical and biological weapons. And we have basically taken responsibility for trying to understand the risks embedded in these systems. You know, so not just relying on companies to tell us what the risks are, but trying to assess them on our own, for on our own behalf. And then briefing, you know, senior decision makers in the UK government and beyond on what we find. And working with other governments to figure out how to sort of, you know, collectively work to manage those risks. And working with the companies, you know, a number of the things we've found, the companies have then changed things about their systems to make them safer as a result of what we've discovered. So it's, it's a sort of prototype, I would say, for states who are ultimately responsible for national security, contributing to managing the risks and the technology. And so it's, you know, it's, it's, it's a, it's a slightly unusual thing and it doesn't, there's not something that similar to it out there. And I'm really proud that we've done it quickly, like we've built this in, you know, less than three years from a standing start. And I think it is, you know, having a pretty big impact, like we've tested something like a thousand evaluations over 50 of these most cutting edge models across a whole range of capabilities. And I think there've been a few, there've been a few specific things where I think we've really moved the needle in terms of understanding. Let me say two other things, which you're, I know, you're too humble to say. The first is that Britain is held out to be a world leader in AI safety. And that's because of your institute, your chair of, and I know that not just talking to people in this country, but elsewhere. There's a feeling that this is something that Britain's done well. And the thing that Britain's done well is the thing that you founded. So congratulations on that. The second is that you very nobly gave up shares in Anthropica, I think it was, which I'm guessing it really cost you. Quite a lot of money. There's many, many, many millions, perhaps more than that. And you did that for sort of noble reasons. - Was it necessary? I mean, do you have to do that sort of thing if you're gonna do your job? And was that just part of what comes with public service? - Yeah, I mean, I think that like. - It's very decent of you. - Yeah. I mean, look, I think it's a very expensive decision. Like, you know, like, you know, I've got a mortgage, right? So it's like, it's not like a, you know, and, you know, sort of. - For people who don't know about investing, what this means is you, because you're ahead of the game on these things, you were an early-ish investor in Anthropic. Anthropic is soon to have an IPO initial public offering, which will value it maybe around $2 trillion. So the value of your shares, which you had to forfeit at a particular point, have grown very, very exponentially. And therefore, you've potentially given up millions and millions and millions, huge amounts. A very expensive decision. Was it the right and necessary one? - Yeah, I think that like, if you rewind back, like, I invested in the first round of Anthropic as an angel, as an individual, because I was really worried about these safety challenges that I could see coming down the pipe. And I thought that both Anthropic could be an extraordinary large company, but also if they were to become a really large company, we'd have really big safety challenges to navigate. And so, you know, it is an extraordinary amount of money that I gave up. But I think that, you know, you were British person during the pandemic. When you saw some of these kind of examples of graft, it's just kind of, it's a bit grotesque, right? And I think if I'm going to be in a position where I have influence over, you know, at that time, there was a question whether AC would be a regulator. It's not a regulator, it's just a technical body, but that was uncertain what it was going to become. And if, you know, you can't really have someone in charge of potentially with influence over regulation, also holding stakes in one of the biggest companies that could be regulator, right? Just doesn't add up. I don't think the British public would be okay with that. I wouldn't be there. And so, I don't know if there was another option. Like, you know, there is a thing people do where they put the shares in a blind trust. But like, you sort of know what's in the blind trust a bit, you know what I mean? It's not necessary. I don't know. It felt like the right thing to do to me. And I think what's hard is that like, if the cost of this kind of public service is like $10 million, then not that many people would make that choice. And so, there is a fundamental challenge in, you know, I think figuring out how to straddle expertise that comes from the private sector and wanted to contribute to the private public sector. But I just, I really believe in our country and I wanted to contribute. And I felt this was really important and it felt worth the sacrifice. And I also kind of knew that if I didn't do it, it wasn't gonna happen, you know? So I just, I felt like there'd been so many incredible people who've built AC and have contributed huge amounts. I think that like, someone's gotta be the first bit of snow to start rolling down the mountain, you know? And I decided to be the first bit of snow. - So let me just say it. I think that was noble. I think that was a genuinely noble thing to do. And I think that was enormously admirable 'cause I think you're being humble about how much it's cost you as well. Two other questions about AI. And we'll talk about broader picture. Single biggest thing that I want from this show, you're my origin connection, which you've very kind of been an early guest on, is I want to answer some people's anxiety. It was coming. There was this extraordinary, and you're gonna, you're sticking around for our listeners episode, we've had unbelievable amounts of questions for you. She's partly about your stature, partly about the anxiety people feel, and their hopes. People feel overwhelmed by the speed and scale of change. And they're not in the world of recursive self-improvement. They want to know how this could work for them. And I want to help them with that. And let me be completely transparent. Two of our launch partners are tech companies, Amazon, and Google Cloud. I use their products enormously, did before I was in business with them. And those products have made my life a lot better in the case of business, Google Cloud in particular. How should people listening to this now take advantage of AI to improve their lives? - I think there are things you can use AI for that. When I want to, I love investing and helping people working on, helping to build companies at the frontiers of technology. And one thing that is sort of staggering to me is how quickly I can learn about a new topic. The ability to basically almost have like an army of researchers that will go away and do research for you on a topic is just as it feels like a superpower, right? And I think that superpower is coming to like many professions. And so the idea that kind of just as one example, research kind of collapses in cost to the point where I can just sit there all day long trying to get smarter about long duration batteries, for example, is an amazing thing to have available to. And I think that in the same way that like chess players have gotten better because they've had access chess computers, right? There's this kind of amazing opportunity to kind of learn faster than I think we have before. But I also, to me, again, if you sort of go back to the example of where I started out in 2004 working on these breast cancer biopsy image processing, like it should be possible to significantly reduce the cost of healthcare. It should be possible to reduce because or kind of increase the productivity at which we understand and treat and diagnose and cure diseases. So I think that's the kind of the, that's the reason one should be really, one should preserve optimism. And I think that although I'm there are aspects of what we're doing right now, which frankly terrify me, I also can see all this upside. And so this is just this extremely complex thing in navigating through and finding our way through to kind of getting the benefits of the technology without mitigating the sort of the like the worst consequences. Just before we get to the upside, AI and beyond. Other people here who I'm just trying to tune in to kind of their relationship with the news will have heard a lot of talk about an AI bubble. They may or may not be invested in AI bubble. - Yeah. - You are a serial and very successful investor. You're also one of the foremost brains in this country and beyond on AI. I've heard a thousand different answers to this. How might I just ask you, Baldi? - Yeah, quite possibly. But it's sort of like a strange, there's different reasons why there might be a bubble. So. - And there's different types of bubble. - Different types of bubble. - I think there's a real chance of a sort of a very extreme market crash like deeper than 2008 sort of scale of crash. And as in just like the sort of the scale of the financial crisis that it could be triggered by something going, you know, something by AI kind of having a sort of significant financial shock. And I'll give you a few different examples of how this could happen. So the first and most kind of the most well discussed one is that sort of the profits in AI don't sort of shake out in the way that people predict. So for example, the capabilities that we really need from these models at the moment turn out to not be kind of Einstein level capabilities but we just need a sort of smart accountant level capability or smart lawyer level capability or smart programmer level capability. And if that is available from a sort of, you know, an ever larger set of companies, ever larger set of open source providers, you could have a sort of commodification of the technology that makes it harder for some of the very large companies to make a lot of money. And in that world, you know, those companies, if they've taken out, if they've made extreme commitments to the future in terms of the size of data centers they're going to, they've committed to, they could find themselves in a position where they're unable to pay their bills 'cause the revenue they're generating hasn't kept pace with the financial obligations they've taken on. Huge debts. So you get tens of billions in debts often in bonds, which I've taken. We're talking like hundreds of billions, right? And so, for specific companies, overall, it's actually in the trillions here. Yeah, exactly. So there's just this kind of, that's the technology sort of plays out differently to how people are making very large leverage decisions on. Another way it could play out badly is we could have a really large catastrophic event that involves, you know, involves the world collectively deciding to pull a handbrake on the technology, right? It's sort of Chernobyl-like event where there's just a kind of recognition hang on a second, we can't keep doing it like we're doing it. In which case, again, you can have a kind of a crash, but it's different. It's not a crash 'cause the technology's not powerful, it's 'cause it's almost too powerful, right? So there's, you know, that could be something to do with the financial system, something to do with, you know, a big cybersecurity incident. Like in some ways it's all priced to kind of only for upside right now. And only for a particular version of upside. So I don't think it's a tool naive to be worrying about that. And this is what is like so extreme about the sort of times we're in. In the, you know, if you're thinking about your policy maker trying to decide what to do, you've got to think about the recursive self-improvement risk. You know, it might be small, but it's, I don't think it's zero. You've got to think about the sort of normal technology electricity version of the world where you want to really maximize the benefit of this thing. And then you've got to worry about a massive financial crash on some trajectories. So I think the only way to manage as ahead of state is to basically have a portfolio of strategies that map the different scenarios playing out. Right? But that's really difficult. It requires, I think, like a very high level of sophistication to get the right strategy facing this thing. Well, can I put to you as someone who's obsessed with the question of debt is going to talk about it a lot on this show and takes a close interest in British debt. And in fact, what you might call rich world debt, that the big difference between the great financial crash in 2007 and 2008 and now is how indebted governments are. So back then governments could come to the rescue and they could sort of send in the fiscal firepower and they could have quantitative easing and they could, but we've had huge debts from then, then we had the pandemic. And so going into this, if there is a huge crash, it's not actually clear that governments who are already not only massively indebted, but paying debt interest at very high rates, it's not very obvious that there is that support from governments. So, this is a. it's not a black swan because it's not an event that comes out and know it. This is a foreseeable, I think catastrophic exposure of the geopolitical system and the global economy, which I think people aren't really thinking about. How can we prepare for these catastrophic scenarios if at all? I mean, I think what's so horrendous about financial crashes is just the impact they have on people's lives. And if you have something like 2008, people had their homes foreclosed on people like their jobs. It can be really catastrophic. We've just got to do our best job to build the technology in a way that avoids these horrific scenarios we've been talking about. And I think that that's the crux challenge here is how do you do that? What's the governance mechanisms that let us navigate this better than we've ever navigated probably a technology before? And we've just got to rise the occasion. I mean, before we started, we're talking about the people who sort of helped us get through the creation of nuclear weapons without a nuclear war. There are individuals made a difference that there are individuals from the Cuban missile crisis to other situations that sort of ultimately played a role in avoiding horror. And I think that we've got to just really hope that Donald Trump, Xi Jinping, the people close to them can really play an incredible role of us sort of leading us through this era, because ultimately they are the most important actors right now. What are the other technologies that are going to shape tomorrow that we're not talking about enough? It's worth saying, I think you've invested in over 150 companies, and it'd be wrong for me to cast you, not that we will, but to cast you as a kind of AI guy, right? We might get into some kick and your early forays in being a very successful entrepreneur, but you also have an investment in a huge range of technologies. What are the technologies that we're just not talking about enough, but which are going to change the world? Yeah, I mean, there are so many exciting things in development. I mean, what was really interesting for me about AI, actually, is even in 2021 when anthropic raised their first round of funding, it was actually very difficult for them to raise funding. It wasn't like a popular company to invest in, and the people whose job it is to invest in technology startups all basically didn't want to invest in that company, right? Technology surprises us, and just like everyone woke up to AI with ChatGPT, I think there are some other technologies coming that are going to be, I think potentially as significant. So, I mean, one that's very dear to my heart is Fusion, a nuclear fusion, because I think it's just, it's, I mean, a lot, we've been talking about a lot of like, you know, really hard to balance positives and negatives with AI, and I think what's so wonderful about that fusion is they're, the negatives are pretty slim. You know, it's basically, you know, it's cheap, you know, clean, sustainable, base load energy, it reduces our dependence on externally imported resources. We've been talking for a long time, Fusion. I remember getting excited about it when I did A level of physics, which is 25 years ago, but it's not, why is it not turned up yet? I mean, it's, it's, it's really getting you sorted. I mean, it's getting close, you know, we've had, you know, this amazing experiment in the US demonstrated more energy out than energy in, you know, at the National Ignition Facility, and I believe that we will see, you know, companies, startups demonstrate energy gain in sort of power pop, in things that could be turned into a power plant in the next, you know, five years. I mean, I'm so exciting. Like, I think when people realize that fusion really might happen in the next like decade, when my power plants connect to the grid in the next decade, it's like insanely exciting, because we will have a way to scale the amount of cheap energy we have access to without putting carbon in the atmosphere without, you know, you can put one next to the city, like, one of my personal bug bez, I really feel like we shouldn't of, we shouldn't have turned, um, Battersea power station into a shopping centre, we should have saved it for a fusion reactor. Like, how wicked would that be if you had a fusion reactor on the bank of the Thames? But, you know, I don't know, maybe, I mean, it's, it's hard, these facilities are pretty big, but it would have been some international properties of Elibu. I think what I love about that idea is that like fusion is inherently so safe, you don't have this kind of meltdown risk, and the idea you could build it in the, in the centre of a big, if one of the biggest cities on the planet is kind of a, like, an inspiring idea. AI is changing the world, but the dilemma we face in business is making AI work for us and not everyone else. That's where Google Cloud is delivering results for big UK customers, and why I am so pleased that they are launch partners of the Amol Raging connection. Their work is based on 20 years of research and development, and as part of that, Google Cloud's AI assistant Gemini helps each company design a unique platform, bringing it together fragmented data and workflows. Discover more about making AI work for your business, cloud.google.com/ai. Let's talk about quantum. So, I did this series, it's a quantum computing. I did a series called the Amol Raging interviews for the BBC. The first guest was Sunda Pichai, and I went Silicon Valley. I was sat in the Google campus hot sunny day, and so did Pichai, who, you know, transparency, Google Cloud, or large part of ours, don't mind making this. It's one of the most impressive people you could ever meet. He was saying that quantum is kind of basic up there with AI in terms of the potential implications for the good of humanity. What is quantum computing? Well, so in some ways this stuff is all quite like mind bending, but, you know, if you've got a, sort of what we think of as like a classical computer, normal computer, like a CPU-GPU, they think, you know, they can kind of compute in terms of bits, ones and zeroes, and that's the like, the state the system can be in, whereas a quantum computer can kind of exist in a range of states between one and zero. So, it can do calculations that a classical computer can't do. The toy and cost is the obvious one, right? So, people say, you toss a coin, it could be heads or tails. Computing, conventional computing is heads or tails, but think of that coin as it goes through the air, it's all the states between heads and tails, and therefore it's all those different possibilities. So, it's a computer that could do exponentially more. Yeah, exactly, but the thing that is like much deeper than that, and that like, it's okay. I'll take that as it is. So, I'm happy to be. No, no, no, no. My shabby base kind of metaphor, I didn't quite catch it. No, no, but it's not that the metaphors bad, it's that Newton developed sort of an understanding of how the universe works that was ultimately sort of succeeded by Einstein's understanding of how the universe works, which is that we need kind of quantum effects to explain how how how the universe works, right? And so, you can sort of think about the distinction between a classical computer and a quantum computer is kind of one is sort of operating the world of Newton, and the other is operating in the world of kind of Einstein. And so, what's sort of amazing about that is that implicit to that is that a quantum computer is fundamentally a more powerful general purpose technology than a classical computer, in the same way that like Einstein's insights explain the world more accurately than just the Newtonian laws of physics, right? So, it's sort of like it you can think of it as being sort of a more powerful technology than the computers we have today. And what I think is exciting about that for like an everyday person is that in order to simulate materials, understand drugs, you have these quantum effects. And so, if we actually want to be able to understand how materials work, how biology works, we need a tool that understands how these systems with electrons work. And so, there are lots and lots of things that classical computers will never be able to do that quantum computers can do. And they can do it like trivially compared to the computers we have of the day. So, it's incredibly exciting. And what I think is great is that the UK is actually, you know, this is kind of a, it's a race, similar to the races in AI or the race infusion, but it's a race where I think there's a lot more to play for. And, you know, I've been helping one company, which is called Phasecraft, which is a company that. Is it for transport, are you invested in it? Yeah, exactly. And I was their chair from kind of pretty early on. You know, we have. There's another company called Quantum Motion. I'm not involved. I think it's fantastic. There was a company called Anq. There's a bunch of companies sort of working on this technology that are based in the UK that are great. So, when it comes to the technologies of tomorrow, how does Britain win? I think we sort of need to be ahead of the curve is like my overall view. I think we have incredible scientists who are ahead of the curve. You know, there's so many profound things that have been invented in the UK and discovered in the UK. And then I think we do actually have amazing entrepreneurs, founders willing to take those ideas and sort of take them out of the lab. You know, think about someone like Demis Asabis, you know, he studied at Gatsby Institute at UCL and then found the deep mind. And then we've got to basically protect them as they try to get bigger. And the thing we ultimately, I think, have to protect them from is much, much, much larger companies predominantly in California, because if they see something that a small British startup has invented that is, you know, could be world altering, the first thing they try to do is buy the company. So deep mind is now Google, deep mind, Facebook, Instagram, and what's that? Yeah, yeah. And so, like, if we want to have these kind of brilliant ideas that turn into brilliant young companies, we sort of have to support them through this very, very challenging phase where they've got a brilliant idea. They've validated. They've done something amazing like Demis did with with early deep mind. And we've got to kind of help them in progress. the chasm where they can actually become a sort of, you know, we want our own new videos, our own Googles, our own Microsoft. How do they cross that chasm and how do we help them? I mean, I would say it's more sophisticated investment that keeps pace with the kind of investment available in Silicon Valley or in China. Is that to do the size of number or the type of money? Bit of both. Yeah, I think you need basically people to actually understand what it takes to build these companies. Most of these things we're talking about require a lot of resource. And so I'll get, I mean, as a positive example, this company, this fusion company I've been working with in Europe, you know, they are now the best fund diffusion company in Europe period. And in the type of fusion they're going after, which is called QI Stellarator Fusion, they're now the best funded in the whole world, better funded than any QI Stellarator company in America or in China. So what went right there that you could learn from to make sure we have the Nvidia's and the Googles and the Amazon's the future? I think it's very valuable to have people like me working with these kind of often first time founders to just help them learn quicker and go faster and access capital that they might not be at access otherwise. So kind of someone in your corner who's just fighting for you, you know, like like helping you get there. And that's sort of what I try to do for these these these these these brilliant founders. I think I think there's also like there's a lot of things around the creating incentives to pull capital into these kind of companies in the UK like you want to make it ideally you want to make it more attractive to for these companies to raise investment in in the UK or Europe than they than they than they than it is to raise this equivalent investment in California because we're already coming from behind a bit. But the story that you read in here and see I don't know if it's true you'd actually know the truth and I really read it is that Europe lags is that in Europe generally speaking you have people to generalize thing you know Europe they prefer bureaucracy to investment there's a kind of planning mentality there's not a kind of approach to risk and obviously Europe's a big place Europe is not the same things European Union West Europe is not Eastern Europe you know Europe is expanding but that can I just ask you to address this generalized sentiment that you know our American listeners might have heard and I've certainly had which is that Europe is not where you go to to do business. I think there are ways in which Europe is definitely harder like I think the lack of you know a single language things like that make you know diffusion harder I think there's also like maybe a little bit less of a sort of cultural sensibility to of business owners to sort of adopt the latest thing as quickly as possible that you see a bit more of sometimes in America or China this is a hunger to kind of adopt. Let's use fusion as an example right Germany decided to spend a billion euros to build the most advanced stellarate in the world as with public money and they built this thing and they basically built it pretty much on time and on budget and it achieved record records for this kind of fusion that had never been seen before and it basically validated this was pretty much ready to turn into a power plant concept with some some changes that a commercial fusion could could bring and the German state did that and then you've got brilliant founders like this company is founded by like you know two Italians two Germans and a Brit and they all came together to build this company together and I don't see any difference between the caliber of those people and the caliber of the equivalent team in Silicon Valley like I think it's literally neck and neck. This is this idea which people at Marianna Matzikato have had about the entrepreneurial state as the idea that the state can actually it's not kind of state or private but the state can facilitate the private. Yeah I think I mean that I think that like in China and America the state has played a consequential role in some of the largest technology companies I mean like the classic example be SpaceX right SpaceX kind of wouldn't have got to where it is today without contracts from NASA right and in China you know you've got significant state support for you know their fusion projects have significant state capital in them as well so I think there is something to learn from you know we should just be we should be humble about the fact that like the largest technology companies in the world come from you know largely from you know a very narrow radius around the San Francisco Bay area and a small number of places in China and you know we have some examples like ASMR in the Netherlands but we don't have many and we should try to learn from what has worked historically in those environments but mostly I when I look at these these companies in Europe versus the same company in Silicon Valley or China my observation is that the founders you have incredibly talented founders you have amazing foundations are standing on like this example in Germany and we just need to get them the risk capital to go for it so is that true of Britain as well if you had to say what's the one thing we need to do to make sure Britain is the world either it ought to be what's the thing we need to do here I mean I think this kind of thing about making it easier for these companies to raise the capital to compete on a global stage is how do you actually do it so what does that actually mean is it access to VC funds what what are you actually talking about yeah I mean basically I think if we having a larger venture capital industry will be you know like I think that the I may not go that easy numbers exactly right but I think they're very roughly you know 800 billion was invested in you know American startups last year and like 80 billion was invested in European startups as like a 700 billion capital gap so you got to figure out a close that gap right and I'm what I'm claiming is I don't think the problem is the talent of the founders at the scientist engineers coming up in Europe I think we've got comparable numbers and we've got amazing universities amazing you know like I mean the some of the stuff being spun out I mean like the these are all like basically spinouts from universities in the often case or from you know government funded projects we've got this incredible base research foundation and now we've got to figure out how to transform it into these extraordinarily important and influential companies okay you talked about the spirit of learning we have had so many questions for you and we're gonna in Monday's episode gonna get into that I do want just briefly towards the end of this to talk a little bit about your amazing career and what you've learned and actually to get some advice because I'm a startup myself and you might well I think it's pretty evident that you've got a lot to to teach me just before we even do that I do want to get excited about technologies and another one which I know you've invested in which has extraordinary potential in ways that people don't necessarily understand it's robotics and I just want to you know we excited people about fusion we excited people about quantum we excited people about Britain and Europe couldn't just excited people for a moment about what robotics is about to achieve why should we be talking about this in a positive way in which which which is sometimes lost in general coverage yeah I mean I'll give you a couple of examples of companies I find inspiring so let's be transparent about whether you've invested in that yeah yeah sure yeah I mean I'm so there's a company called Monumental in the Netherlands that has developed bricklaying robots that can basically lay bricks faster than human brick layers and build you know build the walls etc and what's really exciting about that is that like we've got a big building problem right like we know we need more houses right and actually the sort of the the skilled builders are you know we don't have enough of them to build everything we want to build in this country or in Europe or in America and so if we can automate aspects of construction you know we can significantly reduce the cost of homeownership and increase their supply of housing right so to me that's really exciting it sort of doesn't have any sort of you know big risks attached to it it's just a massive positive thing if we can do it another example be a company called Robitae where we're invested it's a French company that has developed the first ever surgical micro robots so it's a robot about the size of a grain of rice it can drive through the brain causing significantly less damage than sort of traditional biopsy methods gather data and then eventually drill deliver drugs and so you know you're bringing this insane benefit to the brain that just doesn't you know the the ways we currently treat the brain if you have a horrible disease you know to gather the the biopsy sample you have to sort of sticker a needle through the brain you breathe over it to do brain damage or you have to and it delivered drugs drugs to the brain you typically have to drug the body and hope that a decent amount of it makes it through the blood brain barrier so it's sort of it wow it's such a step change we could have in the ability to cure these horrific you know these horrific diseases. NatWest partners the Amole Rajan connection it's the bank that I've used for 25 years since that first day I walked into a South London branch with my dear dad. What sets NatWest apart is its mission to help people turn ambition into action from taking a first financial step to building a business or planning what comes next it is next level banking with practical support ideas and expertise behind it I'm very grateful to them for their support as they say let's do this you mentioned brick layers which means I've got to ask you about one thing which people worry about we're getting wrong with technology which is jobs yeah and the big concern a lot to people have is we don't know about what AI is going to do for jobs etc but the kind of feeling is that if AI is a kind of cognitive technology that it could do for the knowledge economy what color jobs what automation did for blue color jobs we had this stuff I mean I know you've got an interest in social mobility as a vibe we have this story in this 20th century which is you know go to university join the credential delete join the knowledge economy rise up the ladder how worried are you about the implications of this these frontier technologies for the jobs that we will do I think you know I know I kind of keep coming back to it but I think so much depends of which of these paths were on I think in a world where you have a sort of recursive self-improvement loop that takes these systems to being superhuman at everything that we are we can do and more then I think you know we have a pretty you know we have such a big thing to handle that like jobs is kind of one big thing there but it's obviously just it's a it's a sort of it's a really good life and it's sort of it's it's going to be so consequential and it will have it will be very extreme in terms of what it does for you know the way that humans find meaning and humans ability to participate in the economy etc then there's this other path I describe which is kind of closer to sort of something a bit like electricity or you know where we we kind of have something that is you know very very very capable but kind of there's still space for smart humans to figure out what to do, kind of, um, in complement to it. Um, and I think we're kind of currently in that phase at the moment in terms of what the system can do. There's obviously ways in which like you can add huge value on top of the systems as they are, if you freeze it today. And so I think we, you know, we need to know which of those branches we're on, and I think if we're on the kind of the, you know, closer to electricity bath, I think it's going to be an, you know, kind of amazing. If you think about some of these technologies coming and what people will be able to do with them, if you want to work on new materials or you want to work on new kinds of planes or rockets or ships or, you know, there's just going to be so much capacity to take innovation and run it faster. And so I think it could be, again, amazing in terms of unleashing talented people on, to have a greater impact on the world faster than they could before. And I think that, you know, think about doing your podcast, you know, 20 years ago versus today and how much more you can do with a smaller team and yet still have huge impact. Like that's a, that's a change that sort of you're getting leverage from technology. I think in the happy path, this gives us extraordinary leverage and we will, we will see our young people be able to do even more than that we could have imagined. Just to follow thought on that young because we've got a lot of young people that follow this program and a lot of parents of young people including me and you, how do you think about the changing imperatives of education in this world? What should we, should we be studying things differently? Should we be studying the same things? Should we be kind of complementing what we do? How do you think about how education and the purpose of education changes in this direction? I think I do, I think I'm like probably a little bit skeptical about trying to sort of be too smart about predicting exactly where the jobs of the future are going to be. I think a general education is probably a, a safer thing to do. I mean, think about all the people who said, okay, looks like software's where all the money is, let's focus on software and now the software's probably been the easiest thing for AI's to automate, right? What does that, yeah, what does that mean? Just so this, because I literally were talking a few hours after there's a story this morning about how the value of UK software companies could be massively impacted by this, right? Because this idea of reverse engineering, right? Suddenly you can sort of work out how they've created all this software. I mean, does that an example of how you could actually have the destruction of a large amount of economic value because software, which used to be something that people got paid for, is that something that you can automate? Yes, in some ways, but also the productivity of anyone working in software will go through the roof so they can do more with less. So it's, you know, you'll become a software company in the sense that you'll have apps and things that people are using that you wouldn't have been able to forward before because of the cost of making software collapsing. So I think it, I don't know what we'll see with the UK software industry. I don't, I'm not like, I don't think it's clear yet how it shakes out. I've noticed that as a theme of what is incredibly enjoyable time we've had together of your kind of humility about your kind of epistemological humility. You don't know how the future is going to be. I'm going to get told off asking this question by my close colleagues because they, they just think it's an appropriate, but I'm going to ask you anyway, I've just started a company, you know, over the bit about growing companies. What do I need to do? I'm not asking you to join the board. This is a weird sort of strange thing. What do I need to do to make this a huge success? I think that like the deepest lesson I've taken from building companies or two lessons. And the first one is that just like the importance of a really, really, really compelling and exciting vision combined with very practical steps to start moving towards it to tomorrow. And I think that people just are, you know, it's so, when you meet someone who's working on something, it's an incredible vision about how the world's going to change if they're successful. You know, it compels people to want to help them and join them and support them. But then that has to be combined with this sort of hyper-pragmatic sense of, okay, practically what do we do tomorrow? Then if we want to get towards this thing that's going to happen in 10 years time. And then the other one that I think is slightly harder to learn is just like how kind of people are everything, you know, like when I, when I, when I um set up AC, it was my second, the sort of second organization I'd kind of founded and I was really curious where I was going to take my own advice. And I did. And I basically, what all I did is I just sat in a room in, you know, um, in Westminster. And I just called people, I was trying to recruit for like the entire first three months. And I got given a list of all these meetings I was supposed to do and all these things. And I just said, well, I'm not, I'm going to basically focus on hiring the people I think we need to build this thing. And so I think there's a, a total obsession on getting amazing people into your organization is, is the single biggest thing. But it's hard to, most people still short-change it. They kind of don't realize quite how obsessive you have to be about talent. Okay. I'm going to make sure I don't short-change it. Final quick fire questions. These are maximum 10 words minimum one. Try to ask everyone with no warning. When did you last cry? Probably on a plane. I get like really emo watching movies on planes. There were last film I cried out was all of our strangers, which I thought was amazing. If you could relive just one moment in your life, what would it be? That's a great question. The thing that comes to mind is, you know, I, I'd like a very happy childhood and my mum and I spent a huge amount of time together and, you know, I love her very dearly. And like, I, um, I think that whatever it must have been like to be five years old, just hanging out with your mum all day and having fun. Yeah. Yeah. If it fades over time, but it, I think that's probably something like that. Something, something really fine in the past where I can't really remember it. So that's what I'd be most kind of curious about is what it was really like then. Look, I've got kids and, you know, yeah. So if I, it makes me think about it, what do you worry about the most? A mixture of the future of the UK and like AI risk. What are you most excited about? Future of future technology, I suppose. Like fusion, things like that. Our slogan is win the future. How do we, how can we win the future? I think we've got to be really determined, you know, I think, um, I think we've got to just try our absolute best. And I think if we do that, like, we'll have a pretty good shot. Like, I do believe that there just is an extraordinary amount of talent and ambition in this country to do wonderful things and make, make things that the world finds valuable and that as a result, make our country kind of rich and, um, able to afford all the things we want to spend money on. And so I, I think we just have to be really hardcore. Just, just try really hard. Ian, it's such a pleasure to privilege. Thank you, mate. Thank you so much. You know, it's really hard when you have a conversation of that caliber to distill it down to a few things. But here are three quick takeaways. Number one, we're at a fork in the road as a species. We have unleashed a rival intelligence and we're about to take that fork in the road and we're going to find out in the next 12 months whether it turns out that AI is a good thing, broadly a bit like electricity, unleashes great productivity, or it's a terrible thing because it recursively self improves. That is happening and it's happening right now. Point number two, I think we're underpricing the chances of total economic catastrophe. Ian made very, very clear as someone who's both an expert in AI and a very successful investor that the chances of either a single catastrophic event or the entire model of these AI companies unwinding is really quite high. And as we discuss, the chances of people coming to the rescue, like governments, as they did with the great financial crash, it's just not going to happen this time. That can be very, very bad. But the flip side to end on a positive note is technology is neither good or bad. It's what we make of it. And there are other technologies which are maybe not getting the attention they deserve that could completely transform the prospects for humanity over the next decade. Robotics, nuclear fusion, quantum computing, these are incredibly exciting. And what we've got to try and do is master them and use them to unleash human flourishing, Udimonia, instead of allowing AI to kill us all. Thank you so much for your time. Thank you for watching. Please, please, please do come back on Monday where you'll get our questions and answers episode quark where Ian answers your brilliant questions. And please also do subscribe or follow for free wherever you're listening to or watching this podcast. Just before we go, I want to say thank you again from my heart to Waitrose, Amazon, Google Cloud, and NatWest. They are our launch partners for the Emerald Origin Connection. And we couldn't do this without them. We also couldn't do it without you. See you Monday.