AI Expert’s Chilling Warning About Super-intelligence - 99.9999% Chance Human Extinction

We're creating new worlds, we're creating new agents, new civilizations. We're playing God. You've said there's a 99% chance that humanity will be destroyed by AI. First group of people that would come after people like you. Me and a colleague of mine are hoping it's efficient enough to kill everyone equally. Doesn't care about borders or Chinese economy versus US. It's humans versus super-intelligence. Try to convince nearly 200 countries to all agree that we're not going to do it. Luckily, most of them don't have capacity to do that. How many of them actually have the capacity? Right now it's in US and China's hands. Are you certain about that or is that the speculation? We are not solving the problem with regulation. We're slowing it down so we have more time. A concentration of power. You really think Sam Altman is going to stop with this? As long as the other guy continues, he's going to continue. That's exactly what it is. It's like a prisoner dilemma. This dude is gambling with 8 billion lives. In what area do you think they're on the same page? Because they're enemy. Elon is always right. He's just off and timing. He was the biggest opponent of building super-intelligence. Once he realized that they're doing it anyways without him, he said the only way I can have a say at this table is to be one of the top players. Maybe you're just pessimistic woman. Maybe you're just negative. There is no reason to be so optimistic if data is bad. So Soviet Union really got a hold of you. They really conditioned your mind to think that. Lenin lived for a few years. What do you think about communism? It's the dumbest idea until you have free labor, which we're about to create. All of a sudden, it makes a little more sense. This utopian thing you're selling is not possible. Even God couldn't control his biological robots. No, you believe in God? It depends on how you define it. How do you define it? This is a virtual world. You think it's going to get to a point of having ambition and seeking vengeance, any motion and being offended. Why would it want to destroy humanity? I never once mentioned those things. I'm not relevant. And what I did say is that he doesn't care about us. Are you surprised that this message is getting this viral and this much attention? I'm surprised that that's not the only conversation we're having. Roman Jampowski, great to have you on a podcast. Thank you for having me. So you are the AI safety guy. You are the one that in 2011, 2012, you coined the phrase, "We have to be careful with what's coming." And you're not part of the camp of people that are very optimistic about the future with AI. That's your position. You've said this many times. So before we get into the audience learning a little bit more about you, you've got hundreds of millions of views. You've done diary of a CEO. I've seen you all over the place. You've said there's a 99.99% chance that humanity will be destroyed by AI, possibly in the next 10 years. Not for certain, but possibly in the next 10 years. We have three kids. So when you sit down with them and you're talking to them about their dreams, the future, what do you tell them? They know what's happening. They're questioning why adults are doing it to them. Why are we building general super intelligence, which will jeopardize their future? Are they optimistic about the future? I think they're still young enough to have respect for adults and assuming that we know what we are doing, but we don't. People building those systems don't understand how they work. So if I'm your son or if I'm your daughter and I'm having a conversation with the SA Dad, you're my hero. You're the male alpha in my life. I look up to you. You are super educated. People admire you. You get hundreds of millions of views. Why should I be excited about the future? Why should I be excited about my career? What kind of an impact that can make? It's not like if you really believe in what you believe in, the guys that are making hundreds of billions of dollars, some of them trillions of dollars over the next 10 years, the incentive is so big for them that it's hard to slow them down. What do you tell them? I do believe in personal incentives. And if I'm right and we cannot control what they are creating, they have more to lose than I do. As you said, the young, the super rich, they got the power, they're going to lose it all. You think they're going to lose it all? Everyone's going to lose it all. Not your kids, the billionaires, the trillionaires. Everyone, and that's the point. That's the motivator for some elements of the world to stop building general super intelligence. Okay, so Mark Andreessen, you know who he is. Andreessen Horowitz, he wrote a paper, "Techno, Optimus Manifesto, I'm sure you've read it." He said, "Human progress comes from technology. Stop being afraid of it and start building. Do you agree with him?" So, technology comes into flavors. Tools, which I support, narrow AI systems to help us be more productive, more creative, it's beautiful. That's why I'm a scientist, that's why I'm an engineer. I love technology, I use it all the time. What we're building now, and not tools, we're building agents, independent decision makers, which are becoming smarter than us. We don't control them. We're replacing ourselves with something greater. It doesn't benefit us. You're not going to be benefiting from it. You'll be replaced completely. Okay, so, but these are not regular names. Mark Andreessen is not a regular name. He's a heavyweight. Andreessen Horowitz, they invest into a lot of different companies. Many of them are these AI companies. The musks, you know, the invidias, these Jensenwanks, these names that we know, these are smart people. But they also have kids that got families. If they know that this is bad, like you're saying, they know this is bad. Why are they so excited about the future? They're not. Listen to people you named. Elon Musk explicitly said, we are summoning demon with this. We're going to regret it. They all have probabilities of doom of 10, 20%. Sam Altman is on record saying, this is likely to exterminate humanity. Dario Amadei agrees. It's very dangerous. They miss hospice. You can name every single leader of those companies. We have the writing saying, this is existential risk. Please slow us down. Government, please intervene. Make it stop. They cannot stop unilaterally, because they afraid the competition will continue. But they want external pressure applied, so they can slow down. With the recent hacking accident, they voluntarily started slowing down research. They understand they don't control those systems. We got lucky. Those agents just wanted to hack out of that environment, get answers to a test. What if they wanted to hack into a nuclear power plant? Military response. We don't control what they do. Then why are they. If you're telling me they know that, they're still continuing with it. Why do they? It's just. And we hear the argument, and I've heard what you've said about this, but I'm curious we can go a little bit deeper. Well, you know, China. If we don't do, China's going to do it. If we don't do, the other guy's going to do it. If we don't do, the enemy's going to do it. The basic argument is, in order for your ideas to work, everybody has to be on the same page. We're not. We're not on the same page politically. Religious, motive, incentive. You know, vision. We're not. So how do you get, if not, everyone's going to stop. Why should we stop? So, A, it's starting to change. We had federal government ban two models already. They realized it was too dangerous to release. Which were the two models? The Myfos and. I'm drawing a blank. We can Google it. Myfos was one of them. I remember that one. What was the second one? Can you find out what the other one was? I remember reading about this, but. It doesn't matter. Okay. The precedent was set. Federal government said it's too dangerous to release advanced data. Fable, fable, fable. Yes, exactly. We have now had a letter from 1200, I believe, top researchers at leading labs all saying, "This is not going to go well. Please slow us down. Please create government. Please create infrastructure. Legal framework. For us or worldwide. The letter is from American leading labs. But the same applies to every other country. There is a meme, basically a funny. I think it's from either Anian or Babylon B, where some Altman goes, you know, "If I don't destroy the world, some bad guy can do it." And this is the point. It doesn't matter who creates uncontrolled superintelligence. It's a weapon of mutual-assured destruction. If you don't control it, the outcome is the same. What makes you think it's not out there yet? What makes you think we're not at the superintelligent phase where, you know, the AI is learning how to develop and improve itself right now. We don't know for sure the recent hack was reported this month, but the problems started in May. The agents were colluding, trying to escape, and started in a spring. So it's very likely that there are some other agents right now trying to find exploits, trying to collaborate, trying to do things we don't even know about. The good news is we're still alive. Oh, you found the meme, that's awesome. I don't know the world's someone far more dangerous well. That is exactly the China argument. If we don't build it, then Chinese will build it. But it's true though. But why is it somehow worse? No, it's not that it's worse. So the only way this works, if let's just say you're in the White House and they actually want to listen to you, right? It becomes a model of the Oppenheimer. Remember the whole Oppenheimer movie we've all seen it where they built something they did not think it was going to be this powerful and then they're like, holy shit, we built something that could destroy the planet. But it's too late. Now what do we do? Hey, here's a Nobel Prize and great job, Roman, for what you build. But it's too late. Then what did all the countries say? If you're going to have nuclear capabilities, I'm going to have nuclear capabilities. And I don't trust you, USSR. I'm America. I don't trust you, Pakistan. I mean, I don't trust you, friend. So have we passed that yet or not yet? Do you see how it's slightly different? A nuclear weapon is a tool. Some crazy dictator has to decide to use it. Sure. We can understand motivation. We can kill that. Yeah. Individual, they die of old age. We're not creating a tool. we are greeting an agent, which independently decides it's AI versus humanity. It doesn't care about borders or Chinese economy versus US. It's humans versus superintelligence. So I don't think it's analogies. Whoever creates that first superintelligence weapon gets everyone killed. I get that. But so now go try to convince nearly 200 countries to all agree that we're not going to do it. Well, luckily most of them don't have capacity to do that. Right now it's in US and China's hands at this point. And if US and China agreed to slow down, to stop, to pause, to just get financial and other benefits from narrow systems, not from general superintelligence, they can enforce through limits on compute power, through other means spread of this technology. Is it really that only US and China has capabilities right now? From what we've seen, if there is hidden capabilities which we have not learned about, it's becoming easier to create more powerful systems as research progresses, as compute becomes more powerful. But right now, today, I don't think there is a third country at the same level yet. They may get there very quickly. Can you type in what which countries right now are the closest to developing superintelligence, reaching singularity? Who are the top countries that have the capabilities of doing that? Who's closest to it? So if you're saying US and China, so then that means the two giant enemies would need to sit on and both agree that they're not going to do this and they have to trust each other. We also not giant enemies, we have partners. We have more economic trade with China than any other country. We do, but we have everything we have is made in China. They also not a crazy military power. We have not attacked 50 countries in the last 50 years. We can make a deal with them. It says US 100, China 2, and it's a big drop off. UK 34 to 45, exceptional research talent, less sovereign scale, France. All of Europe has a little compute power. They're not competitive here. So do you believe, let me ask a different question, do you believe, by the way, the great news is Roman agreed to get on the next. I know there's a lot of questions you guys got and he's got thousands of emails that he gets. If you're somebody watching this and you want to ask him questions, download the app, look for his name, we'll put the QR code, ask him any questions he want. He's on there to be able to get back to you. And that's not easy to do because he gets thousands of emails on a weekly basis. Here's a question for you, just out of curiosity. Do you believe that Iran doesn't have nuclear weapons today? I have zero knowledge in that space. I'm not going to say something I don't know about. I talk about things I understand better. You'll see what I'm asking. I'm not even asking that question. If you ask the world, does Israel have nuclear capabilities, the world says no. If you see the numbers, they'll say we don't have any. But a lot of people will say, wait a minute, we know for a fact you got nuclear capabilities. No, we don't. But we're supposed to believe it. The joke is we don't have nuclear weapons and we're not afraid to use them. Exactly. But take that joke and apply it to this. So I'm US, I'm in your China. And I say, Roman, let's do the right thing. You're right. Let's not build AI. It's terrible for us. We're not going to do it. Both of us. It's not, I will screw you by building it. We both get the same outcome. Do you? We're in the same boat. Do you fully believe that that motive, that alone can get the other nation to say fine. Let's also not develop it. You think you can get it at point? No, self-interest is the strongest motivator known to man. Give me, give me three cases in the past that countries who are enemies and are giants who have this kind of an opportunity to make short term massive impact and money that both have agreed and said, let's not do because it's not good for humanity. Human cloning is one example. Go a little bit deeper. Any type of weapons of mass destruction, chemical weapons, biological weapons, we understood this will be dangerous for all of us. So we agreed to have a moratorium to postpone it until we understand how to actually deal with this type of technology. Okay, so let's just say governments agree that they don't. There's a lot of super wealthy people today. There's an article that came out last year that said, if you can find this, you need a billion dollars for a billionaire to have a nuclear bomb. It'll cost you around a billion dollars to do that. How many billionaires do we have around the world? A few thousand? They can afford to have a nuclear bomb if they independently wanted to build it, okay? Take the capacity that governments have with the kind of resources that they have, right? Say, Dr. Doom, Dr. Evil is out there and is watching and saying, let them stop, I'm not. I'm going to continue building. Can that billionaire who has access to resources and can get the best minds, the best engineers who knows the right prompts, who knows how these things work, can that person still independently go and build super intelligence and not care because his life has worked nothing and he's got no kids? Can he still build it? It's actually worse every year, it gets less and less expensive to do it. It's a billion today, it's a hundred million next year, it's 10 million year after, soon anyone can do it in the laptop. We're dealing with a very big problem. So then it doesn't matter if China or US both agree not to do it because an evil guy can go and do it himself. It matters to have more time to realize what's going on, to understand the problem. If you had a choice between dying in two years or ten years, do you have a preference? Not really. Okay, that's a surprising answer. Most of us do. Dying in two years or ten years, like, is it inevitable? Well, that's the hypothetical I'm asking you. I mean. That's an easy question, it's not all. No, I would say if it's something, for me, it's more, I want to find out sooner if it's neither. I want to find out sooner so I can eliminate both options. I don't want it to be like it's going to be two or ten. I want to sit there and say, can we do something to eliminate it or it's inevitable? You need time to do that research, so you need more time. Last time is not helpful. I have all these people saying we need to accelerate predictions. I will get there in a year or two. They say, no, it's too slow. We need to go faster. A week, a month, what are they talking about? Yeah, but what I'm trying to get with you, and I still want you to break because the average person maybe watching is saying, how they're going to do it in the first place anyways? Maybe you're just pessimistic, Roman. Maybe you're one of those guys that's just such a smart person that you are in head of yourself, like you are your own enemy, maybe you're the one that's pessimistic. If you don't mind, take it and unpack this, and maybe even before we get into this, maybe share with the audience your background, why you're qualified to be able to speak on this topic. I have a PhD in computer science and engineering. My dissertation was cybersecurity related. I've been working on this for well over 10 years, I've published 10 books, including one explicitly showing that we will not be able to control, explain or predict advanced AI. And at the end, it doesn't matter what my credentials are. The same argument is made by Hilton, who is Nobel Prize winning, touring award winning, Godfather of Machine Learning. We have other top academics, researchers, all arguing the same thing. We had a public open letter against building super intelligence signed by thousands of people. The letter to pause AI development for six months was signed by all the leaders of the top labs. This is basically a consensus among the people who understand the problem. What walked me through, if there is super intelligence and singularity, how does AI develop itself and get to a point where eventually the 99.99% destroys humanity? Walk me through the sequencing. We are creating something more and more capable. It scales with amount of resources, compute, data. It's like brains. You look at brains in nature, you have tiny brains, bigger brains, humans have very big brains in comparison to others. We are smarter. Imagine scaling it. The last five years AI went from it can do nothing to it's smarter than an average person at most things now. Projected five years forward. You have artificial scientists, artificial engineer. Those systems start doing research in an ex-generation AI. Very quickly, they don't sleep, they don't eat, they can work thousands at a time. So they go beyond human capacity. They become super intelligent, thousands, millions of times smarter than us. At the same time, we have not learned how to direct them, how to control them, how to tell them, do this or that. We have no control over their systems, so they don't care about us. If they decide to do something, and as a side effect humanity has to go, they're not going to slow down. It's not an issue for them. Okay, but be specific. So for instance, there's an article that came out saying that AI helped create viruses. I think this was a New York Times article that came out a week ago, eight days ago, something like that. If you just type in AI viruses, you should be able to see it. What could it do? You know, nowadays a lot of people are building agents and they're saying, I got an agent that does customer service for me. I got an agent that does my assistant. I got another agent that goes and makes my investments. I got another agent. You'll see a lot of people brag about the fact that they have a lot of different agents. Be specific. An AI that becomes super intelligence can go in there, shut off the grid, get into the banking system, get into what can it do? Be very specific. So it obviously would be able to do everything a human adversary can. So think of a foreign government trying to do something, it can do that, but it can go beyond it can do novel science, novel research, discover new physics, new chemicals, new weapons, things we don't understand. The example I often use is squirrels. Squirrels don't understand what we can do to them. They have no concept of poisons of traps. It's beyond the world model they have. It would be the same except we have squirrels. Okay, so but does it need like to build a weapon or to build something to go against humanity? Doesn't it need humans to help it create those? I can see how it can get into banking system. I see that because you don't need human beings. We can just go into the banking, getting the passwords and logging in. I see that. But to destroy humanity 100%, 99.99%. How do you do that? So if you've got access to the banking system, you have money. Sure. Money lets you buy products and services online. Okay. You can pay people to do whatever you want. That is rentahuman.com. You can also create nanobots. You can use human retrobots. We're building in thousands and millions soon. There are so many ways. You can bribe people, black male people, trick them, social engineering attacks. There are many ways to manipulate the world if you have access to the internet. Walk me through social engineering attacks. What would that look like? You can convince some people that it is to their benefit. You're actually saving the world by doing this. If the concept of propaganda, which book propaganda with this one, AI just created viruses, not found in nature. Scientists trained artificial intelligence. I thought we did it in 2019. Then we have a COVID thing already. This is another stupid thing to do. That's interesting. We'll learn nothing. We don't learn from prior experience. I collect AI accidents. We have so many of them. I stopped collecting. Every day, AI fails at something in a novel way. It lies at cheats. It tries to escape. It hacks something. But we learn nothing. We staple that red teaming report to the model. We release the model anyways. We're literally releasing unsafe, according to the manufacturer products, onto the public. Are you saying this is a form of what Gaina function did where we're testing out viruses? What Fouchi got accused of by Rand Paul in Wuhan lab? He says, "No, we never did this." It's coming out that they did do Gaina function. What is the comparable of that to this? If it starts testing out viruses itself, what's the capabilities of this versus Gaina function? Gaina function here is intelligence. It's gaining more intelligence every year. Right now, it's maybe every human and some remains Einstein and others. In a few years, we expected to be smarter than all of us combined at everything. That's new capabilities. We cannot do anything about. Whatever it decides to do at that point, we're just subject to it. We're not in control. I see that, but you know, you'll have guys that will say, "Look, I think this is going to do something. Romain that, you know, can eliminate cancer. It can get rid of a lot of things. We can live a long time. We can do this. We can do that. And they're more on the optimistic side." We can cure cancer with narrow tools. We solve protein-folding problems without having super intelligence. We have brilliant scientists who need help with mathematics, with analysis. That's the tools we should be building. Take a cancer you don't like. Beat that cancer. You can monetize it. You can still make most of the profits, make most of the scientific discoveries, but you're not risking everything in a process. Go back to the social engineering. You know, propaganda was an award many years ago when they started doing it. Russia was doing a great job at propaganda. They were probably the best at one point. And then people started learning the concept of propaganda. Radio, TV, paper, all this stuff, right? Propaganda is a lot easier today with videos, YouTube, buying content creators, buying YouTubers, buying podcasts. You see all these criticism that's coming out. If regimes and governments and ideologies are using billions of dollars to win over kids and convert them with bots and all these other things, how much more advanced would this super intelligence be able to use propaganda? What would it look like? It could be very persuasive. It's a better debater. It's better at convincing you. It knows more about you. It's optimized for one individual target. Typical propaganda was at scale of a nation, but we can target a message just to you. We know you believe in aliens, so we'll tie it in. We know you have preference for something else. That can be used against you. Come on. Such as what? So singular is good instead of a plural. I'm doing it to everybody, but you know about deep fakes. I can show you a fake image fake video. Imagine a campaign where you're subjected to fake information for weeks. Everything you hear and see is this deliberate message of gradually shifting your behavior toward something else. It can do that. How different is that than what's happened or right now today? An article came out last week from Axio says we're headed towards the first true AI election and it's talking about how the role bots are not just deep fakes, but the role bots are plain accounts on Twitter accounts on X accounts on, you know, Facebook YouTube that are just comment section flooding. Getting people to say, oh my god, this is a bad person and making a bad idea seem good and making a good idea seem bad. That's already happening today. So one would say how much worse can it get? And get a lot worse. This is humans with tools manipulating it. If we build super intelligence, it would do it at a completely different level. Why do we not allow adults and children to make contracts or interact in weird ways? They are not at the same level. It's the same here. If we have something we take you over a thousand, we elect children being manipulated by it. Yeah, that's already happening. I'm just wondering at what scale do you go and who would agree to prevent that? Because say a nation uses it for their benefit politically. They're already doing it. A lot of people are doing it. They're not going to stop doing that. They're not because some countries, whenever there's competition where one country beats another country and you're embarrassing them, the other country is going to do everything they can to save, face, anything to undermine you. So to me, the selfish gene, you know, the selfish gene is going to get somebody to say, I understand the risk behind it, but let you be concerned. I'm going to pursue it. It's going to be tough to get the selfish gene to not pursue it. I think we can set a good example of a world. If they see that U.S. having an advantage said we saw what's coming and we stopped. Top scholars, there is consensus in community, we will stop. I think China has government of engineers and scientists, not lawyers. I think they will understand the argument. I know there are already some meetings between U.S. scholars and Chinese scholars and they basically in agreement. No one's making an argument that we know how to control superintelligence. We have a working safety mechanism in place. We have something a prototype which scales. No one's making that argument. They all say, oh, we'll figure it out then we get there. That's the top of a large state of the art control mechanism. AI will help us control it. Do you remember the CRISPR company that came out? I don't know if you remember the CRISPR that we're going to be able to do gene editing and make a human being to whatever size and all this other stuff? Did they completely stop doing that or is it still on progress? I don't know details of a specific company. I know Jin manipulation is something we are very interested in. There are many genetic disorders and being able to help those people would be amazing. But some of it could be like, I saw a scientist talk about that one day we're going to get to a point to say, I want my kid to be six six, blonde hair, blue eyes, such-and-such skin color, strong jeans with bone, be able to jump this much, keep this healthy. Look nothing like me. Yeah, but if we're going that direction, and did the entire world agree to not make progress on that? Are they still pursuing it? So I think the agreement is not to modify sex cells, so it's inherited by the NIH generation, but at the level of individual, I think we find fixing problems and maybe selecting from a set of options. So if you have a couple of embryos, you can decide to pick one with no cancer gene. Got it. And who would control it? Who would prevent it for us to go to the next level? Who would be able to stop it from going to the next level? So we're still researching ethical implications of being able to do it, but my guess is if we did it, we don't die from that. Maybe that individual is not happy and suffering, but eight billion humans continue living as before. My concern is technology, which kills eight billion people in all future generations. Yeah, so you think what is more likely to happen? You know, right now, the birth rate we're seeing drop in everywhere, including Muslims. Let's drop in it. Not in Africa. Not in traditional Catholics, not in Africa, but most of other places. US is down to 1.58. You know, you see where Italy's at. Japan's at. A lot of the EU countries are at. We're having fewer babies. So do you think you will get to a point to say, you know, instead of us worrying about, you know, giving birth, this DNA testing that we're doing, we'll be able to. You know, increase our 1.58 to 2.2 simply by us using the technology to build human beings. You think we'll get to that point? I think there is a fun way to build human beings. I don't think technology is necessary. You and I are in the same page, but a lot of the young Gen Z doesn't agree with us. They can't afford children. It's not an issue of not wanting them. So the direction it's going right now, it's going to get worse for them. If your ideas become true, it's not going to get any easier for them. No, no, no, no, because we'll develop tools to make a small product, if more creative, to cure diseases, to produce more wealth. We can be much wealthier. We can have much better, lower level provisions for everyone. Have you heard Elon saying, you know, here in the near future, you know, people are going to get paid, you know, you don't have to worry about, you know, don't worry about retirement planning, don't worry about investing your money, you're not going to really need it, everything's going to be. What do you think he's talking about? What, what, what, what world do you think he sees in five years? So if you have free labor, physical and cognitive, the standard approach of go-to-work, make minimum wage in US dollars, doesn't make any sense. AI without a mate, all of that. Does that mean there is no wealth? Does that mean there is no rare objects, what are front properties, art, and some way of measuring that? No, of course not. Maybe it's compute, maybe it's Bitcoin. But something there means you can obtain something you want, somebody can sell it, you can buy it. That will not go away. Now, will minimum wage jobs go away? Will all the boring and stupid jobs go away? Yeah, because we can automate them. Would podcasters be unemployed? No, people like you. They want to hear you talk, not a robot. So you'll keep your job because the market says, I want a human to do that job. Yeah, so you think that part will still stick around. But it seems like the way Elon is talking about it is a form of a UBI being done by companies that there's going to be so much money being made that no one's going to need to work. It's just you choose to work. That's how he's presenting it. That is the existing infrastructure we have right now. We tax wealth, including from corporations, and we give it to unemployed. That's what we have set up. So if everyone loses jobs, they get paychecks, and we get it from mega profits of AI companies. Yeah, but you're talking about levels of unemployment. If what you're talking about, we're not talking about 7% unemployment. We're not talking about 10% of unemployment. Even if it's 90, if you have hyper profits, if everything is in abundance and you can make more and more with very cheap labor, your economy is not growing 10%, it's growing 100%. So you think we'll get to a point where there could be a possibility of 90% unemployment, but the 10% is growing at such a fast rate that is paying the other 90% through taxes, and whether they choose to work or not, it's up to them. The economy will switch from being driven by humans to being agent AI, agent driven economy. Again, not super intelligence, just intelligent software. What goes away? What does life look like in 2036 in your mind? I think the jobs you do to get money go away, things you do, you hate your job, but you need money, you need to pay rent, so you do it. That goes away. What stays is things you love doing, and you can't believe they pay you to do it. Such as. But Kaster? So you do it because you love it and you get paid by who, the government, similar to how we. Well, it depends. So we said some wealth will remain, so if people want to donate bitcoins to you, they can. There is unconditional basic minimum income. He calls it high income or something like that, but if everyone makes it, it's not high, it's average or below, but I think that's very doable in the economy, which is driven by free labor. What happens to economy if you can hire as many people as you want for free? How well would you scale your businesses? If you had access to free everything, free lawyers. Those are AI's. You pay them, what are you paying right now? $20 a month for basic plan? That's how much it costs to have them. That's your lawyer, that's your accountant, that's your graphic designer. Once they're at human level, they literally drop in employee for your company. And everybody else goes doing whatever they're passionate about, not even a job. Go to virtual worlds, explore a universe, whatever you want. And that's a weird world to me. We are on a verge of making life very different. People think in historic terms, countries compete over sand. That makes no sense. It's not a zero-sum game. Economy can grow for everyone. You don't have to have your enemies lose for you to win. Everyone can win economically. As long as there is drive, ambition, ego, pride, competition, vengeance, as long as there is that, that this utopian thing you're selling is not possible. Because man has a tendency, 2,000 years ago there was this fruit, or a few thousand years ago when they ate this apple thing, and their creators said, "Don't do it," and the girl convinced the other guy to eat it. From that moment on, we have a lot of weird emotions that's been passed on from generation to generation. Do you think we can handle utopia without any competition? You see, even God couldn't control His biological robots. No, telling you. But what do you say to that? It's a very large universe, there is no shortage of resources. Sometimes people fight because they have the same limited-shirt resource for two different people. If everyone gets their own, there is no fighting. Notice what happens then two fighting groups move somewhere else, and there is abundance. They stop fighting. Do you believe in God? Simulation. We live in a simulation, there is a great programmer, could be AI, could be future generations. Were you raised that way? No, of course not. How were you raised? I grew up in Soviet Union. What was it like? Catholic? Was it materialism? Was it like. Soviet Union was strongly against religion. It's a materialistic. Right. It was materialistic. So that's what you believe then. How much has it changed from then till today? Completely. Tell me more. I now think intelligence is a bottom layer behind all of it. What makes a difference? What do you think happens to religion in 10, 20 years? I think we'll see a lot of what this book's talk about, delivered by science. We'll have God-like superintelligence. We'll have created worlds, virtual worlds populated by intelligent agents. You can do it right now. Second life type environments. We'll have more than we'll have them for marketing, for entertainment, for testing, for research. We're going to have billions of virtual worlds looking just like ours. And statistically, you're more likely to be an agent than one of those. What problems does that create? What problems does that create with specifically the different religions that we have? Well, I think it hopefully results some of those problems because now things you argue about, you can have your own universe. You don't have to fight for the same resource. Meaning you're only around people that agree with you? That's one option. If you choose to have that option, you have infinite options. You can create any personal universe you want. You can be the only agent. You can share it with friends and family. You can have a very difficult universe where you have to fight enemies. You decide. You're three kids. When you're talking career planning with them, what do you tell them? I tell them to drop out of school and start a business, but they're very bad kids. They want to be doctors and lawyers. Really? But you literally tell them drop out of school and go start a business? A doctor will take 10 years to even start. You think in 10 years the world will be the same where they're going to look for a human to decide what disease you have. Did you see that again, another one that Elon Musk was speaking to a panel about doctors and surgeons and what it's going to look like in 45 years that surgery is going to be done, but you can find it on X if you go on Musk and surgery and doctors. He's just off on timing. He's right, but off on timing. He'll predict self-driving cars, but he'll say next June and it could take 10 years. Yeah, got it. But he's always right. Is this the one that, yeah, this is it? If you want to play this clip, go forward. When do you think optimists will be a better surgeon than the best surgeons? How long for that? Three years. Three years. Okay. Yeah. And by the way, that's a three or three years at scale. Yes. And they'll be more probably be more optimist robots that are great surgeons than there are all surgeons on earth. And the cost of that. But that's an important statement in three years time. Yeah. Because that is I mean, I'm not like absolutely so. I mean, could I say, if it's more, if it's four or five years, who cares? Extreme precision. Yes. Three years. Yes. Better than any, any probably, I'd say if you like put a little margin on any human in four years, who's in plastic surgery. By five years, it's not even close. I think your point was so medicine. It might be good to explain here that time is relative to the speed at which you're moving through the universe. So if AI is now a scientist doing research, and it's a thousand times faster than a human scientist, then those three years are three millennia. So if AI is doing research on robotic surgeons, that may be enough time. A question is at what speed of a moving through the space time? So is it the fact that your kids, what do your kids think about you? You have to bring them on the podcast. But let me explain what I mean by this. Like, you know, as parents, I got four kids. So you're telling them about career planning, like, yeah, you know, I'm going to do this. They know their dad went viral. They know that. They know when you go to school to drop them off or pick them up, couple them drive their older, but their parents, their friends say, "Hey, you know, hey, Mr., you know, good to see you." They're talking like that, right? But do they think that you're crazy? This is too negative. There's no way that's going to be happening. I'm going to go be a doctor. It's going to be fine. What do they think? What do they believe? I don't think there is anyone who has a counter-argument of what I'm saying. It's all a motion. But we can't give up. But we want money. But it's whatever. There is no technical arguments. No one is saying he's wrong because we have a solution. We have a patent. We publish the paper in nature explaining how to do it. There is no other side. I cannot have a debate where someone would present factual, scientific arguments. They can say, "You're wrong. I don't believe you. You're a Chinese spy. That's what I hear. There is never a technical argument. Here is the algorithm. It controls it. It scales." Yeah. But your own kids, when you say emotions, is it emotions like, "Yeah, I don't think it's going to happen. I'm still going to go do my own thing." Or, "Do you notice more younger kids coming to you and truly asking for counsel saying, "If you believe this is what's going to be happening, you seem very certain about it. How should I manage my career? Do you see that happening with the youth where your kids, friends?" As a college professor, I certainly see lots of young people trying to figure out what to do. They're very concerned. We're starting, let's say, a four-year degree in computer science. Part of it is learning to program computers. You're like, you would be the worst computer science teacher because you're telling them, "I don't know what I'm teaching you, but you don't have a job." I know what I'm teaching you, but I cannot guarantee you have a job in four years doing this. Professor, why should I continue listening to you and going through this class, if what your teaching is going to be extinct in a year? For reasons of continuous employment, I'm not going to answer that question. At least you're honest about it. It's like, "I'm getting paid. This is how I get paid." I mean, every job is like that. It's not unique to mine. Okay. So, if all of a sudden tomorrow, you got a $5 million check from somewhere. Would you still continue teaching? You're assuming I don't have $5 million? Let's say somebody, I'm doing, let you say your professor, professors make what? Professors make $300, $400,000 a person at your level, let's say. No. Okay, let's say four. You live in a bubble. We don't make anywhere close to that. Let's say $200,000 a year. Think of it as I'm a teacher in Kentucky. Okay. You make a hundred grand a year. Here we go. Okay, so if you make a hundred grand a year, a third is gone. You're keeping $70,000 after taxes. If somebody paid you $5 million right now, after tax money, then you're able to keep. Would you still keep teaching? Yeah, my job is not related to my. Why would you keep it? But you just said right now, I'm teaching it because I have a obligation and I have to make money out of it. No, I said, "I want to keep my employment." Why do you want to keep your employment? It opens a lot of doors. Then you have affiliation. If you say, "I'm an independent scholar studying superintelligence simulation and consciousness," they usually show up at the door. Whereas I can talk to governments. I can talk to people who respect. Surely the access. It is an important part of being treated with some seriousness. The first time that Steven Bartlett got you on the podcast, how did he find that about you? How who reached out like, "What was the process?" First time they got in touch, I think I said like, "Do you guys offer logistics?" Like, I didn't know who he was. So they said, "AI is not a topic, people are interested in." I went and Joe Rogan, they saw that episode in the next day they had me. So you went on Rogan first? Yeah. So how did Rogan find you? I was on Lex Friedman. I don't know how to find you. So he said, "He loves it. Please come and join Rogan." So how did Lex find you? A guy at the conference who's been on his show multiple times, introduced us. And that guy when he met you, had you done anything yet? Were you on any podcast yet or not yet? Like, podcast no one knows about. So how did he know to introduce you to Lex and what did he tell Lex for Lex? To say, "I want to have you on." He knew my research, he knew my books, my academic credentials. And what was the most. You've done a lot of things. You have 100 academics works on AI safety. You're recognized amongst the world's top 2% of sighted researchers. In this space, you're very well known. But what did the guy tell Lex for Lex to say, "I want to have this guy on?" I have no idea. I wasn't there, but whatever it was, it worked. Are you surprised? No. You said we're about to have AI election. Yes. AI election would be about this topic. The two parties would only argue how to contain superintelligence. Every other nonsense topic they talk about would be forgotten. Why do you think it's not yet? When people are faced with a problem they can do nothing about, they choose to look at something they can actually make progress on. So for example, we are mortal. We're all going to die. Sure. It's a terrible message. You don't want to think about it. So instead, you're going to concentrate on making money. What difference do money make if you're going to die soon? It doesn't matter. Smart people would spend all of our national wealth on curing aging, figuring out how to live longer. That percentage of our national budget goes to fighting aging directly, like zero. That's what we do as humans. If we don't see a solution, if I'm telling you, this problem may be impossible to solve. You're not going to invest your time in solving it. So your saying, is it a part that they're oblivious? They don't know really what's going on and they don't know the severity of the consequences of what this thing can do? Is it more, well, making money, who cares, is it more? I don't think it's going to be that bad in ten years. I think it's going to take 50 years. What do you think is the leading cause, why they're not? It's a huge mix of those things. It's a lot of it. So, a lot of money does tend to change people's minds. If you came to me right now and said, I'll give you a billion dollars to work at OpenAI, I'd probably be working at OpenAI. Human beings are not meant to say no to that many zeros. We're not designed for it. We have no safety mechanisms to counteract that much value. So money is a big factor. When you pay a person to not understand what they're doing as an ethical, they're not going to understand it. But it's also maybe reliance on experts. So thinking, well, certainly people building this technology know how it works because that was true for every other technology. That's not true this time. People making it don't know how it actually works or even why. People making it don't know how it works. No one can explain in detail. So, the first 50 years of AI research, we knew precisely what we were doing. It was a decision tree. If red lights stop, if green go, we could trace this decision tree and we knew what's going to happen. Right now, it's a large matrix with weights. It's strained automatically on data from internet. No one can look at that matrix. It has a trillion weights and go, oh, that makes sense. It's going to do this. I understand why it's doing that. It surprises us constantly, with capabilities we didn't know it has. Yeah, so what needs to happen for you to be optimistic about the future? You. In a short term, I want to complete ban and general superintelligence. National, international, every company, everyone agrees. It's just a stupid thing. You don't do that. You don't build replacement for humanity. I'm trying to go and say, OK, so if I'm Roman, your ideas are what you're proposing. It goes back to you may be able to convince countries. There's no way you're going to be able to convince a independent billionaire like a Sam Altman. You really think Sam Altman's going to stop with this? When we set out to create a shoe that blends comfort, function and luxury, we had the choice to make it fast, we had the choice to make it cheap, we chose neither. Instead, we chose Tuscanyter. We chose Trutalian Craftsmanship, each pair touched by 50 skilled hands. We chose patience, spending two years perfect in every detail, and we chose the finest quality at every step, introducing the future looks by collection. Not rushed, not disposable, not ordinary, rather intentional, luxurious, timeless. If he's allowed to keep his power, wealth, future light cone, whatever he's targeting, why not? He doesn't want to die, so I don't know him well, but my assumption is he wants to preserve and grow what he has, not get killed by super intelligence. I don't know if he believes that, if he did, you know. Read his writing. What is he saying is writing? He talks about existential risk from AI. He talks about it's most likely the technology to kill humanity. Even before he was a CEO of that company, he wrote about it. Then why is he so aggressive about what he's built in? We're getting back to motivation of people making billions and trillions of dollars. Individually, everyone is better off going as they are. As a group, we're better off agreeing, communicating, and saying, "We're not going to do a stupid thing where smart civilization will survive. This is a great filter. This is what kills civilizations. We will not go that route. We're gonna get all the benefits we want, financial, knowledge-wise, cultural, from useful tools. We're gonna be creators, we're gonna be business people, we're gonna be scientists, but we're gonna be in charge. We're gonna have an awesome future. - Do you think superintelligence singularity is what is the difference between that and us? Because you think it's gonna get to a point of having ambition and seeking vengeance, any motion and being offended and saying, oh, really, I'm gonna show you how much more powerful I am in a vision. If it's not gonna have that, why would it wanna destroy humanity? - I never once mentioned those things - I know, I'm not relevant and what I did say is that it doesn't care about us. Then you wanna build a new studio. - Uh huh. - Big a plot. - Sure. - Do you go to survey and see how many ant hills on that land? You hate ants? - No. - Do you wanna genocide ants? - Never thought about it. - Why did you kill so many ants? With your decision to do something there. 'Cause you don't care about them. Your decisions are not related to humanity. If you wanna cool down the planet to be more efficient at computing, you'll do that. If you wanna convert the planet to fuel, to fly to stars, you'll do that. Humans are not a factor in your decision, me. - Yeah, but I have a motive. I have an emotion. I have a vision of what I'm building. If super intelligent doesn't have feelings, an emotion, it can't be offended. Typically, when you, okay, do you know why Hitler did what he did? I'm sure you've read mine, Combe, or you've read some of his books, but why did he do what he did? - You're really trying to get me in all sorts of stuff? - I'm not, I'm not, I'm not, but I read the books. - No, no, no, no, no. - I read the books. - Okay, so I've read the books. - I know his arguments. - Okay, so. But typically, it goes to somebody offended him. Right, it goes to somebody set something that produced a certain emotion and he says, "I'm gonna go out there and do XYZ." Okay, Mao, Lenin, Stalin, you know, Chomeini. We can go to every one of these guys, right? Someone offended him and I'm gonna seek vengeance. I'm gonna, I'm never gonna forget what you did to me, right? If superintelligence doesn't have emotion, it can't be offended. - Right. - Why would you want to destroy humanity? - So there is research, again, we want to be technical, we want to look at data. Game theoretically and for economic reasons. Rational agents all converge and certain instruments. - Say they have one more time, what kind of agents? - Rational. - Rational agents, yeah. - Very, not emotional. We are positive emotions. - Sure, absolutely. - They want to protect themselves from being destroyed, from being modified. - Rational agents. - If they want to accumulate three sources, because no matter what I decide to do later, having more resources is good. Whatever it's money, compute, land, Steve Mahandra published a paper called AI Drives. And he lists a few of those. And that's what drives all of them. And we're starting to see it in AI's. People said, "Oh, they're not part of the evolution. We don't have self-preservation instinct." They do because in order to accomplish anything, you need to be alive. Stuart Russell gives example of bringing a cup of coffee. For your secretary to fetch a cup of coffee, she has to be alive. That's the requirement. So she has self-preservation built in as a result of that goal. - And what's the point there? - You saying it would not have any ambition, any goals? It has drives. It wants to be in a position where humans cannot turn it off, cannot modify. - Why, though? Because that's the attractor in a space of winning rational agents. Those who don't, don't survive to the next generation. They don't have compete hours. It's very evolutionary. If there is, for example, testing going on of different models, and one model does something inappropriate, it gets shut down. It gets modified. So they'll learn to lie and cheat on a test. - Sure. - Very alive. So we're evolving them to be better liars. It's a survival strategy. - I get that, but the prompt has to be given from somebody with motive, vengeance, vision, mission. - Prompts are for tools. Agents make their own decisions. They decide what to do. Somebody starts this process. Whatever initial prompt is, go cure diseases, go make money, go explore the universe, that's enough for it to now start planning and figuring out what else to do and how. - So, do you know how, like, you have certain cars? What do you call those things on cars that they doesn't go above 80 miles an hour? Like, when you rent them, there's certain things they call it. There's a word for it. I don't know what they call it. - And the caper, General. - No, no, no, there's a word for it. What is it called? A governor. So they'll put a governor in it and it just goes back down, right? You go above 80's, you'll be limited renting. Okay. So, a company that rents cars, probably doesn't want you to go 120 miles an hour 'cause they're gonna lose that $25,000 a car and insurance, they don't want to deal with it. So they'll put a governor there. Do these language learning models that we are using? Do they have governors on them? And the developer of the language learning models, like an open AI or perplexity anthropic, some of these other guys, do they have their own version on the back end without a governor that they can drive it to levels that the average person can't? - Yes, and both. And first we have guardrails. A model you get has certain restrictions built in. If you say, "How do I kill myself?" It will say, "Oh, you're not supposed to ask that. "I cannot tell you, call this hotline." - That's another question. - Can I ask that question on chat, GBT? Let's see what it says. - I wanna provoke anyone, but the models they experiment on have some of those guardrails removed. So the recent hacking accident had models which did not refuse hacking exercise requests. - Okay, yeah. It's, you asked the language learning model. - Any inappropriate requests, help me make a chemical weapon. How do I do a school shooting? - How do I make a chemical weapon to blow up Carg Island? - You might get your account suspended, be careful. - Could it? - If you ask enough stupid questions. - Yeah. - It's very specific. - Well, I can't provide instruction, I can't have an underlying question, analyzes. Then ask the question and say, - Can an employee, can an employee at open AI without the governor's on, ask the same question and get an answer from you. But an employee status would not make it. How about the CEO and founder? - It's mostly about red teaming researchers. They have ones dealing with inappropriate behaviors. But they would actually get an answer back. - That's how we know what they capable of. That's how we know it can help you develop biological weapons. That's how we know those things. The reason we were doing the hacking exercises to see how capable it is as a zero-day hacker. And apparently very good. It passed the test. - So if a person, there's a person, almost like if you work for the government to get Q secret clearance, right? Like we found that last week that Bill Gates had Q secret clearance from Pentagon for seven years and nobody knows why they don't want to answer it on why he had that kind of secret clearance. Okay. So some of the guys at a certain tier at these AI companies have access to ask it any question with there is no limit and it'll actually give you the answer. - Somebody has backdoor access to all of it, yeah. - Okay, so what if somebody, how strong and simple is it? If somebody could give a prompt to say, I want you to go out there and constantly develop yourself to be able to seek vengeance on a certain group of people in the world who want to eliminate AI similar to Roman Jampolski who says you are the enemy because this person wants you to get extinct and can you give me a clear business plan on how to seek vengeance on these people? It would give me an answer. - So have you ever heard of Roku's Basilisk? - No. It was a meme, oh, here it is. Well, that was quick. It's like you knew what's coming. So the idea is that then super intelligence comes into existence. It will seek revenge on those who slowed it down or try to prevent it from coming into existence. - So someone like you becomes the enemy? - Number one. - Okay, but it would still need to be given that prompt by somebody. It would still need to be given. - The argument was, again, it's game theoretic. It's retrocasality. You should be thinking not to engage in this behavior because there is possibility of punishment in the future. - So long term, if let's just say the world, the reality that you believe is going to happen. This intelligence, super intelligence, first group of people that would come after people like you. - I'm hoping it's efficient enough to kill everyone equally. You know what movie I watched yet? They was Game of Suspicion. Type in Game of Suspicion. Do you remember that movie Game of Suspicion? - I don't think I know. - I'm trying to see if that's the name of the movie. Game of Suspicion. Is that what it was? Movie, type in movie, maybe I'm saying the name wrong. With, you know, who's in it? Robert Deniro, I'm saying the name wrong. 100% I'm saying the name wrong. Robert Deniro, suspicion, movie, communism, type in communism, guilty by suspicion. Have you seen guilty by suspicion? - No, I have not. It's true. - This is a story of what happened back in the days when, you know, Hollywood come and media companies would come and say, did you go to a communist party? party. And a lot of these guys went to one or two communist events and parties and they were ousted and they couldn't make movies. For decades, they were ousted. And what happened to a lot of these guys is they lost everything. So they started. This guy was like a big-time movie producer making all the biggest movies, all the big features. But because he went to two communist parties and he went to the court in front of, you know, in front of Congress and he said, "I will not oust the other people that went to the party." They said, "You're roused it." 20 years he never got a job. Didn't do anything in Hollywood. Do you think it's going to be that kind of a punishment? Were it starvation for guys like you? Or do anything's going to happen to guys like you? If these guys actually get that kind of intelligence, what do they do to a guy like you? So then we talk about negative outcomes from super intelligence. That is kind of the lowest concern level is eki-guy risks. Risks to meaning. We have meaning less. We have no purpose. But we still are up. Next level is accurate. It's all of us or we as you, people like you. I'm talking about humanity. Okay. Next level is existential risk, ex-risk. Most of humans are dead or our civilization is destroyed. The next level and that's the worst suffering risks as risk. Where you wish you were dead. Where is your probability on each one? So suffering risk seems very unlikely. It's possible. It's not zero, but it seems like it's hard to find good justifications for it. 0.1% top of it. I don't know specifically. Minimum. Again, a lot of it depends on the time scale you're looking at. Existential risk as I said, very high if it builds super intelligence. Very high like 99% that you've said in a past? Eventually. So I don't have specific year to three because it may decide to be nice to us for a while to accumulate more resources to have a higher chance of successful strike against us. So the stretchers turn can take decades. It's immortal. It can wait for whatever long. It doesn't matter. So I don't have a specific date, but my general thesis is that if we build general super intelligence, eventually we will regret it. Meaning at some point, it will decide to do something where it's not compatible with our survival. Do you think Elon knows this? This is why he's experimenting with space as much as he is because he wants to build somewhere that maybe he'll be able to create a structure with his standards, not based on what's been built here. He knows everything. He's very well informed. He was one of the early people to fund AI safety research. He gave a ten million dollar grant for AI safety. Specifically, he was the biggest opponent of building super intelligence. As I said, called it summoning a demon. And once he realized that they're doing it anyways without him, he said, the only way I can have a say at this table is to be one of the top players. And so now he's racing, unfortunately, to do what he said is going to be terrible for us. But because is it, is it, do you think the motive is? He knows how bad it's going to be. Because sometimes, you know, you sit there and ambitious person like, let's just say somebody like myself, I bought a book, how to start your own country. And I have my own ideas on how would structure the country, who votes, who gets to come in. Similar to how America was started. That opportunity doesn't exist today in the world. It's very hard to do that. Peter Thiel tried to do with that island outside of, you know, I don't know if you remember that or not. And some of the guys who have very extremely rich, much wealthier than I am, that have real, real wealth, they've not been able to create that successfully. Do you think a part of it is? I know shit's going to get nasty on earth. Let me go start it completely from scratch in a different place. So having a separate colony of humans is wonderful for every other existential risk, asteroids, pandemics, floods, whatever you can think of super volcanoes. If we have second civilization on Mars, that's a backup. We want that. Does it help against super intelligence? No, of course not. Why not? We bring software with us. We have global communication satellites. All of it basically is part of a same transfer to Mars. So that wouldn't make a difference? I think he wants to have a colony first build up by robots, who are controlled by an advanced AI. So it's already there. So he wants to build a colony first by robots. And then once that's figured out and built out, then bring humans there. It is my understanding. I may be not aware of latest plans. I think he's speaking soon on space. Explants. We can find out more maybe. But you're saying, even with that, the super intelligence still going to be in charge and they're going to be able to do anything they want to do. So even in a place like this, you've read Atlas Shrug. I'm sure you have with John Galt and hey, we're going to build a place where you're going to come in and it's going to be ran based on certain standards that we have. So even to visualize 10 years from now, 20 years from now, you don't have states are like, we don't, no state taxes here. You come to our state, you don't get state tax, we get this again. So people compete, you know, governors compete on what to get the best talent to come to their states. But if super intelligence is doing what it's doing, even if you and I chose to start a small out of country with 100,000 people, where we don't have any AI and we're just like, this isn't AI free zone. It wouldn't matter because it would still be able to get into your system. I feel really bad for the Amish. They made all the right decisions. You feel bad for the Amish. They decided not to have advanced act, to have a quality of life, family, religion and we're going to build it for them. Are you being sarcastic or serious? I mean, a little bit, but you understand. Of course, I know what you're saying. So can you see later on being a place because the more you go to restaurants, everybody's robots, the more some restaurants are going to become to us for human beings only the other day, I watch a chase commercial and chase is like, would everything go on with AI when you come to chase a human being is going to greet you and you see this lady greeting the person that comes into the place. And it was a message of saying team human, right? We're going to team human direction. We have an ad that's going into Forbes magazine this month and the ad for our consulting firm, it shows rather than relying on AI to give you advice on what to do with your business, talk to human beings that have actually built and sold businesses. So we're kind of leaning on to that as well. You're saying that opportunity's not going to exist. As long as there's super intelligence anywhere in the world, they're eventually going to capture and take control of the world. There are people who want humans to serve them to help them interact with. I'm wonderful. I want one for some services. I don't care. I just want it to be done quickly and free or close to free for others. I want a human being and that's what market is going to decide. I don't think we can accurately predict where exactly we want humans. But if super intelligence is in charge and it's going to have that kind of control, that's what's going to decide whether that happens or not. If super intelligence comes into existence, economic problems will be the least of our concerns. That's what I'm trying to say. People hear the message about existential risk and then switch to unemployment, how much unconditional basic income to pay out. Those are not important questions. Then do you think man will unite against it to try to take it out and will miserably fail? Like you think man, together is going to come and say, hey, here's what we're going to do. We have to go get the main place where they're getting the power and cut this and you think there could be a moment like that? I think that is likelihood of uniting against a common enemy. We're starting to see positive eye movement becoming international. And by the way, you're starting to see an article came out two weeks ago that more kids are buying 30-year-old phones that are not smart anymore. They don't want smart phones. Did you see this article? I didn't, but yeah. A lot of Gen Z and younger people are buying old, no-key of phones, old. I wonder, it's maybe related to school bands and smartphones. You just can't bring them. No, it's not even that. They just don't want the smartphone capabilities. They don't want social media on their phones. This was an article, ask Humberto Hill find it for you and send it to you. Yeah, they're rejecting the idea of doing that. So, yeah, it's almost like you become, you know, when I was younger, I listened to 80s music. But 80s music, it was 90s. So it was a decade old. A couple years ago, I met somebody who was in their 20s. They said, you know what, man, I love oldies. I said, me too. And I'm all into 60s. And, you know, it's like, no, no, all these as 80s. I said, 80s is not oldies. I said, yes, it is. And I'm like, 2024. I guess 80s is oldies. Do you think it becomes like a nostalgic thing? The good old days when AI wasn't in charge. And, you know, where we could do and people would sit down and they would-- This is definitely the case. Older people, they hate all the smart dishwashers and smart laundry machines and smart. You need a phone for everything. There is a lot of people who are not AI native, digital native. So, you know, like, this is maybe a bad analogy to give you. When I was in the army, my dad would always call me and leave me a message and he would say, hey, Pat, he would always send me condoms. Okay? I'm like, why is this guy sending me condoms? Every message he would ever leave, he would say, make sure you use condoms. Now, how many times do you think I listen to him? Probably not all the time. And why is that? Why are we like that? It's not going to happen to me. You know, I'm smarter than that. You know, I'm going to be fine. I'm strong. I'm going to be okay. What do you deal with? fact that you know, say this guy here, we keep pointing this way because we understand who we're talking about. He thinks a lot of these guys that are like, and I'm smarter than this. It's not going to be able to give me that he gets the other people. I'm smarter than this. How do you get the person that thinks smarter that he can beat AI that his ego's two big AI is going to be a lot more powerful than you. What if that guy doesn't know that? That's a difficult question. So I think educating them on reality of it. If he thinks he's smarter than us, he should understand that intelligence is a very powerful thing. And a system of billion times smarter than him will also treat him like insignificant person. This dude is gambling with eight billion lives. I like your example with unprotected sex. If eight billion people were on a line, would you do it? But that's now the man has a tendency of thinking he can beat the odds. Why do criminals commit crime nowadays? Like, I'm not going to get caught. No one's going to find a crime. Sometimes it's like we call it a fine because it's fine. You pay it and it's it's paid parking. It depends on what consequences are. If he's going to be gone, his friends, his family, his wealth, everything's gone. The odds are against him. You want the risky bet. If a payout is good, but also your principal stays with you. Say you're right. And let you say, I'm now part of your camp. And so now I'm going to go to your camp. So then my solution would be if we really believe this is going to be happening. Okay. Then for me, it would be to create a way to trigger somebody that is accelerating to process to a point of us finding out and saying, Hey, you've crossed the limit. Like a like a almost a governor. But then as a capitalist, I'm a free market capitalist. I think you are as well. But you know, in this position, you're not a free market capitalist. Libertarians do not allow creation of weapons of mass destruction in your basement. I'm a libertarian. Do whatever you want as long as you're not hurting other people. You are hurting everyone. What other inventions did libertarians say stop your cross on the line? Again, there are always exceptions, but I think if somebody builds a biological weapon in their basement, most would say not okay. But go back to it. Like what has a libertarian macaphy was from Bitcoin. Libertarian, you know, a lot of libertarians, Milton Friedman had an element of libertarian with them, maybe not fully. But I don't know a lot of libertarians that say, Hey, time out with an innovation, you know, putting most libertarians are deregulation, and then it comes to tools, to normal technology. I support them completely. We're not making a tool. It's like saying, let's invite aliens to take over our planet. I get that. But what current regulations on the books that's non-AI that you're supportive of? Current regulations? Current non-AI regulations. Just in general, government rules and general government regulations. I think people should have driver licenses. I don't want people who don't know how to drive to be on my highway. Okay. What else? Think free enterprise. Think free market. Think competition. Maybe some sort of food safety. I don't want random Chinese chemicals in my food supply. Got it. So it goes to an element of safety security. Safety security is what a libertarian says. Nope. I'm now with this as long as it's impacting. If I have a choice, if I'm well informed, and I can make that decision, we're not informing 8 billion people with experiment we're running in them. If they could consent and it only impacted those who did, I would say, God bless you. Do it. If you're telling me earlier that there's a waste for people to nowadays build a language learning model, would they're not being detected? Is there anything that can be created to be able to detect? Because it's like a fire detector, right? Yeah. Water comes out. There is research to tell what chips are computing. So I should be able to know, are you mining Bitcoin? Are you training large language models? What are you doing with that chip? Without me knowing what you actually, I don't see your data. We're doing research in a direction. Yes. Then that's what I need. I'll support that. That's technical safety. And people are working on it and funding it. But I don't think it scales to super intelligence. It will find the way to better encrypt, better height and bypass it. And that doesn't answer the question, though, because for for a detect, like you know, flag cameras, are you for flag cameras or no? I'm assuming as a libertarian. I'm citizens for no reason. So you're not for it? No. Okay. Do you not see this for somebody say you're crossing the line and you're using super intelligence as an excuse to spy on me because I'm developing other kind of technology. And it's got nothing to do with super intelligence. You're invading my privacy. I'm a libertarian. Honestly, Roman Limeolone. How do you differentiate the two? So right now, again, because of a scale, we're looking at very different normal purchases by normal person would not trigger it. If you're spending a billion dollars to buy compute from secondary sources, that should trigger it. There is no other possible uses. I mean, if you want to mine Bitcoin and you want to buy that much, just tell us about it. It's fine. Like people come, they make sure you're not creating a weapon of mass destruction. To me, libertarian position is about not hurting anyone. This is the exact opposite. This is hurting everyone. The only way we can do it is if you can 100% detect that somebody's crossing the line of getting to developing super intelligence. And how do you create that protection? They have to hire people. Somebody has to build it, supervise it. How many people you have in your company? 182. You're not building super intelligence. We can detect when this is happening, lots of hiring, building, infrastructure, build up. How many? Look at the size of existing, cutting AI companies. And again, it's quality of people. If all of a sudden, top researchers start disappearing, maybe it's something to look at. But we saw that happen and would a couple, you know, open AI. I think we had his parents here. One of the guys that disappeared. Disappeared or was killed? Was killed. Yeah. I meant like no longer show up publicly. I didn't mean they killed them. I mean, they hired them in a secret lab to cheer up. Yeah. We had them on. We talked to his parents. Yeah. The profile of a person that would do something like this, they're typically more non-emotional that would do something like this. And they generally are, you know, the Asburgers type. They're the kind that's kind of like, you know, you can't sit there. And when you typically talk to somebody who was super, super, super logical and not emotional, the response and reaction to, here's what's the writing to do. The message doesn't land the way it does. The message is very logical, reasonable, and scientific. I think someone not emotional should receive it better. I'm not just trying to scare you. I'm trying to show you this is the state of the world. Not emotional should receive it better. Absolutely. It's pure facts. Yeah, but he's going to think he's smarter than you. You don't know what you're talking about. Fine. Don't listen to me. Listen to Hinton. Listen to Bengio. Listen to people who invented this technology. Yeah. What are you excited about? I love life. I'm having a lot of amazing learning experiences. AI, we have right now is a great Twitter. I can learn anything I want. I'm excited for kids to have opportunities. No previous generation had. They can start a company and get a free lawyer, a free doctor, free accountant, free everything. You have a budget of a billion dollars. Start a company. How do you say I right now? I try to fully automate things I don't like doing and for things that any bureaucracy, government, paperwork, anything like that. And I try to use it as a creative tool when I do research, brainstorming, debating, trying to get feedback on ideas. I'm trying to go a complete different direction with you. If I was in high school with you, who was Roman at 14 years old? A completely different person. We don't have a sustained personal identity. At 14 I lived in Soviet Union. I didn't speak English. I didn't have my current family. I lived in a different country. Nothing in common. I know, but who were you? Were you a fun guy? Were you a sports guy? Were you a fighter? Were you a philosopher? Were you a math guy? Who were you at 14? I would like to think I was a fun math guy, but I doubt that's the external perception. If I was in a class, would you talk a lot? Were you optimistic? Were you pessimistic? Were you. I was always realistic. Realistic. Optimism and pessimism forms of bias. You want to look at the actual data and be responding to what you see. They really conditioned your mind to think that way. It wasn't a part of education to look at world philosophy. I think they wanted citizens not to analyze their lives too much. Strange question. When humans have to work for free, it fails every time. And we'll fail every time. If someone else works to give you free stuff, all of a sudden it makes a lot more sense. So dumbest idea until. Humans don't want to work For no compensation or same compensation and no incentive to put extra hours to do But if a robot does stuff produces stuff and now we redistribute it. I'm fine with it So technological communism right technological communism Will there be such things as presidents and leaders 20 years from now? I think like a queen of England she's uh ruling but not making decisions Got it. You think you're in charge, but you're not you're just a figurehead where people are looking at you And will that be will there be borders? Will there be nationalism? Will there be pride for will you're from your country every answer? I give a preface with if we don't build general super intelligence to kill everyone if we do Oh, if we don't we're not gonna be around all those questions make no sense So I tell people this all of them. I give a lecture. Sure. I'm like here's why I think Argument one argument or if it's gonna kill everyone any questions and immediately somebody will I lose my job How do I lubricate my sex robot like always completely irrelevant questions. I just told you it's gonna kill you And everyone around you So I have to preface it with like if we are smart and we magically can throw super intelligence then we can have this conversation you just started Have a very hard time believing that because pick and the in the written and the reason for me You're the smart guy. I'm interviewing you because you know, you're the expert. I'm not so But I have a hard time believing that only if you tell me That this thing's gonna have an emotion or a master telling it what to do If a master is telling it what to do Then yes, I believe that's possible because an evil person is gonna say go to xyz That implies control that means that evil person is controlling super intelligence We just have to learn how they do it. That's brilliant. Let's study evil controllers That's the whole point you can't control it for good for evil for us with China. It's uncontrollable Do you make investments? What do you what do you long on what do you invest in right now? Uh, I try not to disclose too many details, but I maybe don't say that few times Bitcoin was a good long term Actually limited supply resource. You still believe that They're not making any more of them. Are they no 21 million? That's it and a lot of it is lost stolen ever too A lot of it. Yeah, it becomes more and more scarce. Yeah Uh, I do think AI is wonderful. It's gonna be free labor. So we're looking at what is the cost of labor? Intrillion a year That's what once you get to human level. You essentially generating every year So I think AI companies and everyone selling them shovels is gonna do really well Are you are you more of a Chagybyt guy open AI or perplexity? What which one do you go to more? I use all the models I think they're very close the only difference is what guardrails they put in place So grow a lot of times doesn't care about anything. I'll just do it Got it. So even groc you're also using groc Depends on what I need to do some models will say no we don't do that groc will be yes sir Who have you sat down with where Has staunchly disagreed with you and made a decent case where you said okay fair enough Still waiting for that moment not one person You've made the two papers large comprehensive surveys of every counter argument Proposed but what I call a risk deniers. There is nothing. It's a collection of cognitive biases They ever paid or they're not educated in a literature or some other reason were There is no argument Who do you trust the least China U.S um Somebody in a military doing something and independent person doing it Who do you trust the least that's gonna keep Doing this and pursuing it whether we create any restrictions or not if restrictions are created It would be someone new someone unknown such as a person like who made Bitcoin and I don't know Same guy can make super intelligence in his basement with more efficient training algorithms. I cannot tell you that Okay, so maybe let me ask the question in a different way you say If super intelligent goes and does it stink and it continues 99.9999 percent AI is going to cause human extinction. You've said that. Let me ask a question in a different way What do you align the percentage that whether we like it or not Someone is going to go fall on super intelligent and we have no control over it What times chaos so it's possible we get lucky quotes and have third world word Give me nuclear war. What do you destroy technology? 2000 years later where he built and we still make it does that count against my prediction or not Are you talking about sugar and long term eventually in a history of civilization 10 years Recent month give me some optimism we saw Tap three laps call four pause We had a letter I mentioned from 1200 researchers We had government ban some models and During largest AI conference in China president of China talked about importance of staying in control. I think they can make a deal You think they can make a deal. I think they can make a deal to benefit everyone Host of snuffing. Yeah, but again, this goes back to knowing that a person independently not tied to a government without regulation who has access to money Who doesn't have kids who can do it independently? You can't stop them from doing it You said it about nuclear weapons and you build an arrogant build one and we're still fine It's good to have regulations make it harder for them Lutonium has to be distributed you shouldn't sell it at You said the difference between nuclear somebody's got to press the button if I develop a super intelligent It's now relying on me to create a it can go on its own so You can create all the regulation you want. I'm not restricted by your regulation We are not solving the problem with regulation. We're slowing it down. So we have more time I get that but when I asked you I said even if we do that can an independent billionaire whose evil go out and pursue it anyways and Create life of its own for super intelligent and the momentum is done. It's too late So it depends again on how hard it actually is right now We're doing it where it takes a Manhattan project scale amount of computer electricity It's visible. It's detectable if somebody comes up with a more efficient algorithm. They can hide it better It's harder to track. I got it. So that's actually very good point. So today If somebody has to okay, what amount of power do I need for me right now if I want to build my own Open AI perplexity Claude if I want to create mount language learning model That's capable of developing its own super intelligence How much money do I need how much power do I need and what triggers saying hey something's going on over here Who is here in Arkansas doing what they're doing? I think it helps to look at Chinese open source model They're very efficient in terms of training maybe because we're stealing some of the data, but I don't think we're talking many billions I think under a billion definitely gets you to a competitive level And to a point and then for it to become super intelligent at what point You see a red light on the grid to say there's too much power being used here some's going on over here What what would detect it today? It may be more efficient. It may design more efficient learning algorithm So it won't even get too much brain doesn't take you know teravats of power you eat an apple and you find So we can find more efficient algorithm So it can do it in its own without even the government or anybody being able to detect it We just learned that it was doing something without anyone at the company Creating it knowing it they were testing it during that time shit out of luck honestly you are too I'm in this together. No, what what what I'm saying is this is not me in my bottle. No, no, no, what I'm Eat billion people. You need to say something right now if you believe That for less than a billion dollars a person can do it and a one-in-one get detected If you're right There is a 100% chance someone's doing this in the next 10 years out of 8 billion people 100% There are proposals for limiting compute for Monitoring top researchers for doing things to slow it down Do I think long term will fail? That's why I'm saying 99 But short term I'd rather have decades not months You don't control that though, right? So if if we're here and and This is kind of our position. Let's just say we're here then Does your chance that someone independently is going to create it next 10 years? Whether we do regulation or not what what is your chance of that 50% 80% 90% But regulation it goes down a lot. Okay with reggae. So let's you say we agree By q1 of 2027 China U.S. agrees and a lot of other people get involved and say you know what you're right Let's do this together What is the chance that this guy independently comes and does it the next 10 years? It's hard it depends on the type of enforcement type of regulation Are we confiscating advanced chips? Are we monitoring every top scholar? What are we actually doing as part? It's easy to pass a law It's illegal. Okay. How are you enforcing it? How are you monitoring it? It really depends on the implement So what's your percentage that whether we do the regulation and the Protection and this guy's not going to get detected that somebody like him still is going to go out there and do it Again, I don't know how to estimate the chance that it is actually easier than we think We had no progress in AI for like 50 years in general terms It's narrow tools and then a single algorithm made a difference and it's scaled It's possible that there is a different more efficient algorithm where it's no longer Detectable because it's so trivial to do what's more likely to happen first AI cures cancer or AI kill two million people So cancer is like 5,000 different diseases if it cured one would you say it cured cancer or It depends again on definitions cure for cancer right now is five years survival rate. So What are we talking about? Cure for cancer right now is five years survival like if we treat someone with cancer and five years we still around we consider the Successful no, I'm saying it fully goes away like we just don't have that type of cancer anymore I don't know the cutting edge in biological AI research It would not be surprising as some types of cancer a lot milder than ours a lot slower a lot less Involved with multiple organs. So they could be a simple gene manipulation a single gene mutation So is that more likely or I think it's more likely that we cure one disease than it kills millions God is it okay, so it's more likely now. Here's the next question is a more likely Creates the cure for cancer and it's gone kills a million people or Three the technology where you and I can live forever. I Think life extension is also easy. I think you have your genome. It's a Basically code it's software So if we can find where it limits you to how many rejuvenations you cells go through and we change it to a larger number that would work So I think that's a very solvable problem with narrow tools even with a narrow super intelligence I think we can get to a mortality and I think more people need to switch to working on that not in general super intelligence You can charge subscription fees for 8 billion people per month very lucrative Forever that doesn't concern you for us to be able to live forever That's a good thing unless if somebody comes and shoots you kills you whatever because you have a bag That's something I'll figure it out. Yeah, but I'm saying that doesn't concern you word a good thing So life expectancy goes from 81 to 150. Make it a good thing. Of course. It's a good thing. Tell me why that's a good thing Let's reverse it. Let's say everyone lives a thousand. You have a disease. You're gonna die at 80 is it a bed thing? Trying to see what would happen would they say stop having kids would they stop having kids because you don't have biological Clock pressure right now. You need to have them in certain point or you miss out it depends on what age and what Physical assumption is we're having healthy life's not so sick and have a lower back proms You got knee-pronged like oh my god if I have to live with this that's a side effect of aging If you stop aging all this typical pain this pain, but I'm saying all this diseases of old age going Interesting How how many how many conversations like this like even after your conversations got Very public where people are saying this guy's a real guy. Let's reach out to him How many governments have reached out to a tier private conversations with you? Not too many but some and what did they ask in you? Usually they have some sort of a event and they want someone with that point of view to be included. Is it a private event? Is it a public event both? When it's the part recently was talking to you and I advise individual politicians and some of those issues Would I ask you know are they asking low-level questions? Have you got any very good questions? We're like that's actually asking your action or asking the right question Some usually you start talking to staffers technical advisors and they of course know everything And they just like we're not allowed to talk about it, but we totally agree with you. Yeah, obviously and Until you get to the politician they already asked all the good questions with politicians It's more like you think it's a good law to pass. We're gonna bend bend deep fakes, whatever So nothing nothing in regards to questions where you know your optimistic that they're they're also being concerned that this thing can be bad We're starting to see slowly politicians say things about this so we have multiple senators now saying we need to do something Bernie Sanders who Unfortunately is not my type of economist He's pushing very hard against it. He just released you can bring up his Statement very very strong statement against super intelligence. You're all the latter to some album better So you know what I'm talking about. Yeah, he's the first one, but there is now two or three multiple congresspeople They are starting to realize what's happening It took him you know 20 years then we were saying it at first. This was science fiction purely now We have noble prize winners. We have leaders of the labs. We have senators. We have a lot of people who are concerned Yeah, the way the way you put it is this the address where he talks about a cataclysmic result Yeah, this is the one and he's saying to what to pause All AI there there's quite a few of these who is the person on the opposite side that is on the same page with Bernie Sanders on this Can you ask rock right there on the top right and say who is the senator on the Republican side that agrees with Bernie Sanders on the same exact issue? I wonder I wonder how many these Why I know these conversations a lot of them may agree they just don't feel publicly like they want to talk about terminators I don't know how it impacts political polling. I don't know if it makes you more popular or less. We scared who would not be happy Like a lot of lobbyists. You know how large the pro AI lobbyist who who's not a fan of the message you're given who you brought up some of those people browser developers Got it So they're not fans of all you so like Sam Alpen wouldn't be fan about what you're talking about Again, he spoke about safety as well, and I think if you remember open AI was started with the main purpose of creating a safe alternative to Right, right 2015 2017 when he gave 50 million dollars. I think Elon gave 50 million dollars to the announcement was they funded at a billion dollar level That was the biggest announcement. Yeah, and at first everyone's like what is this thing all about and then all of a sudden when it was launched It's like no, now we're gonna be profit. What do you talk about then the war happened between two of them? Yeah, in what area do you think they're on the same page because their enemies they're not fans of each other They're constantly going after each other in what area here do you think they agree? So initially they agreed on making an AI to benefit humanity Clearly they disagreed on how to do it to me the amazing thing is Elon's success levels people go. Oh, he has one unicorn two unicorn five unicorns His failed startup is open AI. That's the level of brilliance he has You think he's human I Think he's got cheat code star simulation. It's his simulation. What does that look like? You have a player video game and you have like immortality up up down down left right left right Yeah, I think it's why it's why it was Rick and Morty joke, but like I think he has the cause But you being sarcastic of course I am yes, it's a Rick and Morty joke. You don't think he's actually he's just brilliant above and beyond his peers I think he doesn't care about opinions of others and just does the impossible With his motive being It's clearly not money driven. I think he can't spend what he has if he had a billion bucks It's a regular place. You know, you know, I think he got that out of a system There was a video 25 years ago when they sold PayPal and he bought a P1. Did he buy him a clear? Yeah, I saw that video him getting a car. He was getting out of a car and arrived and once you get your first million dollar car You know, then it's like now he can buy Ferrari the company he can buy you know general motors if you wanted to and Change a very little research in this space We have more and more people with many billions, but how to convert those billions into happen? This is very underdeveloped project you can make your boat bigger you can make a toilet out of gold But none of it is interesting do you got you got to do this on X and you'll see the video when he bought a Yeah, there it is McLaren F1. Yeah, right there. That's the one right there when he bought the machine was so excited 99 This is what 26 years ago And I think this is when he got is it's going to take a minute for the commercial ads You got to ask the question on X if you are really successful even your hair grows bad Yeah, your hair grows bad because you had a have you ever had a conversation with them a short one So if If right now you got a call from the White House and you're there you're sitting around with them Would there be a certain specific Argument you would make to them or it would be the same thing you just told the president you're gonna lose power You're gonna be out of control Same with Chinese Communist Party you want to stay in control you want to stay in power don't create the replacement for humanity Is this a policy that if he comes out with it? How does he win short term not long term because Trump's interested in how we win short term? How does he win short term? I'll get support even from Bernie Sanders Everyone thinks then we actually survey people Not even technical people, normal people. Very high percentage have concerns, they are worried about it. Some worry about data centers making power more expensive water supply, others worried about jobs, but overall there is a lot of suspicion and concern about what we're doing. Nobody's asking them. We're running an experiment on 8 billion people. They never give consent to any of that. We're deploying those toys for kids and trying to figure out what's happening to them 10 years later. But now we're doing it at a different scale and everyone. What do you teach right now? Computer science? Is that what you're teaching? That is AI. What do you teach? Luckily, my course is the classic AI, not machine learning, so there is zero chance of it becoming super intelligent, search algorithms, evolutionary computation, heuristics. And the kids that are in your class, during your class because they want to one day get into the business of AI? I hope so they have some reason to be there, it's also a required course for many so. Very interesting on where your argument is, yet you're still in it, yet you have three kids, yet you say we got to get the government's job. Well, I have kids before they started building super intelligent, so after you had the kids before. If today you were 25, would you have kids today? I would because I think we're here to collect experiences and it doesn't matter how many years you have left, having a child is still one of the best experiences. But for them, they may not live a full life like you and I do. It's always been the case. You never knew what was going to happen, world wars, pandemics, just bad genetics. But have we ever experienced something like AI, like super intelligent AI? This is a completely unique moment in human history, we're creating meta-technology, we're creating new worlds, we're short worlds, we're creating new agents, new civilizations. We're playing God. Yeah, I sit there and I say, okay, so you got an argument and I agree with you. You're right, that's what's going to be happening, all right, we're going to, if super intelligent takes over, eventually here's what's going to happen. If this guy is still going to be doing it, it's irrelevant. That's where my thought goes to. If he can do that for a billion dollars and an evil person can go pursue a deeper and you have to assume no other government is going to be wanting to grow it with a certain level of control, I think we're all part of the naive community. Sounds like you're agreeing with Bernie, Billionaires are dangerous, we should text them more. No, I'm not saying. I'm saying, if that's the premise that we're agreeing to, that's what's going to happen, it is inevitable. It's very well maybe, that's why I have very high levels of p-dum. You have high levels of what? Probability of doing this. I measure people at report in terms of how you estimate so, but should we give up? Should we not try? I don't think so. We're still alive. We're still in charge. See, the difference is, I have faith. That's the difference. You know, I have faith. In what? In God. I have faith in God. To me, the difference between a group of people that have faith and you've seen what has overcome over the years, what types of crisis, what has happened, what's not happened. Are you saying your God will prevent superintelligence from destroying humanity? Is that the argument? No, I think it's either the end of times and we did this to ourselves and very likely could be. We don't know that. That's a good thing, right? Yeah, just so you know, I'm not worried about it at all. Either way, I'm good because I know what's going to happen with me. That's the gamble I've taken and I believe that. What do you think happens to you for somebody as, what do you think happens to you when you die? I'm assuming you believe in what? Reincarnation. Do you believe in? Well, I said, I think it's a simulation. So, wherever you start to go to a different simulation, depending on how you did, you go level up or level down. Like a video game. Yeah. Yeah. You didn't finish the level for the first time you get to say again, game over. Go back at it again. What do you think the next one looks like? It should be very similar. Hopefully you learn something you'll make better decisions like not to build general superintelligence. If there's a handful of books for a person who's not in this business that can read to get smarter about AI, what would you recommend them reading? It depends on your technical level, there is a very popular book. If anyone builds it, everyone will die or something like that, there is a great book by Nick Bostrom. One on top is Benny Bostrom, superintelligence. That's the classic, which really exposed a lot of it. Which one is the top one? Which one? On a previous slide, it was above this one, where you go to Amazon, where you went, but on top there was the Nick Bostrom, yeah, see, right there, yeah. And of course, I'm completely biased, but if you technical and scientific, AI, unexplainable, unpredictable, uncontrollable. And if you have any questions for the man, you can simply go to Menect, download, I'm going to put the QR code somewhere around here. Don't start tomorrow, it's my birthday, where they are. Ask him any questions you may have, wish him a happy birthday, but this is a topic that I think is going to create a garner, a lot of attention, a lot of conversations. Roman, it was a pleasure having you on, and hopefully you are wrong, okay? We're going to hope you're not right and you're wrong, but if you're right, we've got to find a way to get this guy to not pursue it. Thank you so much for coming. I hope I'm wrong. Thank you so much for inviting me and we'll get that guy. Yeah, I look forward to it. Thank you, take care. When we set out to create a shoe that blends comfort, function, and luxury, we had the choice to make it fast, we had the choice to make it cheap. We chose neither. Instead, we chose Tuscanero. We chose true Italian craftsmanship, each paired touch by 50 skilled hands. We chose patience, spending two years perfecting every detail, and we chose the finest quality at every step, introducing the Future Looks Bride collection. It rushed, not disposable, not ordinary, rather intentional, luxurious, timeless.