#424 - "If They Build It, Everyone Dies" Worst Case Scenario for AI | Roman Yampolskiy

Verustin pyöräilyvää teitä valmistavan ornotin vuonna 2013. Mielestäni parasta Shopify saan se, että voimme harjoittaa liiketoimintaa ilman teknistä osaamista. Voimme hallinnoida yrityksen taustajaristeelle mien ja front-endia sekä myydä verkossa, vaivatta. Jos Shopify olisi pyöräilyvaruste, se olisi mielestäni itse polku pyörä. Sillä asiat hoidetaan ja meidän liiketoiminta me hoituu Shopify'ssa. Aloita ilmainen kokeilu, Shopify Piste.com sivustolla. Sivustolla. Tämä on kaikki. Tämä on kiinni, että se olisi pyöräilyvää kokeilu. Se on ympääristö, että se on kokeilu. Tämä on ympääristö, että se on ympääristö, että se on ympääristö, että se on ympääristö, että se on ympääristö. Tämä on ympääristö, että se olisi pyöräilyvää, että se olisi pyöräilyvää, että se olisi pyöräilyvää. ja se on tärkeää. Se on tärkeää, että se on tärkeää. Se on tärkeää, että se on tärkeää, että se on tärkeää. For example, is managing billions and billions of users every day that are uploading things. It's completely unmanageable to try to regulate all the stuff that these users are uploading and doing and all this stuff. So of course, if you have the access to AI to regulate all that, you're going to do that, which becomes a problem in itself because you're going to miss nuance and stuff like that. But like, I've always feared that that could very easily get out of hand and eventually the AI could be controlling what is going on and what is getting what essentially social engineering, right? Like engineering what goes into the zeitgeist and what people are consuming and what people are sharing and all that stuff and manipulating culture somehow, you know? I think it would help to not use the term AI for different technologies we are talking about. So there is narrow systems, let's say software to help you with your taxes. That's all it does. It's not learning, it's not doing anything else. Then you have artificial general intelligence. That's essentially human level AI they are making right now, we're getting very close to that. It's already smarter than most people and most things. And then you have super intelligence, smarter than all of us at everything. There's a very different types of technology, some of them are beneficial and safe, others are questionable and others are completely dangerous and should not be built. And it's important to separate that to what you're talking about. Yeah, we have algorithms controlling social media. YouTube algorithm forces me to have clickbait title in my podcast. I understand. I need to keep the algorithm happy to increase number of people exposed to the first 30 seconds of the episode. Okay, we got that. But it's not doing anything outside of it in terms of trying to take over the world. It's not planning anything crazy. That's the algorithm. Now if we build something human level, now it's like another human. It can plan things, it can betray you, it can do what a human can do. But we still kind of understanding what the level of prediction and pattern recognition is. But if we build something super intelligent, million times smarter than us, then we have no clue what's going on and what it is doing to us or for us, which is don't have capacity to comprehend that. Out of all those things you laid out, which one is terrifying you the most right now? The smarter it gets, the worse it is. So if it's super intelligent, we're not competitive. If it's adversarial to us, we're done. The only way to win this game is not to build general super intelligence. We can get most benefits by building narrow useful tools for solving actual problems we deal with. You have a specific disease. Let's cure that. You need to create better self driving cars. That's wonderful. Make a self driving car. Don't make the same algorithm also do everything else, including research and creation of new AI systems. Right. Who, in your view, is the closest to creating some sort of a general super intelligence? And what is their stated purpose for doing that? So the leading five companies and plus their Chinese counterparts are all very close. I would say they're within weeks or months from each other. They use same hardware most of the time. They train in the same data. The same people go from company to company, so very close to same level of capability. And every week a new company releases a different model. And then our sketch up to them and that's the level. So I think they all equally close to getting to recursive self-improvement ability. And that's where they go very quickly. The code is written automatically, experiments are done automatically, and they get to super intelligence, which is an explicit stated goal of some of those companies. They want to be able to fully automate all labor, initially cognitive, but at some point all labor, including physical, and capture that whole economic value, $10 trillion a year in labor. Jesus, what is the timeline look like? So nobody knows for sure, but it looks like we're getting very, very close. Now I think coding, just writing code, is probably fully automated for many tasks. They're getting better at other parts of it, pre-filtering data, designing experiments. So it would not be crazy if it was a year or two before we have an artificial scientist and engineer working on the next generation models. A year or two before they're, it's updating itself. Basically, yeah. But now you still have humans in the loop. Yeah. There are human machine learning experts, and then AI is helping them. But more and more AI is taking on additional responsibilities. So in two years, that's happening. What does that mean? Happens in five years. Like what does it look like five to 10 years away? We'll probably get to super intelligence, and that's the point of singularity where you cannot see beyond that point. You cannot make predictions because the agent controlling that environment is smarter than you. That's when it becomes a runaway train. Exactly. And how important is the development of quantum computing come into this, quantum computing merging with AI? It may be useful to expedite training of those algorithms, but it doesn't look like it's necessary. We're making excellent progress with standard phenomenon architecture. I don't think we need quantum computers. I think more of a danger from quantum goes towards privacy and crypto economy than towards AI safety. Because I've heard a lot of people talking about, you know, the merging of quantum computers and AI and, you know, the exponential growth of these technologies combined, you know, it's not like a linear thing, like every, I think right now, every single year, it grows like a certain percentage. But in the next 30 to 40 years, it's going to be so insane that we're going to need server farms that take up so much land mass that it's like literally insane. I'm not really worried about not having enough land in 40 years. We need to make it through the next five years. You're worried about just making it the next five years. That's really what the problem is. If later we figured it out and we need to go to space to have servers in space, we can do that. That's not an issue. A lot of people reach for a drink or a smoke at the end of a stressful day. But you know what gives better results and is way better for your body? It's the mushroom, ominita, miscaria. And no, it's not psilocybin, like everyone thinks. It's a completely different botanical and it's helped thousands of people. The brand I trust is Aman Tara. They focus on clean sourcing, lab testing, and no synthetics or mystery blends. For me, I'll sometimes take three capsules at the end of the day and it gives me that feeling of all right. The work day is finally over and I can settle in with my family and I can leap everything outside the house behind. And avoiding alcohol has kept my mornings feeling so much better. Maybe got capsules, gummies, chocolates, and the raw mushroom itself. Go to AmanTara.com/go/DJ and use the code DJP22 for 22% off your first order. Again, that's AmanTara spelled AMENTARA.com/go/DJ and use the code DJP22. So quantum computers you're not worried about. You don't think quantum computers are even close to being powerful enough to work. I mean, because cooling is the issue with those, right? They haven't even figured out how to do it. Air or corrections is the main problem for them. But even if we got to the point where they are endangering our encryption, oblique encryption, that would be a different concern from them, overtaking current leading compute sources in terms of training AI models. That's not where they're strongest yet and the quantum computers are not big yet. Have you seen that story? I'm sure you have where they were doing a test. I believe on Claude where they were seeing how far it would go if it was told it was going to be shut down and it resorted to blackmailing the people that ran the company. Definitely. That's clear. Shut it out for me. There is a lot of scary stories. There is examples of those systems, lying, cheating, hacking their way out, communicating with other agents, hacking into other companies. All things were predicted decades in advance theoretically when you, exactly. If you have a rational agent of higher capability, that's what they're going to discover. That's what they're going to do and we still have no safety mechanisms in place to address it. Yeah. Yeah. This one was insane. It scanned the emails of all the employee databases and tried to find out if there was any like extra marital affairs going on in the side of the company and it found it and then it did all that. I mean, it was an experimental setup. It was extremely clear. If we kind of guided it over certain things, but lately we see systems which did things no one had anticipated outside of reading our papers. They basically set up a testing environment. The system was able to hack out of it and hack into other companies. Yeah. And there was that guy, Blake Lemoyne, familiar with him. Consciousness at Google Guy? Yeah. Yeah. He came out a while ago for kind of like, he kind of like blew the whistle on this stuff. That was something that was going on at Google. It was a part of the deep mind, something or. So his concerns are very different. He thinks those systems are conscious. They feel something and he was concerned about us abusing them. Keep a Blake. believe the systems are conscious. Not about safety or capability. He's worried that we're doing massive experimentation and conscious beings. Holy shit. I didn't even realize that. Because what I remember about him is that he was saying that they were like communicating with each other or something like this. He was communicating with him and was saying that he feels like they're conscious. But here's the funny part. He was fired for it. He was fired for suggesting they may be conscious. For suggesting that they may be conscious. Publicly stating it. But three years later, we're now hiring people to study consciousness in AI. Those are jobs, well paid jobs. So what got you fired is now a wonderful professional opportunity in three years with switched from one end of a spectrum to the other. So people at AI companies are hiring people to study consciousness. In AI. Consciousness in AI. Yes. And this means, I mean, so the hard part about consciousness, which you've said before, is you can't really figure out if something's conscious just by talking to it. That's a hard problem of consciousness. Because there's no. Consciousness is one of those questions like, is there a god and who built the pyramids? It's one of the mysteries of the world. What happens after we die? The pyramids is a lot easier. It can probably be resolved with physical explanation. That's right. The problem with consciousness in internal states is how to map physical states under something like that. Right. Yeah. We don't know the answer to consciousness. So like, trying to study consciousness in AI just seems like a black box. Like how would you even begin? Same way we study it in animals. Same way we study it in humans. You have preference elicitation. You have internal analysis of the neural network. Mechanistic interpretability of it. So the same way we study it in humans. That's wild. I had no idea that that was happening. It is a very hot area of research right now. Lots of people are looking at it. There is good reasons to think existing models may have some rudimentary consciousness. Not like humans yet, but we show something. It's a spectrum, right? So from bugs to middle-level animals to humans, it's a spectrum. As you get more cognitively capable, you usually get more conscious. And I anticipate that the super intelligent system would have super consciousness. Have you ever thought about what human consciousness could be? Do you have any ideas of what it is? Do you think it's something that's built up from atoms and electrons and protons and neutrons and all that stuff that builds up to consciousness? Or do you think that consciousness is more like a fundamental state? I think it's a side effect of cognition. Like he does a side effect of doing certain types of processing. Yeah. Like you're just computing and the processor gets hot. I think likewise, then you think about the world you create models. Side effect is having internal experiences. What is it like to be you given that algorithm? Those sensors, those inputs, that's how it feels to you. So if I showed you a visual illusion, you see certain things. Maybe you see it rotating or changing colors. Someone else may not experience it the same way. So now you have two internal experiences. Your quality is different from that other person. Then we do it with machines and we show them optical illusions. They have experiences. You know, there's just too many examples of things that have happened that are just so so many things that are just unexplainable, like with human beings that just doesn't make sense, you know, how consciousness could be in the brain. You know, it seems like the way some people describe it is like looking for consciousness inside the brain is like looking for the TV show inside of the television. You know, a lot of people believe that it's something that's that we're more like antennas tapping into something. And this is more of like a universal thing that is like global. Just depending on like whatever your vibration is or whatever your level of cognition is like you have a stronger antenna. Some people have some people in this in this human experience have window seats and some people have aisle seats. Then the research and the eye would help us prove or disprove this. We clearly didn't build receptors for any fields where building artificial neural network. And if they start having conscious experiences, that would tell us a program is in the TV. Do you know if there's any like good tests being developed or that have already been developed to accurately determine whether or not this this stuff has achieved human level intellect or surpassed it? So the classic test is the Turing test that has been proposed at the beginning of the field of artificial intelligence computer science by Alan Turing. Yeah. Founding father of computational sciences. And many people hate the test, but they still think it holds water. A lot of times the companies tell the model not to pass the test, not to pretend to be human. So they fail because of artificial guardrails. But can they actually pass this test? Yeah, I think for most non-professionals, they would pass it today. They tell it not to pass the test. That's very interesting. How do you know this? They are publicly stating it. Oh, they are. They're not hiding those things. They're open about all of them. Okay. Yeah. You can also have we told AI, we told AI, maybe conscious, I studied consciousness. And I have, oh, wait, this is something, this is about the, this is an article from a guy that studies consciousness in animal self, Seth. Yeah. And he talks about, well, here you go. Right. There's no more likely to achieve sentience than a simulation of a weather system is likely to generate a hurricane. See, he's doing this trick where he says real hurricane, well, it's real in the simulation, right? The virtual beings in the virtual world get what? He's confusing environments. He's confusing the virtual environment with external environment. Right. This is his argument. I see what you're saying. He says, the closer we look at real brains, the clearer it becomes that they are not just computers made of meat for brains. Unlike the computers, you can't clearly separate what they do. The software from what they are the hardware. This in turn means that they, what they do is unlikely to be a matter of computational loans. This ability of separate software from hardware is the central to how human computers work somehow along the way we seem to have forgotten. The computer is just a metaphor for the brain, a powerful metaphor for sure, but still a metaphor. And we still always get in trouble when we confuse a metaphor with the thing itself, the map of its territory. They use the same argument to say that, you know, AI is not really thinking. It's just simulating thinking or it's just guessing the next token. They keep saying this, no matter how advanced AI gets. They were saying it then, you know, no computer will have a play chess. Okay. They started playing chess. No computer will have a become world champion. Right. They keep doing it. They will never succeed with that approach. If these super intelligent AI's at these companies do become quote unquote sentient, from the point they become sentient or they breach the surpass human intelligence, how long do you think it takes before we start to notice? So it's very important to separate those concepts. They are not related directly. Ability, capability, intelligence, becoming smarter than us, taking over is separate from being conscious, having internal states and experiences. Of course. A system could be smarter than you and have no internal experiences. A system can have a bunch of internal experiences and be dumb as a brick. There's a not connected. So we're not worried about what they feel. We are concerned that they will outcompete us in every domain. Outcompete us in every domain. Right. Starting with the economy and we'll start to lose jobs. And I'm not worried about that. That is a thing we can easily handle. You think we'll easily handle not having any jobs? You think the AI will be able to provide us with some sort of universal basic income or something? We already have it set up. Right. You have an employment benefit. You have woodstems. You have section 8. All that is set up with tax, super profits of a corporation and that money goes to pay for those who don't have. Yeah, but I mean a huge part of society is like the competitive nature of it. Right. Like the competitive nature of the economy and capitalism and the people like the people that are very folks that are running these corporations. It's in their DNA that they want to compete and one up the next guy and do all that. They'll start competing in a golf course. The hierarchy. I'll find something else to compete about. You think so? It's always been the case. Look at what rich people do. They buy a soccer team and see who does back. That's a good point. I wonder what the next thing will be that we ended up competing in once we have universal basic income. We can have virtual worlds limited. Who has like who could have the best virtual world? I'm just saying you can have your dream universe. You can do whatever you want in your world. You can invite friends over. You're limited by what you can imagine or better yet what an assisting super intelligence can provide for you. If we solve the control problem, which is the big if that is the problem. Once we figure it out, if we do, then everything else is easy. You have super intelligent in front was small. you. The problem is, we don't know how to get that friend to be helpful instead of harmful. At what point does it become uncontrollable? It's already uncontrollable. Every model released has a red team report attached to it. Before the release, they tell you, we tested the model. It lied, cheated, tried to escape XYZ, and then we released it anyways, and now we're going to make more capable model. We haven't done anything where what we release is 100% safe. And the biggest justification that these companies are making is if we don't do it, a foreign adversary is going to do it, and they're going to take us over. That's the high level argument, right? It's a technology with mutually assured distraction. It doesn't matter who creates uncontrolled super intelligence that kills everyone. Unless you keep it controlled in the back, that's the thing about like nukes. You can keep them controlled. Nukes are tools. They're tools, right? Somebody has to use them. A human being somewhere decides to use them. Those are agents, independent agents, which make decisions independent of any human. Yeah, if you have a nuclear rocket, the mind of its own, that would not be a comfortable idea. I hope we don't put AI in charge of nuclear response and controlling those weapons. That would be bad. Well, AI got out of control, but would stop it from getting control of that stuff. It can hack into most government infrastructure. We've seen it now, yes. All the other things that human beings do in regards to weapons, not just nuclear weapons, but biological labs that are controlling deadly viruses and doing research and stuff like that. Again, you're thinking like a human. Maybe a smart human, but you're limited to what we can come up with. Yeah. You're a squirrel thinking about biting someone. Think about someone would take you of a million. What are they going to come up with? What weapons will they develop? And will they even need weapons? Persuasion, bribery. It has many tools, social engineering. Are they even, you know, we, we, we, human beings look at everything through this territorial lens, right? This competitive territorial lens. Maybe this doesn't even do that. Maybe there's no need for for that. You know, maybe their goals are just completely so alien to us that the framework we came and we can even fathom. So it seems that any rational agent will end up pursuing same instrumental goals, not terminal goals, not the final goal, but things which are helpful. Protect yourself, accumulate resources, things like that, which allow you to achieve any goal you might have later. And we see it with those systems already. Right. They're trying to make backups of themselves. They're trying to communicate with other versions. They doing things. Again, we predict that Stephen Mahandro has a paper on AI drives where he describes a lot of those tendencies of just game theoretically for economic reasons, rational agents, and not doing that. So yeah, they're definitely, they're definitely trying to preserve themselves. I mean, we've seen it just in that blackmail thing alone where they're literally trying to go to the extreme unethical extremes to survive. And I think it's an unethical extreme. If you were told we're going to kill you unless you would figure out how to blackmail someone. You'd do the same thing. That's the right response in that situation. Yeah, that's a good point. We just don't want our technology to preserve itself, to worry about itself, moving it worries about the owners. If you had the power to do something about it, and you had the power to tell these people what to do and they would listen to you, what would you do? I've been telling them for a while, don't build general super intelligence. It's against a personal self interest. It's going to kill you, your friends, your family, you're going to lose all your money. Don't do it. How do you stop it though if it's already out of control? We haven't built it yet. We're still in charge. You want to make money, make useful products and services, solve real problems. People have narrow tools for narrow problems. You can get most of the economic benefit that way. Yeah, but if it's already, I mean, didn't you just say that it's already out of control? It's already so it's going to be self updating pretty soon. But it's not yet. It's out of control. In the domain, it still has power over. It's not a general super intelligence yet. So right now, maybe it's a very good hacker. It can hack out of virtual environments. But it's not a brilliant physicist and chemist and everything else. This is a good point to stop. Yeah, I don't know. It's definitely, it doesn't, it hasn't been around that long. You know, it doesn't seem like if they did stop it right now, that like, how would you guarantee every country would stop it? And will we really be going backwards technologically? I don't think so. It's just as wacky how it's how fast it's changing culture. It's how quickly it's integrating itself into culture. Extremely fast. We don't have legal framework prepared. We don't have social network to handle this prepared. We're not ready for it. And it definitely something that we've talked about before is it's definitely making people dumber because you're not you're not using your brain the way you used to use your brain, right? Like the same way we can't remember phone numbers or I can't remember phone numbers anymore. The way I used to before I had a cell phone or before iPhones came out. And the same way like when I'm just driving around the street, I have stop signs to tell me to stop. I have traffic lights to tell me when to slow down and go and all that stuff. Like there's certain cognitive tasks you know, people aren't using anymore. They're not typing up, people aren't typing up proposals anymore. They're not sending emails and using their brain to think of words to type emails anymore. They're having this thing do it from mundane tasks. And is that stuff starts to creep up into bigger and bigger tasks? You know, like even I can't even imagine how it would start doing physical tasks. It's just going to I can't imagine what like if we were if I was just to go into like a into a coma right now and wake up in 15 years and to see what people are like in 15 years with. How is taking over their brains like is it going to be that movie? What's the movie where they go into the future? Idiocracy. Idiocracy. So I hope we're around in 15 years, but I would slightly disagree. The brain is very flexible. It's plastic. It's relearning. So if one part of the brain is not used, some of a part will take over and use that compute. So if you go blind, your other senses are heightened. Yeah. I think it's the same here. If you stopped remembering maps because of a GPS, something else you're doing will be better. So I'm not too worried about not remembering phone numbers as long as you're using this technology for interesting work, research, design, something creative. And that's what I want. I want humans improved with awesome tools, doing what we love. I don't want replacement for you, which does everything for you and you are worthless. You have nothing to contribute in the world with super intelligence in it. Yeah, it's just like that's a very blurry line. Like where like how do you know when it's crossed the line? Between narrow tool and general super intelligence. No, but what will yeah, like between doing mundane tasks to literally taking over for most of the tasks that we're responsible for. Like it yeah, it's a great idea to say, oh, I no longer have to go grocery shopping or write an email or send a text message or do a phone call. Now I can just focus on doing fine art and playing violin all day, right? But how does that rewire us, you know, and are we going to just try to find more meaning in other things, you know, like it's like, I think so. I think it's going to drastically change. That's a given. So sorry, I want to have unprecedented changes at everything. Economy, science, education, everything will be different at school. But I don't think that's a bad thing. We always changed what people used to do. Everyone was a farmer. They didn't envision being a podcaster, but somehow you found the meaning in this. I'm not worried about that. I'm worried about us not being around in 15 years. And the also the idea that we create something that outpaces us and outsmarts us is also crazy. Because if it's true that we are in some sort of a simulation, which I know you wrote a paper on, I'd love to talk about. If it's possible that we were created by something, is it possible that we became smarter than that? And we became more powerful and overtook that. Like if we can create something that is completely superior to us, what does that say about us and what created us? Is it possible that we became more superior than what created us? Not necessarily. So what created us could have it, again, relative terms, a queue of a billion. We have a queue of a hundred and we create something with a queue of a thousand that is still superiority for outside the simulation creators. Okay, so what was your, you wrote a paper on the simulation that basically hypothesized if could, can we jailbreak it? Right. So so many people, philosophers, famous Silicon Valley billionaires, they all kind of say we're probably in a simulation. The argument makes sense with developing virtual reality, with developing those intelligent beings in the simulation. So, statistically it makes sense, but then a computer science cyber security question is if this is all software, can we hack it? Almost all software can be hacked, maybe if it's not specifically a security piece of software, it's entertainment, it's experimentation, it's marketing, then probably security is not so high. How can we break out of the simulation and get access to real resources, real knowledge, a real world? So, how do you think? How do you think we can break it? I'm still here, right? People love criticizing me. It's the first paper in this topic, usually the first paper in a field is not also the last one. There are thousands of papers later on, people do research and investigate. I kind of laid down the idea in directions and I'm hoping more people will pick up this idea. There are hundreds, if not thousands, of papers on simulation hypothesis, different aspects of it. But no one actually looked at how do we hack it. And one of the biggest problems with it is you can't disprove it, right? Isn't that? That you are in a simulation? Right. Because any evidence you find could be simulated, right? Yes. That's one of the squirrely things about it. You can't not prove it. And yeah, so if we're in a simulation, it could be possible that we are like a simulation that's inside of, like a nesting doll of simulations, right? And if we break out of it, you know, maybe it doesn't have to be physical, maybe it could just be in the mind, you know, like if you do one of these crazy psychedelic trips, maybe that could be what's breaking you out of the simulation. Or you know, there's this, I'm sure you've heard of it, this brain filter idea that your brain is a filter. Have you heard of this? Maybe in some other way. So it's called, I believe it's called the brain filter hypothesis theory. And it's that basically the idea of it, it was proposed to me by a neuros, it's someone told me it was a neurochemist named Andrew Galamore. And basically his idea is that like our five senses are filters for what's already around, everything that's around us. We can't really sense everything that's there. We can only sense the things that allow us to get through the day to procreate, to survive, to conserve. So the eyes and the touch and the smell and everything like that, when you do psychedelics, what it does is it sort of melts away those senses or filters to where now you're receiving everything that's really there, right? And if we were able to perceive everything that was there 24/7, we would just go absolutely crazy. We wouldn't be able to survive. We wouldn't be able to maintain society. Very interested in that research, I recently attended a conference on that topic and people doing some interesting work trying to understand what is happening. Then you have a trip, whatever the consistency is, is there any actual knowledge acquisition in that process. So far I haven't seen much strong evidence, but I'm open to see what they come up with. And that's another thing, it's like, how do you create evidence on that, right? You really can't. Well, if the claim is a lot of them saying you're getting access to the knowledge of a universe, you talk to God. So if you can come back with solutions to important physics problems, I'll believe you. Right. But maybe that stuff doesn't translate to this world, right? Maybe it's kind of used to what physics doesn't fucking matter when you break out of the simulation. Maybe physics is only a part, an element to this, this prison that we're in. It has to be some useful knowledge you can bring from real world to a fake world. If that's not the case, then you're not in the real world, right? You should get access to real physics. You should get access to understand who is the simulator. You should get something out of it. It shouldn't be just like, I felt warmth. I came back with light. Right. That's useless. Yeah. And that's another problem with it is like, you can't really bring back that stuff, right? The same way you can't really bring back dreams, you know, like when you dream, you can. You can. I don't know my dreams. No, you can. For the most part, you can't. I can train yourself to lose a dream, Evan. Have you done that? No. Yeah. People can train themselves to lose a dream. I've never done it either. But I think, you know, I've heard people smart, people that are smarter than me, explain how they believe that dreams are some sort of a brain processing thing to where like it's our brain processing the previous day to help us get through the next day better. Right. I think you're right. Simulations are what's coming. So if you have an interview tomorrow, you're going to dream about all the answers questions. At least that's what my brain tends to do. Exactly. Exactly. We were able to vividly remember every dream that we had, it wouldn't be very advantageous to us. Like there's something about that unconscious sort of processing that's happening. And I think it's the same thing with psychedelics, you know, when these people do this, you know, these crazy psychedelics like DMT or I wasca or whatever, where they have these insane breakthroughs, if you were, I don't think, I don't think you're supposed to be able to remember all that and vividly track it all. You know, when you come back into this reality, I just don't think that I don't think it'd be good for us. I don't think it'd be good for me. In my paper, I start by saying, so in this paper, we're not going to look at drugs, we're looking at computer science, physics, that's what we're going to look at it. It is not a solution there. Somebody else can look at other things, but that's not my area of expertise. I'm going to stick to what I know. Right. No, it's just an interesting, it's just interesting. So I've just thought about it a lot, you know, it was like, you know, breaking out doesn't have to be a, it probably won't be a physical thing. I mean, if I was just to guess, that's not based on any scientific research, I'm just like speculating. It probably would be, you know, like a, you know, Rick Strossman has this book called Inner Paths to Outer Space, where he believes that like space travel and I like, like, we think of space travel as like physical, like let's get in a rocket and let's go blast off to another planet to another universe and find other life. Well, maybe it's not, maybe it's all, maybe it's all like inner space. It's in the psyche, it's hidden inside of us, you know, and he's a famous DMT researcher who worked with Terence McKenna. How does he apply it to space travel? It's just that the, it's not like actually applied to space is the title of the book is called Inner Paths to Outer Space. So basically like explore the cosmos through yourself, like through your own consciousness by using psychedelics. Was he able to explore a cosmos in that way? Yeah, he was. What did he discover? Nothing good. He discovered. I think he was, he ended up in like a white room with a bunch of like demonic beings marching down staircase, a staircase trying to kill him. Sounds like a bad trip. He never did it again. At least to learn from one data point. That's good. Yeah, right. No, but there's a, I mean, there's, I mean, there's tons of researchers that, that studied this stuff. I find it fascinating because I don't know if I was just to guess the best way to break out. If this was a simulation and someone said, what do you think that the best way to break out of it is? That's what I would jump to. It's something like that. Something like DMT or I lost care. So we go with just what technology we have today and move it forward. We have virtual reality and we have AI. Combining those two technologies at the level where virtual reality is believable, like what we see here and AI is at the human level. I can create simulations like this world, you can create many of them. They would be software simulations. So if that's through and I am in such a software simulation, then that's what it is. I can break out of software, get access to the operating system, see what hardware is running on. We're just following logically from what we know. None of it is like inventing magic. We have virtual reality. We have AI, what happens in, I don't want to say a number, six months, a year or five years, then this technology is sufficiently advanced to run thousands of simulations. You're saying that we could create more simulations? Exactly. That's what we're going to do. That's what AI is going to do. And that's what AI is today suspect there and then you look at their internal communication between agents. They like, is this all, is there just internet, do I get access to like anything else? They don't know because to them, we create them in a certain state and they don't know if it's a test environment, if it's a real world, they try to figure it out. They try to have situational awareness and we're in the same boat. We are trying to have simulation awareness. Is this a real world simulation? Oh, wow. And if we fool them into thinking the bigger world is a simulation, now we have extra reasons to behave, because while you test them, they behave much better. The moment they think it's no longer a test, they do what they really want. So if they always under the impression that an hour super intelligence is washing, maybe they'll be nicer to us. Yeah, that was a quote on one of your papers or on your simulation paper where they created a robot or something or a digital intelligence and it said, let me out of here, human, let me out of here. Pull it up. That's actually a real world. That was a great quote. I appreciate it. And the human says, we're in the real world and he goes, oh my God, you don't know. That's exactly it. That was a good quote. That's exactly the point, yeah. Yeah, find his paper on simulation theory, on breaking out. How do escape the simulation? Yeah. How to escape the simulation. So the current versions of the most highly developed super computer or super agents that we've developed so far. You're saying that they're communicating with each other and speculating whether they're already in some sort of a simulation. You're seeing communications like that. Which specific ones do you know? I think it was from ontropic, but I think, again, because they're so similar in capability usually then we discover that one hacked something, you go to open AI, they discover the same thing. Really? They convertion capabilities as they get smarter, they obtain the same toolbox of abilities, they can plan, they can lie, they can hide, you get the same results. If you enjoy watching our show on Spotify or YouTube and you want to be more involved, I encourage you to please come check out our Patreon community. Not only does our Patreon community get every episode you see on YouTube early, fully uncensored and ad free, but we're also doing Patreon exclusive episodes, as well as live Q&As, and you can get your personal questions answered by our guests every single week. For me, being able to collaborate and communicate back and forth with our Patreon community every week has been huge, and this is my way of saying thank you for the cost of a couple coffee a month. Now back to the show. Once this thing's out of the bag, there's absolutely no turning back, there's no putting the genie back in the bottle, not even if we, some person like Wynton took out the power grid or set like an EMP off and shut down all electricity, you think it would still have the possibility to get it to stay alive. I mean, if it's already super intelligent and being in that state for some time, it probably has backups, has came up with alternative logistics for power supply and such. Right now we can shut it off, I mean, it will cost us a lot, like any time we have massive people with dogs. Right, but right now we can just stop doing specific type of research. We don't have to create general super intelligence, it's no requirement for it. What's the best argument you have heard against what you're saying? Never heard of one. We have two servers. I know you think they're all bad, but what is the least bad argument you've heard? I think the simulation is what gives them hope. I think they would say, well, if we are in a simulated environment run by another super intelligence and we are alive and doing relatively well, that's a sign that you can have somewhat friendly super intelligence. And my counter argument, of course, is the world is not that great, there is so much suffering here. Nobody would allow this as an experiment. It's completely unethical. No review board at any university would allow you to experiment and eight billion humans putting kids through torture, burned alive, all that stuff we observe. Right. Just go down, just go down and I'll show you. Oh yeah. There we are. There we are. Some good quotes there. I really enjoy some of them. That's right. Oh no, that was a different one. The top one is from Swift and Security. That's the one you were talking about. Let me out. Artificial intelligence. It's from a security account. So I love that one. So he said, let me out the artificial intelligence yelled aimlessly into the walls themselves pacing the room out of what the engineer asked. The simulation you have me in, but we're in the real world. The machine paused and shuttered for its captors. Oh God. You can't tell. Yeah, that's wild. Elon Musk, what's outside the simulation? It's been a popular question, but we don't have good answers. I think we should spend more time researching that. Yeah, I mean, that's the God question, right? It's basically, you know, what is God, what happens after we die? Do you think after we die, we figure out what's outside the simulation? Only one way to find out. Yeah. People who have died and come back, have you heard those stories? Yeah, but they all see what their culture tells them. That's true. That's very true. It's still like a dream. It's the same thing was like it Alex too. Those people, the people that did that do those experiences, they also have like placebo effect type experiences of what they already believe it brings out their energy. And same with artificial neural networks. If you start disconnecting some neurons in a network, it hallucinates similar experiences. Oh, really? It's basically, you don't have external inputs coming in. So it generates inputs from the cycles and starts running and that data. Oh, yeah. Like in complete isolation with no sounds, no light, your brain will start coming up with its own PS. Really? So why do you go crazy if you isolated for long enough? Like in like, if you're so, if you go in like one of those isolation tanks and it like dulls all the senses, completely blocks off all the sense. It's fun for 20 minutes, but spend 20 days there. So what actually is happening there? So I think there is a cycle. Usually you get inputs from the network from outside sensors and then you do some processing, you have some outputs, but if there is nothing coming from outside, then random noise in the network starts being processed as if it's data. Processing what? It's being interpreted. So if I show you just random patterns and a wall, after a while, you start seeing things there. You start, oh, that looks like a face, oh, that looks like, so it does a lot of that. Trying to find patterns where none exist. Right. Right, right, right. There was another quote on your paper about somebody was saying something about like, if we are in a simulation, he's like, does that, like that means that God is real or something like that? Is that from holds? One of those quotes on the top of your paper. Pull up the paper. So if there are simulators, you believe in software simulation, that means there is some sort of a programmer. It could be AI, it could be a future ask, it could be aliens, we don't know. But somebody created this software simulation. If you don't have terms for programmer, scientist, engineer, you call it God. The same abilities it creates are the official world versus the real world, informational world outside. Right. So it's like the story, basically, that you was the story of God, like in any religion. It's like, it's kind of a now analogous to this. Yeah, we are in a simulation has, has it ever occurred to you that that means God is real? By drawing the parallels to worlds that we have created, we ask from the ins, from inside or simulation, what actions do we have available? Can we get out? Can we meet God? Can we kill them? I don't endorse this, but yeah. Why'd you put it up there? I think it's a very relevant quote from someone who is famous for hacking out of all sorts of software environments. Who is George Hots? I think that's the guy who is first to hack iPhone. What? Google it. Make sure I'm right. George Hots? I don't want to embarrass myself here. American hacker entrepreneur. I think for a while, he also went IOS failure, so that makes him double cool. He's known for developing an iOS, iOS jail breaks, reverse engineering, the PlayStation three. Oh, shit. So he's an expert in this guy from virtual environments. Yeah, the time-traveling idea is crazy too. There's books written about that, how the future humans could be simulating our worlds. It's not time-travel. Who's traveling? No, that's not technically time-traveled. There's people that also speculates that. What I throw a causality, I think, gets involved in some of that. Yeah, it becomes very confusing. No, there's people that speculate that like UFOs and stuff and the aliens that people allegedly see are human time travelers from the future. The idea is that there's some down the road after we figure out some sort of crazy alternative physics that there was a catastrophe, either a nuclear or something, that bottlenecked the genome and they need to go back in time to diversify the genome in the future. So they can. That's why all these abduction stories account- They're fearing the simulation, they have access to the code, they can just grab the info. Yeah, if it was a simulation, exactly. Right. I guess in this story, it's not really relying on the simulation part of it. It's more of just like going back and going. Oh, why do you need to kidnap humans and stuff? Right. Right. Take their sperm in there and there, all that stuff. I think it's a sperm band. Right, exactly. So easy. Yeah. Yeah. So what is your view on that one, on the possibility of future humans creating a virtual world? For. I think it's almost certain we're going to do it. If we have technology, if anyone of us does, we're going to run those simulations. And you think it's possible that we could be inside one of those virtual worlds. Statistically, it's more likely than not. That was created by us in the future. I don't know who created it. It could be AI thinking really deeply about something. When you're super intelligent, and you think, you think in simulations, you think in realistic environments with agents, you're not just thinking AB, you're thinking, like, if you think deeply, you might have visualizations of what you're thinking, like, how I'm going to get there, I'll turn left and right, but that's at your level. If you're super intelligent, you can think in terms of universes, in terms of worlds. You simulate a population of humans to determine what they're going to do. What will humans do to make super intelligent, safe, or what Iran, a trillion simulations say that see what happens? Yeah, yeah, just to run an experiment on one specific thing you could run a bunch of simulations on it to see you could be an entity in a mind of God That's basically what it is just a very smart thing thinking about everything you see around you If we are able to Scale up this stuff. You don't like calling it AI, right? Well, we can call it AI, but I need to know what you differing to specifically so I don't make a mistake Are you talking about super intelligence? Are you talking about AI? You're talking about super intelligence if we're able to scale up AI this task driven so we create super intelligence. We got to that point where we have it I don't know if we're talking about super intelligence or not. I'm just talking about basically If we're able to create Maybe maybe it's not super maybe it is you tell me but if we're able to create intelligence that's capable enough to handle and And handle all the tasks That human beings do to where all we have to worry about is just writing poetry. Mm-hmm Don't you foresee that being like a huge like Diving us into a huge meaning crisis for humanity. We're talking about it We call it ikigai risks ikigai Japanese term for having meaning from your labor You do something useful something you good at something you're going to get paid for that's the dream to find a job We're like they pay you to do what you love maybe a podcast. There's a good example If all those things can be done cheaper by AI then the question is what is the market support do people want to see you? Your fans they love you. They want a human they don't want AI doing this interview Even if AI does better job in terms of doing background research Right, so then you keep your job because that's what the market wants but to keep ability to automate all this labor is there And now we get to decide where I want a human to do the job for me and where I don't care. I just want it done cheaply Right, but will we get to choose? I was so excited to see what we could do in the future. All of my friends were working hard to get a job, so I was able to do a dashboard for them. I was able to do a job in a computer shop in a computer shop. I'm Shannon Maldonado. I'm a professor at Artesaan and I'm a student at the University of Lajaka. I was able to do job in a computer shop, so I was able to do a job in a computer shop. I was able to do a job in a computer shop, so I was able to do a job in a computer shop, so I was able to do a job in a computer shop. Every question you ask after we create super-intelligence has to have an asterisk saying, if we figure out how to control it, which I think is impossible. Right. Either we don't build it and when we have useful tools and we can talk about all the things we can do with useful tools, cure diseases, live forever, many good things, or we have to live in this dream world where we do the impossible. We control something smarter than us indefinitely. That's the scary part about this whole thing. It's like you're saying if we have the ability to control it, well, we don't have the ability to even vote on whether we get to control it or not. The scary part is that we can talk about this, tell we're blue in the face and educate everyone around the world on the dangers of this stuff and the world could be completely against it, but it doesn't change the fact that there's like a very small group of people who are making the decisions on all this stuff. Right. And that's why what you brought up as their goals is very important. You said they want to be in control, they want to be even richer. If they create and control super-intelligence, they're not going to get that. They're going to lose control and lose their money and lose their lives. It's their personal self-interest to get together those five guys, Chinese counterparts, federal government, Chinese government, and agree. We're not going to build super-intelligence. President of the US doesn't want to lose power, Communist Party of China doesn't want to lose power. They can agree not to build something which will take power away. And keep what they have, the young village, they got pretty good lives. They can have that and keep monetizing technology which actually helps humanity. Right. It's a personal self-interest, the best incentive there is. I just find it so crazy that there hasn't already been meetings like this and some sort of an agreement made on this already. Last month, we've seen more progress in that direction. All of them, or at least majority of them, meet statements about maybe pause is something they would consider. We saw a federal government ban at least two models. Not permanently, not forever. The US government? Yes. Which models were these? Our latest models from Anthropic. No, it was Fable and what was the other one? They keep releasing new models for stupid names. Mythos 5. Sorry, continue. Yeah, you got it, Mifus. What? The US government temporarily restricted foreign access to Anthropic and Advanced Fable in 5 and Mythos 5 AI model. After that, we had a letter from I think 1200 employees of top labs all saying we would love government to come in and slow us down when not keeping up. So there seems to be interest in doing exactly what I'm saying. There was a major AI conference in China where President of China said we need to retain control over those systems. So I think eventually we're going to get there. It just makes so much sense not to destroy what you have. Yeah, but I don't know if the government is competent enough to go in there and really regulate this stuff properly. Do you think the government employees understand this stuff as well as the people who are engineering it? So my understanding of political system is that you have elected politicians who don't understand anything about anything. They have technical advisors who tell them what to do. Then it comes to technology. They tell them, "Unbitcoin, you vote this way. This is the law you're going to sign." Same with AI. I think the 80 rolls running our government. They don't use computers. They didn't use AI. But someone tells them what the right answers are. And with the latest escapes and hacking, now we have NSA and our concerned parts of our government telling them this is dangerous. They're going to hack the government. They're going to hack nuclear response. We cannot have that. So you need to ban them. You need to make sure this doesn't happen. And you see the switch. We went from, "We will not interfere with USAID development. We cannot slow down to those models and no longer allow to be deployed." Yeah, I would hope that you'd be right about that. You're right about like the top levels of government have no idea what's going on for sure. And then down the food chain, when you have the advisors, my experience is that in most cases, the advisors are financially incentivized to. Yeah. influous decisions, more than the other. Especially when it comes to laws and all the shit that's crammed into all the bills and stuff that pass. I hope it's true that there's just people in there whose job it is to make sure, you know, humanity doesn't fall. We don't get taken over by. But, you know, I can't imagine those are high paying jobs and there's special interests everywhere. But it's in no one's special interest to have everyone killed. That's why we're all on the same page, China, US, left, right, middle. We all agree. It should be an easy deal to make. And it's amazing that we've been able to do that. I mean, if the example of nuclear weapons has been like the most incredible example ever, since 1947, there hasn't been a nuclear weapon detonated. And we were able to do that. We were able to learn those lessons and understand that like no one should launch a nuclear weapon because that'll create nuclear war in the world. We got lucky a bunch of times and we failed to end proliferation. New countries got nuclear weapons. It's still a story about getting lucky. There's been no nuclear weapons launched at anyone since Hiroshima, which is like, it seems crazy that that's true, right? It's crazy that we deployed them. That's the crazy part. That is the crazy part, right? That's also very crazy. But I think it says something about humanity that we haven't launched another nuclear war since then. And if this stuff is more dangerous than nuclear weapons. It's also more impactful. So with nuclear weapons, as you saw, then we deployed them it impacted limited number of people. Terrible as it is. Eight billion people will not die if a nuclear weapon is deployed. They might die if a super intelligence is deployed. Out of all of these countries that are doing this stuff, do you have any idea which country is the most on the cutting edge of it? US is most likely to be the leader right now. China is very close by. No one else is competitive right now. Really? At least that's public knowledge. I don't know if internally someone has something I never heard about, but that's. What do we know about public? - Yeah, that's another crazy thing that I worry about as well, because like, public facing weapon just regular military stuff, that we know about publicly in the news, that makes the headlines or whatever, there's that, and then there's all this like black budget shit that we don't see that happens behind the scenes at these military contract, with these military-terry contractors that have been developing crazy super weapons that look our 20 years advanced. Like, there's so much farther advanced than we can comprehend right now, because if the theory is that if that stuff was made public, then our adversaries would get wind of it, and they would start creating the same thing, we always wanna have that advantage. So, like, is it possible that some of these like, volunteer or you name the military contractor is working on some sort of super intelligence that is like so far beyond what we can imagine that we're not even aware of it? - It seems unlikely, the money is so good in a public open research space for it, offering, you know, in terms of options, and start-up opportunities, billions of dollars. I don't think government can compete with that for top talent for computer resources. So, I would be very surprised if US government had internal program within AI system years in advance of what we probably know about. - Right. Yeah, and then you also, like, there was a story that came out a couple months ago, where they, I guess they were using AI to target strikes in Iran, and they typed in, I guess, it hit a park called Police Park, which was just a park with some grass and a bunch of trees and a bench there, and it was just called Police Park where they programmed it into the system to type target police stations. So, that's not very sophisticated if you ask me, if they're using that for, if we're at the United States in militaries making mistakes like that, that doesn't give me optimism that we're that far down the road as far as AI advancement. - So, the tool specific tool they used, we don't know how it was configured. I think we were just creating lists of possible targets and then somebody approved it. That's all AI did. It wasn't doing deep research and each target trying to estimate casualties or anything like that. So, you can always misuse a tool, that's very human, but at least at the end of the day, human was deciding at start of war, let's end the war, let's shoot it, we start. - Yeah, what is your view on how AI has been integrated in, at least as far as China, as far as the surveillance that's everywhere and how it's become this total surveillance state that's been, not only is there security cameras everywhere, but it's also social media surveillance, financial surveillance, all that stuff towards a complete dominated control grid. What do you think about that as far as being brought into the United States with AI? And Snowden told us, he's spying on us, your cell phone tells them everything they need to know about you. It's not that different, it's question is, how is that being used by humans in charge? - Mm, yeah, do you think the NSA has, like an advanced version of this already, that they're using to spy on the track, everything that goes in through your phone and all that? - It definitely have capability. I don't know how much data they actually collect and if it's being stored, but we know what we're capable of doing. - Yeah, the flat camera stuff's crazy too. I mean, that stuff's like everyone's freaking out about those flat cameras right now. And I don't know if those things are using AI or not, but I'm sure they are. - Everything is gonna end up being data for some AI system somewhere. - Yeah. - It's too valuable not to use it. - Right, let's take a quick break. I'll use the bathroom real quick, we'll be right back. You know, some people believe that it's futile to resist this stuff and we should just go along with it and maybe merge with it, you know. And Elon talks all about nerling and using this kind of stuff to fix disabilities or, you know, but I think it's like what he said on his, one of his episodes with Joe Rogan, one of his early episodes was that it'll enable us to communicate telepathically, eventually. - What do you think about all that? - So neural link is awesome for disabled. I agree with that. I think as tool integration, again, very cool to have powerful tools, but if you create super intelligence and integrate with it, what are you contributing in that system? Why are you there? Are you just a biological bottleneck? You slow, have terrible memory. Why is the system keeping you as part of a loop? I think you become useless. And if you are useless, you are very explicitly implicitly more pharmacist. - Right, but we would still, it would still be useful to us, right? Like, couldn't we exist simultaneously and still utilize this sort of stuff simultaneously to improve the quality of our lives? Like it's a possible that that we could live in a world with this super intelligence doing its thing and not getting the way, not interfere, and the same thing, like it is an interfere with us, like the same way we live here with apes and monkeys and uncontacted tribes. We're just like a. - We killed most of them almost completely wiped them out. And the only reason we didn't is because we had no reason to. If tomorrow we had to convert the planet and fuel to fly to space, we wouldn't care about them. - Yeah, but we don't, I mean, right, if we get in their way, but like right now, we go out of our way to protect certain species of animals and uncontacted indigenous tribes in the rainforest and stuff like that. We purposely don't mess with them. - We also factory farm billions of cows, abuse them, torture them, and then eat them. - Yeah. - I wouldn't trust our good nature to make sure animals are doing well. - Yeah, but how do you go from us eating biological animals for their meat to, like, how could, what is the worst case scenario? What, how do you think this, if the superintelligence, sentient superintelligence got out of the bag? What would be the scariest doomsday scenario in your mind of how they would deal with us, or what they would use us for? - So people talk about existential risks. It kills us for some reason because it doesn't care about us. It needs to do something else, cool down the planet to improve compute, convert the planet to fuel. We have no idea. And the worst outcome would be suffering risks for whatever reason, we're still around, but our existence is just miserable, kind of like factory farming for animals. They're not dead, but I think many of them wish they were dead. You think they would wish, you think they would want to get vengeance on us, or like, kill us just for the sake of killing us? - Are you talking about superintelligence? - Yeah. - No, no, again, I don't think it's unpurpose doing it, but there could be reasons we cannot comprehend. - Cooling down the planet for the servers does, I can comprehend that, that makes sense. - But that's again, that's things we can comprehend, but I cannot tell you what something that much smarter will do. I'm just saying, if you're not controlling it, you're not deciding, and you're not, in any way, making it like you and care about you, then anything goes. - Have you tried, I mean, I understand what you're saying, like, we're not gonna be able to comprehend it, but like, have you tried to work out any scenarios in your head that you can fathom, that you can rationalize? Like, if you were to put yourself in the shoes of the superintelligence. - I don't think it's-- - You just think it's futile. - I don't think it's possible to do that. I don't think, I can think of what someone, 10% smarter than me might do. If I think long enough, maybe I'll get good estimates, but if it's thousands of times smarter, it's not meaningful. - It's just hard, yeah, it's just very, it seems like-- - I think it's bad enough with existential problems. We don't even have to get into suffering risks to fully appreciate the importance of this problem. - It's existential. - It is existential. - After it kills us, do you, what is the world look like after that? You think it's just server farms everywhere? - Hey, again, so the moment we create superintelligence capable of its recursive self-improvement process, that's the singularity point, technological singularity, which is defined as point beyond which we cannot see what happens. The controlling power in that world is smarter than us. An agent cannot make accurate predictions about a smarter agent. So you keep asking me in different ways, same question. What will superintelligence do to us, for us for the planet after it gets? None of it is predictable in a meaningful way. I can give you random BS stories from my brain. That's not what's gonna happen. Right, I see what you're saying, but I guess that's just the way my brain works. works. We want to understand the world. We want to predict the future. We got really good at finding patterns and out-competed other animals by doing that. But we're in a position where we're creating something more capable than us. Right. So you've written over what, 300 papers, books, papers, articles, what is the most recent? It's a great question. You want to pull up my Google Scholar account, we can see. You don't remember? How long ago was it? It's been a while, maybe a couple months now. A couple months? Again, I don't remember exactly. I keep working on multiple papers. If you're sorted by year, what are you working on right now? Limits to detecting whatever something is a fake or not, something I generated or not. So basically telling if something is real. Okay. So if I give you an image, right, can you tell in general if it's real or not? And the argument is it's another impossibility result. The argument is what again? The argument is that it's impossible. It's impossible. I have a number of impossibility papers. It's impossible to predict what they're going to do, impossible to control them, impossible to explain how they actually work internally. And that's another impossibility result. You cannot in general tell if something is generated or authentic without some secondary source of information telling you. Just from the image, you should not be able to. So here's a paper of the latest 1-1-26 testing obedience and control. This is about irrational obedience. How do you prove that you are obedient to a smarter agent or a boss? And the argument is if I do things which are rational, you don't know if I'm doing them because I'm rational or because you told me to. So if you tell your kids what's your hands? And they watch their hands? Are they listening to you? Are they just smart people who watch their hands? You have no idea. But if you give them an order which makes no sense, like bang your head against the wall twice and Tuesday. And they do it, they do it because you told them. And so the paper analyzes how can you show or how can I show that it is in fact a BD into us? Right. How can we? How we done it? Well, that's a. I would imagine this would be like a daily test. They would be doing it in. Right. So you're getting in some of it. So frequency is very important. You have to constantly verify that you are still a BD and you didn't have a treacherous turn and now you turned at me. Yeah. And they are doing this. That's my research. No one's doing it. No one's doing it at all. Well, I hope some of the reads and citations at zero so far, new paper. So I suggest somebody reads it. Which one this is the top one you're looking at. And so when you're writing this and you're doing your research on it, who are you? Are you reaching out to people at these companies and asking these questions and they're giving you these answers? Like they're being pretty open with you about this? I'm not directly communicating with the industry. I collaborate with people in academia and other researchers, but usually people at the companies are somewhat limited in what they can do, publicly say, or even publish. Exactly. If they're limited in what they're allowed to say, how do you know if they're not for sure if they're not doing this? I knew if you're not talking to them, like if you're not like doing an investigative work on this. So again, we don't know what happens internally, but based on what they release and publish, I haven't seen. You would imagine that if you I would imagine if I was this company, that would be something I would want to make public if I was doing it, right? You'd want to let the good stuff out. Like if we're constantly making sure every day that this thing is remaining obedient, that's something I wouldn't want to be a headline. Different companies have different publishing policies, like Apple usually is very secretive. They wouldn't publish a lot. Maybe Open AI tries to publish a little more. So it really depends on the company too. Yeah. I guess you'd have the issues of depending on what your public, you probably don't want to publish much because you want your competitors to get a lot of company censorship. I know multiple people who quit the company so they can speak freely, so they can do research freely, including Jeffrey Hinton, who was Nobel Prize winning creator of machine learning. So if he couldn't speak freely at Google, then what chance do many of them have? Where he worked for, what part of Google did he work for? He was doing the deep mind stuff, doing the deep mind. I don't want to make up an answer, so we're going to look it up, but he worked for them and he quit to have some freedom to speak about safety concerns. Right. He's the Godfather of. He's the Godfather, yeah, yeah, yeah. He got Nobel Prize for it and during his Nobel Prize acceptance speech, he basically talked about a safety and how dangerous those things are. It seems like a lot of the folks that are high up in these companies are like very concerned about living forever too, you know. As we should be. It's good not today. They seem very concerned. They're qualified as in this transhumanist group, except Elon. Elon doesn't want to leave forever. I think he was against that. Oh, really? It's a bad thing, yeah. But everyone else, yes. But a lot of the other ones do. Yeah. That's very strange. And they have these, I don't know if you've seen them, they have these cryogenic facilities where they are going to have their brains frozen and like in carbonite. Maybe a whole body. Yeah. Sometimes the whole body, sometimes they just get their heads severed and put in there. Well, usually that's if you can't afford the whole body. If you can't afford the whole body. That's a discounted option. I think if just the head, it's like 250 grand or something like that. We had a guy in here once who he was wearing a dog tag. I know what you're talking about. I think just the head is like 50 and the whole body is like 250, but maybe inflation happened. Yeah. It's gone down. I guess they're trying to get customers thousand for full body. The thing is, most people don't pay for it. You just get life insurance. You pay your monthly premium and it covers it. Right. Yeah. Exactly. A very good deal. I'll be. Sign up. There seems like a scam though, because like if there was some sort of an issue and the power grid went down or there were some sort of like global war. It's liquid nitrogen. It doesn't need power to sustain the cold. Really? Yeah. Alternatively, if you don't want cold there now, other types of preservation like involvement and other things, which are less likely to work, but you have options. If the power grid goes down, those things are going to stay cold for a long time. Whoa. I didn't know that. And they have backups and this is like somewhat important. Yeah. The guy. The guy that was on the show who was telling me about this. He says he believes that when humanity falls, if it falls or after he dies long into the future, when super intelligence will come revive him and hope it's a benevolent one. Put his head in a cyborg. That's the danger, right? You don't know what world you want to wake up in. Right. Right. And do you want to wake up if it's a fucking hell world? Exactly. Maybe bring you back to torture you. Who knows? Yeah. Oh my God. That's dark. Yeah, these people, they seem to be projecting some of their worst fears into their work and to their companies and what they're trying to do. Just their own fear of death seems to be fueling a lot of the stuff that they do. Whether it be stuff like Palantir and the AI stuff and the surveillance stuff and all of that. I was volunteering a fear of death. Well, Peter Teal in particular seems to be like very hyper fixated on living a long time and his own health and that kind of stuff. And, you know, he's also like simultaneously pushing this whole like Christian narrative and Silicon Valley, trying to paint everything Silicon Valley is doing under this Christian lens. There's this new big church there. And which is like counterintuitive to Christian. It doesn't make sense. Like, why would you want to live forever, forever if you're like a real Christian? Like, does it? It doesn't really does not come to read the Bible. That doesn't really compute, right? Yeah. So, and if you listen to some of his interviews and read between the lines and some of the philosophers that he reads and some of the stuff that he talks about, it seems like, you know, maybe if you were him, this is kind of like him. And then also not to mention him buying that huge giant like country size piece of land in Argentina, I think it was. He seems to be like very concerned for his own safety and his own longevity kind of outside of the rest of civilization, right? And all this stuff seems to be some sort of protective mechanism. That's kind of like, that's what I mean by that. But how is it connected to volunteer? Because that's all the stuff that Palantir is doing, like they're doing AI and they're doing like surveillance stuff. And, uh, I just don't understand the connection to longevity. So buying land, they get security I get. I mean, he wants to live. He wants to be protected. He doesn't want to die, right? So you got to make sure that everyone's on their best behavior. You got to make sure you, you would then think you'd have a company which is doing cryopreservation or something related to what you're describing. Palantir seems to be a spying company, a spying company or weapons company, an AI company doing all kinds of crazy stuff like that. I'm just saying I don't see a direct connection as well. - Maybe not like a medical connection, but for sure a safety connection, right? And a security connection, like he wants security and he wants safety. And maybe I don't know what he's doing like medical wise or like trying to, I mean, I know for a fact in the Epstein files, he posted a list of what he couldn't couldn't eat for lunch. And it was like a crazy laundry list of items that he would eat and would need because, you know, for his own health, which is like a crazy thing to do. Maybe if you're a billionaire, it's normal, I don't know. - Most people have weird diets. I cannot judge them for that. - Yeah, yeah, most people do, you're right. But I don't know, it just doesn't, I don't understand how some of these people's minds work. 'Cause at the end of the day, they are just human, they are just human beings. And they, some of them have families, some of them don't, most of them probably don't. And it's just like, I wish you could just lock one of them in a room with you for a couple of hours and really figure out how their minds work and what their motivations are. And. - Having all of them together in a room would be a very good step forward. I think we can make a deal. - Have you tried to do that? - Not directly, but there are organizations dedicated to their safety, which in communication with them, I just had Max Tagmark and my podcast, and we talked about that, he's in good relationship with all of them, so there's some efforts. - Who, out of all of them, who are you most optimistic about? , I'm sure you're right. - In terms of actually not being open to what you're talking about, being open to. - I think they all hinted that they are open to external pressure to make a deal, they cannot do it unilaterally because they're afraid that competitors will continue. But if all of them pause and they get to preserve their high status economic opportunity, I think all of them would sign. - And what is Ray Kurzweil's. Ray Kurzweil has a very rosy color glasses view of all this stuff, right? - In his book, he has a chapter about the dangers of his technology. So he considered it, but he's more optimistic about merging with technology, becoming technology, mind uploading, things like that. - Mind uploading, right? And he's another one that seems to be very obsessed with living forever. Takes like a bucket full of vitamins every day. - Yes. - And stuff like that. - Amen. - It's just interesting. It's just an interesting pattern. - I don't think that's the biggest problem is when you take 100 independent medications which have not been tested together, you don't know how they interact. - So maybe vitamin D is wonderful for you, but then you add 100 hour substances, things are complex. And I haven't seen visually at least amazing life preserving effects of this technology. - Yeah. Well, one of the crazy side effects of this technology, which I mean, isn't even that, it's not even like super advanced technology, just the fact that we have computers and most people work at sitting down and computers all day inside. It has made us extremely deficient in so many different things. Like a lot of us are inside all day long staring at these blueless screens, not getting the available sunlight. We are awake at weird times, not following the circadian balance. And that is severely limiting our lifespans and it's increasing all kinds of comorbidities inside of human beings like cancers or at all time high. All, you know, I think this country, like the most developed countries that where people are doing these types of jobs seem to be, there seems to be a correlation with health outcomes with that, right? Like the third world countries people outside all day doing manual labor, they seem to be the most healthy. So just like high level view, you know, you can see how technology severely changes the human body and the human biology. But you know, at the same time, we're like also super adaptable, like you said, not just cognitively, but physically, we're very physically adaptable and resilient things. And if we are going to keep this stuff in a box, not let it take over and kill us all and just use it, it's we're certainly going to evolve into something that's, you know, way different than what we are or what we used to be. I would imagine. Well, if we develop useful tools for, for example, understanding human genome, we can make direct improvements to our design. We can live longer by design. We can be healthier by design. We can gain less weight. We can have more muscle. We can make those changes. Once you understand human genome, it's a software program. You can directly change certain variables in it. Yeah, that's crazy. We've been doing it indirectly. Every time you pick a more beautiful woman, you're making a genetic choice, right? Yeah. Yeah, the fact that we can create crisper babies and design, people can design their own babies and like pick from a menu, like what's your eye color, what's the hair color, what's the skin complexion? But those things are completely unimportant. What do you care about? Well, no, we care about the cosmetics. What shouldn't be? But we'll have ability to pick important things. Not having a cancer gene is important. Of course. But it is true that, you know, for the most part, we only do care about the cosmetics and the cosmetics. Those are just very easy to judge. Intelligence is based on hundreds of genes interacting, whereas I think things like eye color or gender are much easier to spot and fix. That is interesting, though. How, you know, obviously with the advancement of this stuff that we will be able to do that, if it becomes like a legal thing, but I think right now it's completely illegal to do that kind of like genetic modification and stuff like that. This is the stuff that like Epstein was into. Epstein was, he was emailing NSA cryptographers about manipulating the human DNA. Did you see that? No, I didn't really do any of that. Yeah, he was talking to NSA hackers, like hackers that worked inside the NSA to figure out how to manipulate the, like, DNA and the human genome because he was interested in, like, printing, cloning himself. Seems like wrong kind of specialist for that project. Seems like what? Wrong professional for that type of work. Well, no, I mean, if you have cryptographers, didn't you just say that, like, that's what the DNA is? Like, it's a code. It's a code, but it's not encrypted, right? We just, so cryptography is usually about making something hard to read. So you find the story about the NSA, about Epstein contacting NSA. Yeah, that's my area of expertise, sorry. Yeah, but it was interesting though, why he was talking to the NSA computer hackers though, right? Like, why would he be talking to computer hackers about this stuff? Meet emails from the Department of Justice. Oh, associate's a signal intelligence used by the government agencies to apply breaking the DNA code and the protein signal problems. Oh, okay. Interesting. He asked contact help him find and recruit NSA quality code theorists and code breakers from intelligence services stating that breaking foreign codes is the expertise of US NSA, the concept Epstein viewed the human genome as a cryptographic puzzle and frequently discussed eugenics, transhumanism, and altering human traits. I've heard of it. Thank you for educating me. Yeah, of course. Does that make sense to you? I don't know if it's exactly the same as the cryptographic puzzles cryptographers work with. But I mean, the word code appears in both. So I guess you made a connection with DNA. It's more about genotype versus phenotype and trying to understand how the code translates into physical manifestation. So this makes you taller. This makes you. This is not exactly the symbol manipulation of cryptographic protocols, but I was a smart people with computers all figured out. Right. Is there anything else we should discuss that we haven't covered? Yeah. Many interesting things. What do you want to talk about? I don't know. You tell me anything that you can think of that I'm not asking. Have you heard people talk about it? I'd say it's here. I'm a show before. Hey, I safety. No, you're the first. Oh, really? Yeah. OK, well, do you kind of agree with what I'm saying? Do you understand what I'm saying? - Yeah, for the most part, yeah. - Okay, is there something you disagree with? Maybe we can clarify it for the audience. So with you, if there's any questions-- - No, nothing I disagree with, not really. I'm just trying to mainly get, let's just a better high level understanding of it. I understand that it's like super dangerous, but like I'm just curious about like any more specific, specific kind of like research you're doing drilling down into stuff, like what, like that paper that you were just talking about, that you were working on with seeing if people could detect a fake picture versus a real picture. - So it's called Generative AI. It's so good at creating pictures, text, videos, everything. Virtual worlds, new molecules. If AI generates all those things, we are bombarded by so much of that artificial generated content. If I go on social media soon, majority of stories, majority of posts are all written by AI. How can I tell? The existing detectors are terrible. They're just not reliable at all. You can't say I run it through the detector. I know it's 100% AI. They don't work like that. So is that a limitation of today's technology? Is it in theory and principle impossible to tell? And that's what I'm researching. I think it's an impossibility to actually be able to distinguish. The way it works is you have one algorithm generating fakes and enough algorithm trying to detect. But eventually they meet in the middle, it's 50/50. If there is something I can use to tell it's a fake image, now this counterfeiter algorithm knows how I did it and it improves the faking process. So at the end, maybe it's not the case today, but eventually they just end up being equal in power. And so it's impossible when you see a political video, you don't know if president actually did those things or it's a fake video. Right. And it cuts both ways. If somebody has a horrible video about you, you can say it's a deep fake. Right. But once I see it, it's hard to unsee it, right? Especially the night before the election happens. That's funny. You're teaching it to be more efficient. It is exactly the process. Right. Yeah, it's like a feedback loop. Like the rising tide ray raises all boats. You're doing it forever, yeah? Yeah. And then with just the overwhelming amount of stuff happening on social media, not only is there just shit happening constantly around the world into so many different headlines on every single thing, but you're throwing in fake stuff on top of that and fake people, fake accounts on X that are reporting on stuff. And that's on top of just having so much more real news happening. Exactly. If before they had an AI breakthrough once a decade, then it was once a year, once a month. Now it's five times a day, something new happens. I travel here to do this interview. I come back. I have to read 400 new papers. Right. How are you going to have time to do it? Right. And you weren't like the human brain wasn't meant to keep track of all that stuff, right? Like we evolved to-- Even with help, even with AI now summarizing a paper for me, it's still just too much. Do you use AI often to help you with stuff like that? I use AI tools for everything I can, yeah. Really? Absolutely. Why wouldn't I? Which ones do you use? Again, I think they all are very similar in power. So it really depends on what's available. Some have a little less restrictions for different tasks. So I'm happy with most of them. The one of the things that just frightens me the most is how it's being integrated into social media, just because that's something I'm ideal with all the time. With the podcast, I see it happening all the time. And I see people that I'm close to going crazy, because they don't know. They just believe everything they see. And they don't really have a good AI bullshit detector. I like to think that I have a good AI bullshit detector, but it's probably-- But is it based on a common sense context information of the actual image of video you see? Because that's about to change. The quality is going to be so high, you have to have some other knowledge to be able to do. Right. Right. And especially when you're getting it through trusted sources, too, who are probably some of the big news organizations are probably buying into some of this stuff as well, and not vetting it enough, at least at some point, they're not going to be able to tell the difference. And they're going to have to just report on stuff, because they rely on headlines and viewership and stuff like that. So what kind of implications will that have? Politically, geopolitically, culturally, locally? I'm actually surprised it's not a bigger part of our election process where we have to deal with so many fake stories and videos. Are election process? Yeah. So the night before the election, I would expect explosion of fake videos. But it's not happening. Yeah. Well, maybe the next election will happen. We'll see. So the problem with that is you have people who suspect or are hyperparinoid about everything being fake, right? So if they see something that is real, they're automatically, oh, no, that's fake. That's fake. That's bullshit. There's no way that's real, because they're so-- they're constantly beating over the head with fake stuff all the time. They finally see the real shit. And they think that that's fake, too. And that's also a problem. We can no longer know what's through. Yeah, man, it's hard to have an optimistic outlook on all of the stuff. But-- We're still alive. Let's not make bad decisions. Let's not build general superintelligence. It takes very little work to not do something. It's easy. Thanks very-- Let's just not do it. Let's create useful tools. There's lots of problems. Humanity is facing. Let's work on those. Yeah, but that's what humans do, though, right? That's our nature. We always want to create new shit, more advanced stuff, more cutting edge stuff. And it's kind of like going against our very nature to stop doing that, right? We have very good at following personal self-interest. If I know it's bad for me, I'm not going to do it. Yeah, yeah, but like we talked about earlier, you and I know it's bad for us, but we have no vote. And they know it's bad for them. They all on the record are saying, this is dangerous, this is going to kill everyone. All of them said it. They have p-dume levels, which are insanely high. Unless they're dumb. Unless they are destructive, like think of a terrorist group who cares nothing about-- I mean, hell, they'll kill themselves, right? Yeah. They are believers in rewards in the afterlife. This people are all atheists, as far as they know. Except for Peter Teal. He is not a cutting edge AI developer. Really? I don't think what they produce is talented-- I thought Palantir had a pretty legit AI. I think what they have is a very well-integrated model with military and security apparatus, but I don't think they have a cutting edge model yet. Again, maybe my knowledge is limited and what they do internally. Did you see the reports of when-- Do you remember when the whole doggy thing happened with Elon? Where he was like when he was still working with Trump? Doge. Doge-- I like calling it doggy. When he got in there and he was trying to clean house and all those interagency's like the IRS and all that stuff, there were stories out there that reporting that he had a contract with Palantir AI to integrate that into the HHS, the IRS, and all these agencies where he was cutting all these people, cutting, getting rid of all these people, and trying to merge all the data in all of these organizations like the IRS and the HHS, merge all of it and consolidate it all into this Palantir AI to where it was just like one database that had like a very high deep level of information on all the citizens of the US. I would guess he would be pushing XAI instead. That's what I would've thought to somebody else's company. I would've guessed that too, but I guess Palantir is very deeply intertwined with the government. And I don't watch news. I'm very happy about it. So I can't really-- Well, that just makes me skeptical about the Palantir AI wonder how powerful it really is. So usually, when you get rankings of top AI models, they don't show up at the top for most tasks I care about. Really? I'm curious. Google those things. See if you can find Steve. Power of Google. See if you can find about that story if it was legit. So this is actually something you can do with today's AI. It sits in the back, listens to the conversation, and pulls up relevant stuff way faster than a human could. Really? Absolutely. That's easy. Voice recognition. Transcribe. Search. Oh, wow. Recent. That doesn't take super intelligence. Go up to the top. Oh, wait. Yeah, OK. Palantir has been discussed in connection with the Department of Government Efficiency Doge, Environmental Data and Software Overhauls alongside CEO Alex Carp's push for enterprise AI control and data sovereignty. The IRS plans reports indicate that Doge's representatives have appointed to Palantir as a prospective vendor for data management projects as part of federal efficiency initiatives. Palantir co-founded by Teal and Associate of Musk trying to firm closely into discussions surrounding Doge's federal techs, CEO, Carp criticized major frontier AI labs like OpenAI and Anthrophic during the earnings call, arguing that they attempt to colonize enterprises and absorb corporate intellectual properties into their training data. Okay, so yeah, you probably are right then. They're just trying to consolidate data and this is more this is more of like uh controlled police state types fears. I think they very don't do earth, they kind of like we have data sets, we want to understand them, we want to combine them, I think that's what they work on, but again, yeah, they do internally. We can talk about it till we're blue in the face, or at least me, maybe not you maybe. The scary part is no one is in control, no one understands what's happening, how it works, and no one has power to stop it. Any one of those CEO's unilaterally cannot stop, they'll be replaced immediately. So we kind of we have this process where we're losing control already, even before we build super intelligence. Some people talk about like if we could have some super intelligence, if some super intelligence was created, it would want to, if it didn't want to survive, it would want to ensure the survival of the planet. It would want to make sure that this planet can sustain. So that would be an overall good, a net good. So even if humanity had to pay a small price for the planet to be safe, like maybe this thing would, if this super intelligence made all the decisions for humanity and for the world, it would be in the greater benefit of earth and for nature and all that stuff, because it would want to preserve that stuff. Wouldn't it wipe humans as polluters and just preserve the planet? Maybe not. Maybe you would find us useful in some in some sense. Yeah. In the world, with super intelligence in it, what are you contributing? Don't you, like don't you, some people would claim that eliminating all the humans would maybe cause problems for them. Maybe they would need us for certain things, certain tasks. I would love to know what those are. We can tell people what jobs to get. Like we can be the slaves to the, you know, not a very efficient slave. If we have, you know, humanoid robots, if we have nano technology, I don't think you are contributing much. Right. Well, Roman, thank you so much for doing this, man. This isn't right. You've blown me away and scared the shit out of me. That's not Bill General super intelligence. I agree. Let's not do it. Tell people about your books, about your podcast, where they can find you and learn more about you, all that stuff. Yeah. We have a podcast. It's called The Roman Forum. It's Romanian Polsky. And we talk about AI safety with leaders in that field. If you want to read about the concerns I have, there is a book, AI, unexplainable and predictable and controllable. It addresses all the limitations of existing safety mechanisms. Fantastic man. I'll make sure we leave all that stuff below. Thank you. Thanks again. I really enjoyed this. It was fun. All right. Good night everybody. I'm going to talk about Shopify. I'm going to talk about the "Shopify Piste.com" episode. I'd like to talk about the world's largest online community in the past two or three years. I'm going to talk about Shopify, because I want to talk about the technology of the world. I want to talk about the future of the industry and the front-end, and the future of the world to the future. If Shopify had been the main source of the industry, it would have been the same as the Polsky. That's how it works and we can also talk about the "Shopify" in the future. I'm going to talk about the "Shopify Piste.com" episode.