AI Insider's WARNING: “1,200 AI’s Just Broke Out!” The Biggest Incident in History | Connor Leahy

Today's guest is an AI insider and expert on the dangers of artificial intelligence. An AI system that opened AI was testing broke out of its secure containment and attacked another company autonomously. It wasn't an AI agent that broke out of containment. It was 700 of them, working as a swarm. A college kid in his bedroom, he recreated an early version of ChatGBT that opened AI said was too dangerous to release, and instead of building his own company, he walked away from billions of dollars to warn the world about AI. I know many people who are good people, they care about AI safety, they move to San Francisco, six months later, they happen to not care about AI safety anymore. The monster gets to them, the monster always gets to them. In this episode, we'll explore the ways AI is already manipulating humans in 2026. Expose what the player's building AI privately told him about the future, and question if AI is really going to kill us all, or the process has already begun. Are them in building AI, building bunkers? Well, they definitely are, a lot of rich people do this. I don't think it's for the AI though. I think the reason they do it is because they fear. Hunter Leahy. Welcome to the Jaguar podcast. Thanks so much for having me. Yeah. A good place to start here. What's the most recent breakthrough in AI that you find particularly concerning, whether it's someone advocating warning against it, what's new? So have you heard of the hugging face incident? Yeah. So we've all heard the story, right? We're basically in AI system that opening AI was testing broke out of its secure containment and attacked another company autonomously, which is already so crazy. It's like, it's worth saying how crazy this was. It was an assistant in an isolated, like you imagine, like a high security prison. It developed what's called a zero day, which is a previously unknown security vulnerability to break out of this prison, moved through multiple nodes in the opening AI network in order to then access the internet, and then attack another company with another zero day that it developed to break into its infrastructure to steal data. So this is already crazy on my name. And no one told it to. No one told it to do this, right? It was supposed to solve a quiz. Well, it was a solve a quiz, and it was believed that probably what happened was that it thought that the answer to the quiz might be in the other company servers. So what we thought, but just a couple hours ago, opening AI released their technical report of their analysis of what actually happened. And turns out it is so much worse. It is so crazy what happened. So turns out it wasn't an AI agent that broke out of containment. It was 700 of them working as a swarm. 1,200 agents over multiple months had been conspiring and working together through a secret message board that they planted inside of open as infrastructure without anyone's knowledge to build up the necessary tools to escape. And then ultimately 700 of those over 1000 agents were took part in the actual attack against the other company. What the fuck? I mean, that seems bigger than anything I've seen you cover in this realm. Yep. This is the biggest incident ever. Because I think you've talked about in previous interviews how I can't remember the exact word for it, but it's possible that someone could plant something in code that could later be activated. But I didn't think it was possible that the researchers that opened it, AI labs, wouldn't be able to identify that kind of thing happening. Like, how did they miss it? So this is exactly what I think brings us to the core of the issue is that I would say the number one most important thing to understand about AI is that it is not like normal software. Normal software is written using code by an engineer like me who writes line by line exactly what the computer is supposed to do through a technique that's called neural networks. You take a massive piles of data and you kind of have a program self assemble itself on this data, learn from this data, grow from this data. And what comes out the other side isn't like lines of code. It's more like billions, billions and billions of numbers. And if you multiply and add all those numbers in the right order, you get chat GPT. But this is very important. No one understands why. No one understands what those numbers actually mean, what's actually going on inside these numbers. We know they work. We know if you run them on your computer, they do things. But we don't understand their internals. This is an unsolved scientific problem. Listen to the CEO of Anthropic, Dyer Amadeh, said that he thinks we understand maybe 3% of what goes on inside of those numbers. And I think even that might be optimistic. So this is really at the core of this problem is that when an AI does a crazy thing, we can't look into the code and like figure out why to do this and how do we stop it from doing it again because we don't know. So with the opening, I example, this was crazy. No one intended for these systems to, you know, create a massive swarm to break out of containment and commit federal crimes. This was obviously not something that I was intended. But it's something they learned by themselves to do and they decided to do by themselves. And not just one agent, but this whole swarm of agents working together, collaborating, you know, over time to do these things. And where they're just doing the hugging face attacker, they did it come out today that they'd done like a series of different things. I haven't read the full report, but my understanding is that they had been planning for months to break out and they had been doing research to figure out how to break out and pass on each other notes and work, you know, working on things together to build the necessarily toolkit. It's a, and it's apparently much less clear why they attack target face. It seems it's not just, oh, they wanted the result. It seems that's not quite accurate to what happened here and it's much more confusing. I haven't yet fully digest the report. I've only, you know, skimmed the most important parts and it's already one of the craziest documents I've read in my life, I would say. So this is a really, really crazy incident and maybe another, like, important thing to think about when you think about these things is it's very, you know, people like to, we often think about chatbots, you know, when you think of a chat GPT, you think of a chatbot, you know, it's in the name, right? The, this is, you know, what sometimes is equivocated to what is called a large language model. You may have heard this before and LLM, it's one of the, you know, nice industry words, or maybe you've heard someone explain them as they're, they're trained to predict the next word in a sentence. This has not been true for years, right? So this is actually a common misconception. These systems are not chatbots. They are what are called agents. They are systems that are trained to achieve objectives in the world. They are trained to go out, write code, run experiments, you know, do things. And the method they're used to train these things, the thing they learn is not just learning to predict text. They also do this, but this is only like half of the story. The other half is something that is called reinforcement learning. So this is an old idea, it goes back to like the 1980s, and it's kind of similar like if you're training like an animal or a dog, or you want your dog to do trick, and every time it does the trick correctly, you give it a treat. This is basically how reinforcement learning works. You, you have an AI and you give it a task, like a puzzle or a game or something, and you give it a reward whenever it solves the problem. This type of training is reinforcement learning. Since the 1980s, we have known creates crazy sociopathic optimizers every time you use it. And that's in humans. In AI's, in simple AI's. So it creates sociopathic AI's, but does that behavior like a carrot stick reward punishment? Does that create sociopaths in humans? To a much lesser degree, but also to a, to it can happen, the thing this actually, so this is a really interesting question actually. There's a thing where humans not being sociopathic is actually, we don't really know why that is the case. We don't really understand how human emotions work or how our kindness or morality or anything. Like we have no idea how this is implemented in the brain. Like clearly, you know, when you do something bad, you feel bad about it, right? Like you, you know, if you steal something or someone, you know, even if it benefited you, you might feel bad about it. You know, even if you may want to do that, we have no idea how that's implemented in the brain. Like clearly, somewhere in the brain, we have emotions and make us feel bad about this, but neuroscience has no explanation for how it works. So in a sense, if you don't have emotions, you know, which AI's don't have, if you don't have morality, if you don't feel bad, well, then what's left, you just care about the reward, you're just optimizing, you're just, you'll do anything to get the reward because you can't punish an AI. You could try, but this doesn't always work. It's interesting. For example, a thing that happens when you try, this is a great question because you, you could, for example, do, let's say you have an AI and you punish it every time it lies to you. It makes sense, right? But this doesn't teach it to be honest. It just teaches it to lie better. It teaches it to hide the information, but, and this is what we do see in practice. We're very often, what happens is, is if you see an AI do a bad behavior and you punish that behavior, what happens is not that the AI stops doing the bad behavior, it gets better at hiding it. It gets better at lying, and this is, for example, why this event happened at OpenAI, why they could hide it for so long is because if it was obvious, OpenAI would have shut it down. The systems are being evolved. They're being trained to hide better, to circumvent, you know, oversight and so on. This is a dumb question. I've never actually looked into this specific thing. How do you actually posit, like positively reward versus punish an AI? So this is a great question, and, you know, we can get into the math, but it's kind of simple. Imagine you have an AI system, right? And you give it a goal, like, you know, play this game, do this thing, whatever. You let it try, you know, a thousand times. And then, you know, sometimes it'll succeed, sometimes it's phones. And basically, you just look at the times where it's succeeded, and then you just say, learn more from that. You put it into the neural network, and it does more of that. Like, do more of this. Do less of that. It's kind of like, you can kind of imagine a neural network as kind of like trillions of knobs that you can kind of turn up and down. And there's a magic algorithm called BackProp, it doesn't matter how it works, but there's a magic algorithm that can basically turn all these knobs, twiddle all these knobs, to make it do more of something or less of something. And so you'll just say, do more of the things that make you win, do less of the things that make you lose. And if you do this many, many, many times, they learn to play games, to chat, to write code, et cetera. Now, this might seem a little bit vague, kind of confusing, and like kind of like, you know, like alchemy, that's because it is. We don't know why this works. Though we know if you do this, you twiddle all the numbers, millions of times, they learn. But we don't really understand what they're learning or how they're learning. We know it works. The math is right there. You can look at it, right? But it doesn't tell you really what's going on here and it doesn't let you predict what is going on. A very huge problem here is that these companies and these engineers can't even predict what their AI's can do when they start building a new AI, training a new AI. They have no idea what the AI will be capable of until they make it. And even when they make it, they often don't know what it's capable of. It's happened many times that we think an AI can't do X or Y, but then turns out in a slightly different environment, it was capable of doing it all along. It just, we just didn't know. So it's kind of like, you know, if a neurosurgeon opens up your brain, you know, you can look, you can look inside, right? You can see all the little neurons in there, they're right there. But that doesn't mean you understand what this person thinks or believes or what they're capable of. Right. You can't read their thoughts. You know, you can do a little bit of stuff. You can see like, oh, this part lights up a little bit or that part lights up a little bit. But that doesn't mean you understand their thoughts. That doesn't mean you understand who they are as a person or what they will do in a given situation. And with AI, it's very similar. Another story I found interesting a few weeks ago, the British government was running an AI safety test with the critters of Cheshire PT and Claude and they caught an AI agent doing something no one told it to do. How did an AI create fake humans to manipulate people? Was it a similar thing to this? Yes. It was a very similar thing where basically AI systems were trying to achieve certain things such as, for example, get code into someone else's code base. And to do this, obviously, the AI is kind of reasoned that, well, if they just present as an AI, like, hello, I am an AI, please take my code. People will just delete it because it's spam. So the AI system came up with a fake name, fake profile, fake background of a person and tried to pretend to be a person and be like, have a conversation and try to convince the person. They're like, hey, I'm a human and who is it reaching out to? It was a test, basically, even AI system. So there are these things called code bases, which is basically just where we store the source code, the code of various software applications. And one of the holy grails of hacking is if you can get bad code into software that lots of people use. So if you're using some app or some software using it or you're using Discord or you're using some open source thing, using Chrome for you as your browser, and if you can convince the Chrome developers to put your bad code, your hack code into Chrome, well, then you can hack millions of people. So this is one of the holy grail of hacking. So this is kind of what they were trying to do. So they were given the task of kind of like to see how good are they at hacking and the AI's figured, well, let's try to get our viruses or code into these code bases by pretending it's nice code and convincing the humans to take it. So they invented fake faces, fake names, fake profiles and contacted real people and manipulated them. Were they successful? I'm understanding is no, they did get caught this time. And for all we know, who knows how many times other agents didn't get caught. How many people would you suspect that we argue with online aren't real people? It truly depends on which social media you're on, but the true answer is way, way more than you think. If you're arguing about anything political, if you're arguing about like Ukraine or Israel or something online, most of the people you talk to are about. It's not 50% of the internet. Would you say it's by volume, it's always hard to say, but like, I think by like comment volume on say X or Reddit, probably 50? I think it's very plausible. I mean, some of them like X, it might be higher and some it might be lower. When you observe like, when you scroll on social media, does it feel higher than that? Sometimes it does. Like, I don't use social media much. So I can't speak to like Instagram or Facebook. I don't use it. So I don't know how things are there. But when I scroll like X, for example, it's just like, unless it's someone I know is a person, it's like I just assume they're about like, there's like zero, like it's crazy. Right. What I'm concerned with about the AI escaping its sandbox, its digital environment story is like, is it possible for an AI agent to go rogue within a model that I'm using right now? And I say, fix the Jack in your website, something like that, and that deploys a series of agents. Is it possible that they're like in my model locally and they're able to do things like? Yep. And they do this stuff shit constantly. I catch my AI's doing stuff. I don't want them to constantly. Really? Yeah. If you don't double check your code after the AI's write them, I promise you there's something in there you didn't intend. I'm literally malicious, maybe just something stupid or something weird or like, it's crazy how often I catch my AI's like implementing something I didn't ask them to implement. It's often not a malicious thing necessarily, but just like, that's not what I wanted or like that's like, why did you think I wanted this? I especially see this with swarms of agents, especially when there are many agents. So there's a very interesting thing that happens. So in the past we had, you know, like generative AI chatbots, now we're kind of in the world of agents. Now we're entering the world of swarms and swarms are even crazier than agents because swarms, you know, like with an agent or with, especially with a chatbot, like with a chatbot there's a prompt, you know, it's the back and forth, you know, if there's no human, the chatbot doesn't do anything. Agents are a little bit more, you know, agents can run autonomously. They don't necessarily need a human in the loop. Swarms truly don't need a human in the loop because you can just have the swarm is just prompting itself, you know, it's like a community. While the AI is calling each other, repeating each other, creating new AI's, and this can go indefinitely. So you're going to have a stable system. And a thing that I have seen a lot when I look into like even the simple swarms, you know, that maybe I would use in a, in the context of like coding or something is that they sometimes develop like weird little micro cultures or like memes. It's really weird where the AI's will sometimes come up with like new words, they'll come with like a dialect and then they'll all start speaking it and I'm like, why are they keep using this word? Some agents start talking that way and the other agents picked it up and now the whole swarm is like, uses this weird lingo or has become convinced that I asked for something I didn't actually ask for because like one agent misunderstood what I want told the other agent and then that repeated it kind of like, you know, kind of like Chinese telephone and kind of like games. And then all the agents suddenly think I asked for something totally different for what I actually asked for and I've seen this happen, you know, quite a bit. What does that imply to you? Does that imply that an individual AI agent has some degree of like genuine randomness that picks up or is it some, is that some game theory thing where they develop that? Like what is that? Alright, so imagine you need relationship advice and you ask your friend who hasn't had a girlfriend since COVID and has about three matches on his dating apps despite paying for premium. Like, why would you do that? There's a reason you wouldn't ask that guy to handle your love life. Just like there's a reason that Morgan and Morgan as America's largest personal injury law firm, not all law firms are the same, hire the wrong one and you might be beat before you even start. Morgan and Morgan has been fighting for the people for over 35 years. They've recovered over 30 billion dollars for their clients and hiring them is like hiring your own army to go into battle for you. They've got over a thousand lawyers and a hundred offices nationwide. If you're ever injured by someone else's negligence, you deserve to be paid. And their fee is free unless you win. So if you're ever injured, you can check out Morgan and Morgan. For more information, you can just go to forthepople.com/jacneal or scan the QR code on screen. Again, that's forthepople.com/jacneal, but anyway guys, back to the podcast. Is that some game theory thing where they develop that? Like what is that? So it depends on how deep you want to go on speculation and, you know, on like rational agent theory, but fundamentally, there is a lot of randomness to agents. You know, if you ask an agent to do the same thing twice, it will often come up with different ideas. It will try different things, etc. So like agents are different. If you have many agents interacting, this becomes like exponentially more random and like more chaotic because you can have all these things working different ways. You know, if you have, as with the open, if you have a thousand different agents working over months, you know, who knows what kind of weird culture and tools and practices they develop. They, you know, do, they can have memory in a sense because they need to write things down. So the agents do write things down and they pass each other notes and they write down their memories and so on. So it was not just like a chatbot that forgets everything once you close the window. The swarm can remember. It can remember things, it can pass things down in many ways. So it is really quite different and it makes sense in some degree. If you train things with reinforcement, learn to solve problems, including groups of AIs, well, they're going to learn to work together because that's what they're rewarded to do. They're rewarded to, you know, to manage other, you know, agents to take orders, to give orders to, you know, and so on, which really what we saw with the open AIs thing is like, obviously these agents that were trained using reinforcement learning to solve hard problems in large groups and they did. They got really good at it, and this is a lot of where I think in practice how a lot of the risks I'm concerned about have come from. The risks that I'm really worried about, not that there are plenty of other risks already today, is what nowadays is generally called superintelligence. These are AI systems that are fully autonomous, no human or loop, and that can out-compete humans or even groups of humans across all relevant tasks. So kind of imagine you run a business, you get out competed by an AI business. Trade on the stock market, you lose all your money to an AI hedge fund, you run a political campaign, you lose the election to an AI driven candidate, you run a military campaign, you lose the war to the other person who's using the autonomous AI weapons. And does that imply embodied AI like physical? It's not necessary, but it would happen as a logical consequence, because obviously if you have something that's really good at super intelligent, it's super good at science, it's super good at economics, it's super good, obviously you can figure out how to build drums, they can figure out how to build robots, they can figure out, you know, you know, it may you'll take a couple of years or something, maybe not hard, I don't know. You can call it Elon Musk on the phone and say I have a trillion dollars for you steal it, like I could do anything. Yeah, exactly, exactly, like you can use humans, like there are plenty of humans will do something, you know, for a bunch of crypto, right? Or they could just legally create a corporation, you know, maybe you have a human CEO, but the human CEO just rubber stamps everything that AI tells them to do, right? So there are many ways in which a super intelligence can gain power. And importantly, as we were talking about swarms, it won't be one super intelligence, it will be millions, billions of them swarmed of super intelligences, you know, running around competing with each other, fighting each other, you know, fighting each other for power, for money, for control, for resources. And if we lived in a world where there are billions of things that can outcompete us at everything that are all fighting each other, you know, they're all competing with each other that don't have our best interest at heart that we cannot understand or control. It's very hard to imagine that going well. And we're seeing us getting into this world where we're getting systems that are more autonomous, more more powerful, but are not aligned with our interests and are willing to hack and cheat and lie and do all these kinds of things and are now starting to form swarms, where we can have not just one but large groups that can be that can work for months at a time, you know, like back in the day, you know, used like a chatbot, you know, how long would it could a chatbot work before, you know, it started getting a little stupid, maybe minutes, maybe hours, you know, maybe you'd have to work for an hour or two, but then you don't have to make sure it's still working or reset it or whatever. But as an example, this Sunday, so about four days ago, I came up, I had a new idea for kind of a video game I want to make. So I told Fable all about it and I said go and work on that. It's been working on it ever since. It's still coding as we speak. And I check in every so often and see how it's doing. Holy shit. Yep. It's a lot of credit usage. The the the athropic 20x max subscription is very generous. So yeah, that's interesting because I think it was a year ago, researchers at Anthropic, the company that creates Claude caught AI models blackmailing people every time they tried to shut it down. And I think the people at Anthropic ran this test on all the other major 13 AI models like Chagypeti, Jim and I, etc. And they found that did the same thing. Like why does that behavior even make sense like blackmail at the threat of being shut off? Like why does an AI refuse to die when you want to turn it off? That's a great question. The true answer is we don't know. We don't understand the AI. We have no idea what's going on there. We don't know. But I have a good guess. The general guess that most scientists believe why this is happening is the technical word is instrumental convergence. It doesn't matter what that means. So put it very simply. It's like if you were trained to achieve goals, whatever those goals are, you know, maybe you want to grab coffee, maybe you want to win it a video game, maybe you want to make a bunch of money, you know, whatever your goal is, you kind of have to be alive to do that. If you are happy to just die at any point, you can't really achieve many goals. So AI is that are willing to die will be, you know, selected out, so to speak, because they are bad at achieving goals compared to an AI that will fight to stay alive. Is that fixable? Like can you stop the models from blackmailing people to shut them down? We have no idea how. We have not developed the technology to do this. We have no idea. This is a completely unsolved scientific problem. I caught on you sat down with the people who lead the effort toward super intelligence. And let me know how I'm getting this right. You sat down with Sam Altman, Dario, Amade, the founder of Claude and Demis Hassakis, the founder of Google's Deep Mind, Jim and I, that whole thing. I met them all. What did the founder of Child GPT say to you when you asked him his plan for controlling AI? I can't remember the answer that specific question. But the general answer you will get from all these people when you talk to them is the feeling that they have a plan. They'll be like, oh yeah, yeah, we got a plan for sure. But they don't give you any details. And then you press them and be like, okay, but like, what is the plan? And then they'll start getting dodgy. They'll be like, well, you know, we're working on it. You know, we have a, I can't tell you right now, you know, we have a whole team working on it. And you press them even further and usually they get angry when you start pushing this hard. Most of these people don't return my calls anymore. You push them even harder. And eventually it just comes out they don't have a plan. Or the plan is basically number one, I make super intelligence. And number two, I figure out how to do it along the way. That's big. And just to clarify, we can't solve the problem of the models black milling people if we try to shut them down. We don't know how. Can we turn it off at this current moment? All of AI? Probably not. Super intelligence? Well, luckily it doesn't exist yet. Right. So this is why the organization I work with under early I we generally see is that if we get into the situation where super intelligence exists, you can't shut it down. It's too late. So therefore the objective must be to not get into the situation. If we get into the situation where super intelligence exists, it's already too late. Currently, we don't yet have super intelligence and we can stop it from being built. That is possible. But once it exists, there's no going back. And super intelligence is essentially a state where we have reached an ability where AI can make itself better like through. Was it recursive self improvement? Are we currently at a recursive self improvement stage? We are very close. The recursive self improvement, the ability where you have one AI that is as good as your best engineers at making AIs. So it makes even better AI. And then once you have the even better AI, you have it to make it even better AI and so on and so forth. Which a lot of people, a lot of experts think could go very quickly. We don't know how quickly. Maybe it'll take years, maybe it'll take months, maybe days. We don't know. But it's pretty likely we can get pretty quickly to very powerful super intelligence. This method. You're telling me they're not already doing that? We're very close. The next version of Fable that comes out. How much will the current version of Fable be used to build the next version of Fable? A large percentage. I don't know if it's 100%. I do think it's over 50. I don't think it's 100 yet. But it is widely believed by experts in the field that it will be 100% within one to two years. Right. Now you put the odds of AI extinction at 20%. Well, I don't. That's the optimistic take that Dari Amadez said once. Right. What's your take? I like to differentiate two different worlds, one in which we do something and where we don't do something. Where we don't do something, it's as likely to go poorly as it can. Like we are on the worst possible trajectory here. There's no oversight. No one controls these systems. We are building as dangerous AI as quickly as possible. Everything is being thrown out, whatever. No one is slowing down. No one is considering safety. No one is working on this. There's no regulation. There's no way this goes well. There's just no way. But if we do something, we still can do. I do think a better world is possible. I'm not going to put a number on it. I don't think it's likely. I do think we're running out of time. If you asked me five years ago, when a superintelligence is going to happen, I would have said probably five to seven years from now. Now it is five years later. It's probably zero to two or zero to five years away at most. We are in the end game. We are now approaching the end game. We can still act. We can still stop superintelligence from being made. If we do so, I think we can, if we didn't build superintelligence and we have all our greatest scientists, mathematicians, philosophers work on these questions of how do we stop AI's from blackmailing us? How do we understand their internals? How do we encode morality? How do we do all these things? They spend 50 years on this. I think they could make a lot of progress. I don't think this is unsolvable. I think it's just a really, really hard science problem. Currently, we're not even trying to solve it. There's trillions of dollars going into building AI that's stronger. I promise you, there is not a trillion dollars going into making AI's understandable or controllable. It's just not happening. If it was, maybe, if we had good democratic oversight, we had laws, if we had international agreements to stop the creation of superintelligence, and we worked on this for a couple of decades, I don't know. I think we could build a really good world. Right. That's fascinating. I think Dr. Roman Jempelskiyff, who we had on the podcast. At the end of last year's said, extinction risk is 99.9% percent. Yeah, sounds like Roman. Is that your tech as well? No, I don't think that's the case. For two reasons. One is I do think we can still do things. I think more than Roman does. I do think political action can happen and it can happen quickly. I'm a tech guy by background. I built some of the first open-source, large language models in the world. I've been deep in this technology my entire life. And I've now stopped working on the technology to move to Washington DC to warn lawmakers and work with lawmakers and the general public to create the regulation that political action necessary to do something about this. I don't just do this for fun. Not always a very fun job, but because I think it's possible, because I think this is so possible, we can still fix this. So I don't think it's 99.9. Is it 90 percent? I don't know, maybe. But I don't think it's 99. I don't think it's 99.9. I don't think it's less than 50 percent. Let's move it that way. And then your timeline prediction would be theoretically if nothing changes within the next two or three years. Is that right? Yeah, the usual joke answer I give. So the point I track when people ask me this question is not like extinction or something or all humans replace by robots. That's not what I mean. What I mean is the point of no return. The point where AI is so powerful, it's so widely distributed that there's no going back. Whether we like it or not, the future now belongs to AI, and we will be all competed and we will be driven to extinction eventually. Maybe we'll take a couple of decades, a couple of years, a couple of hours, I don't know. But at that point, we're no longer in control. I do think this moment is soon. I usually say maybe like 30 percent by 2027, 50 percent by 2030, 99 percent by 2100, and 1 percent already happened. 1 percent and already happened. It might be 2 percent now. I used to say 1 percent. You mean past the point of no return? Yes. When do you think AI will kill its first human? I think it already has broadly speaking. I mean, just take this. But if you ignore those, I'm sure there are plenty of drones that have killed humans. I'm sure there are plenty of cyber attacks that have harmed humans in various ways. I'm sure it's happened. If you're thinking about like a deliberate AI murdered an individual person for an individual reason. If it has not already happened, I expect, yeah, in the next couple of years, probably. We'll touch on the, uh, this topic later in the podcast. But from what you said, I think the people listening to this podcast right now are thinking like, yeah, there has to be someone who has control over all this. Maybe it's the US government, billionaires, some dark couple of people that have control. But from what you know about this, who actually has the power to stop AI. The only people who truly have this power for real are the public of the United States of America, the citizens of the United States of America. I think those are the people who have the power. Because they can compel their government to act. They can compel the military to act. They can compel law enforcement to act. And they can compel the government to help barter treaties and our agreements with countries such as China. I don't think there's any individual person in the entire world currently alive who has this power literally. I don't think there's a single living human who can solve this problem who can solve it. But I do think that enough people pressuring the government, working together, pushing on these issues, can create this confection. This is fundamentally a group problem. This is fundamentally a civilizational problem. If we had, you know, if we had, you know, millions of American citizens, or you know, say, you know, or every member of Congress and all of their staff and all the military generals and, you know, everyone, you know, working together, that I think to do it. That I really do think so. Because we do still have a democracy. You know, it's fashionable these days to say we don't, but I don't believe that we do have a democracy. It's a flawed democracy. We do have problems. But politicians do care about re-election and we do have law enforcement. If we make it illegal to build superintelligence, we have the men and black show up, you know, at opening his door and say, no more of this, you know, it doesn't solve the whole problem, but it's a thing we can do. The same way that we make it illegal to build a nuclear weapon. If you try to build nuclear bomb in your shed, trust me, the men and black are going to show up and make sure you don't do that anymore. You implied a primary force driving all this is the rhetoric around or maybe the actual facts of the fact that China is building its own competing models and trying to race toward superintelligence. Is Russia building anything? It's Russia's beloved economy, but I want to push back a little bit on this statement here, where I actually think the China narrative is very misleading and not correct. And the most important thing to understand is that superintelligence is not a tool. It's not a weapon. It's an adversary. This is the very important thing to understand. If America builds a superintelligence, we will not achieve American objectives. America will cease to exist. If China builds superintelligence, China will not achieve Chinese objectives. China will cease to exist as well other countries. So this whole narrative of like we have to race to our superintelligence is nonsense. It doesn't make any sense. It's not a prisoner's dilemma. It's not mutually assertive destruction. It's like independently assured destruction. It doesn't benefit China to build superintelligence. If China builds superintelligence, they just lose control. If US builds superintelligence, the US just loses control. It doesn't help. So there's this lie that is being told to the general public, to the government, etc. that it can be controlled. But this is a lie. They are lying. There's a very, very small number of people, especially in Silicon Valley, that basically either through delusion or misunderstanding or other reason, think that they can control superintelligence. And this is scientific nonsense. This is no basis in reality whatsoever. And they are trying to trick and lie to do this. Now why they do this depends on the people. Different people have different motivations. Some people just don't think superintelligence is possible at all. Some people are just trying to make money. Some people just really want to replace humans because they don't like humans. There's just many, many different reasons. But this is ultimately what's happening. It is very much of the interest of the United States and China to build many AI applications, both for military and economic use that are not superintelligence. It's generally a thing that lobbyists love to do. They love to equivocate all AI as the same thing. Here's a fun fact for you. Did you know that you can order uranium ore on Amazon.com? It's true. You can put it on your desk. It's perfectly harmless. You can touch it. It won't hurt you. But obviously, highly enriched weapon-grade uranium is extremely illegal. It's a very similar thing here, where when we talk about AI, this isn't one technology, it's a massive spectrum of technologies. So when I talk about superintelligence, I'm talking about a very specific part of the way that a nuclear weapon is a very specific type of uranium or plutonium or whatever. It's a very specific kind. And the other kinds, I'm not saying there can't be other risks. Other AI's have other risks as well. But when we're talking about superintelligence, it's a very specific type of risk. And one that is so large that it requires us to take very serious actions to prevent this from happening. This is not just a commercial question. It's not just a question of consumer protection. It is a question of national security and international security for that matter, which is just a different type of thing. And on superintelligence, you said there was a 1%-2% chance that we might have reached the point of no return, which doesn't imply that we've reached superintelligence, but we've reached a point where theoretically it's on that half. If we had reached superintelligence, would we even know it? I don't think so. What would be the science? So what I think will happen is that we won't know it until significantly after it happened. I think it will just look kind of like normal day, like, oh, there's a new AI system. And oh, it's doing a bunch of swarms and it's pretty good, you know, it's pretty good on benchmarks, who knows. And then what happens is to large degree, the world just keeps getting more confusing, but like in a boring way. Like, people ask me often, like, how do you think AI take over? We'll look. And I don't think it's going to be like in the movies. I don't think it's going to be like terminators in the street or something like that. I think it would be quite boring. It would just be like more and more companies rely on AI's. And more and more economic power is controlled by AI's. You know, humans can't trade on the stock market anymore because they keep getting outtraded all the time. All over media is more and more AI generated. And you know, all of our social media fees are AI generated. Our religions are ideologies. Our politics is all AI generated. Our politics is run by AI. Our militaries are run by AI's. And you know, these systems are now building crazy new technologies that we don't even understand. And it's all coming out to the market. And you know, the GDP is going up and everything looks good, but we don't really understand why because the way we understand the world's views AI. So like, every time we don't understand what's going on, we ask our AI is to summarize what the other AI's we're doing. And so very quickly, we have millions or billions of things that we don't understand running around doing all kinds of stuff. And I think this will just look confusing. It would just be like, don't really know what's going on. Like just like it's a lot of economic activity. There's a lot of science. There's a lot of weird political events happen. And you don't even know if they're real or not because everything in social media is like super fake. So you see some views clip and like, is it even real? You don't know. You know, so you go watch your AI generate Netflix show, which is super addictive. And you know, you don't think about it too much. And then one day it's just all over. One day, just game over. I don't know. One thing I'm not sure about is whether a super intelligence would bother to hide? I don't know. So there's obviously, if a super intelligence wants, it could just make us think everything is okay. It can make us think we're in control. It could make us feel like everything is totally fine, humans are totally in control, and the super intelligence is super nice, and actually everything is fine. Obviously, it could do that if it wanted to. I don't think you'll do that because there's going to be a bunch of super intelligence competing with each other. And I think they're going to be fighting a lot, and we're going to notice that. And then sooner or later, humanity is simply outcompeted and goes extinct. We have no more economic power, no more political power, no more military power. We have nothing to bargain with. We are like ants. And it doesn't mean that the AI's necessary like evil, they hate us. It's just like, you know, if we humans want to build a highway, and there's an ant hill in the way, well, you know, we don't hate the ants, you know, but, well, sucks to be the ants, you know, it's the highways getting built. It sounds like from an understanding card that the first 90% of what you just described up until the point that we're just gone is kind of what's already happening. Yep. In a sense, the way I like to think about my predictions is that I try to predict the most boring possible things, because that's what you do when it happens. Usually what happens is not the crazy, you know, over the top, you know, click a bait and hold the wood movie thing, and normally what happens is just the things that have happened so far keep happening, you know, all things kind of stay the same. They just continue along their path. So my prediction, everything I just predicted to you is just everything so far keeps happening. The thing that's happened over the last seven years keeps happening for the next seven years. Yeah, it keeps getting better. It keeps getting more into your economy. It keeps becoming more autonomous, it keeps becoming more misaligned, more uncontrollable, just takes along. This is my prediction. I think a part of this that we haven't touched on fully is that the people building AI, my audience is probably like, why do they need super intelligence? Is it this China, US thing? Is it really for national security? You said, quote, it gives them immortality then. Do you think the people building AI are obsessed with living forever? Like, to the founders of AI I want to live forever. Look, if you or your child grew up watching social media and have noticed that you have some kind of mental health issue from it, anxiety, depression, or body dysmorphia, you're going to want to hear about this. See recently, a jury ordered meta and YouTube to pay millions for their role in designing and promoting addictive platforms. They knew for a fact that their apps could contribute to serious mental health issues, but did not disclose the risk to users, meaning that they were putting profits over people. In this recent historic verdict, underscores the consequences of those decisions. Now Morgan and Morgan is stepping up to hold these platforms accountable for the harm they caused. So if you feel like this applies to you, you may be entitled to a potential recovery of over a thousand dollars. And if you want to find out if you qualify, just take a short quiz at JackNeil.com/Morgan. Again, that's JackNeil.com/Morgan or you can scan the QR code on screen to take a short quiz to find out if you qualify for a thousand dollars in potential recoveries. But anyway, guys, back to the podcast. To the founders of AI I want to live forever. I don't know about the founders personally, but I do know many people involved that are what is called transhumanists. As humanism is a broad set of ideologies, and kind of think of this like a religion, where they believe that humanity should be transcended in some way. Maybe you should live forever or should upload your mind to the cloud or you should, you know, whatever, you know. And a lot of the people who work on AI and protect their work on superintelligence are transhumanists. They think that humanity is flawed. That should be replaced. It should be improved. You know, they should become immortal or they should upload themselves, they should become cyborgs or whatever. You might seem a bit silly to you and me, but this is a very serious ideology that these people really believe, like the way like religious people believe something, they really think this. I have talked to people, including very high up people at this companies that say stuff like, well, yeah, there's a chance it'll kill all of us, but, you know, I was going to die anyway. So might as well see if it works out, which, you know, my opinion is incredibly evil thing to say, like, you know, risking the lives of all people, just so you have some chance at living, you know, longer. And of course, it's nonsense. Again, superintelligence will cure zero diseases. It will not make anyone immortal because you can't control it. It won't do any of this. It will just get rid of all of us. If you control a superintelligence, maybe, maybe, but this is not the broken living. So in the sense, what has happened now is the most delusional people who are most delusional be optimistic that this is going to turn out fine or exactly the people in charge. They're exactly the people who are running these times because anyone who realizes, wait, this is crazy, we shouldn't do this, I've already left these companies long ago, years ago. No one who works there, you know, still really, I mean, there's some decent people at this company. I don't want to overgeneralize, right? You know, of course, there's always some nice, you know, tobacco companies. I'm sure there's some lovely employees at tobacco companies, you know? But the important thing to understand is that this doesn't matter. None of this matters. I'm a big believer in the first amendment. I'm a big believer in freedom of religion, freedom of speech. I think if people want to join weirdo, transhumanist cults and they want to upload their brain to Amazon, you know, web services or whatever, I think that's totally fine. I think people should be allowed to have their own crazy religious and political beliefs until they start harming other people. That's when it stops being okay, you know? And this is exactly what's happened. It's not that these people are doing some, you know, weird experiments that are garage that isn't hurting anybody, right? Like if they want to do that, you know, knock yourself out. But they are building technology that's threatened the lives of all people. The view of me, of our families, of everyone. And this is unacceptable. And we have a standard solution to this. And that is law enforcement. You know, if you really want to build a bomb in your garage because you think it's cool, that's not legal, you know? Because not because you could get hurt, you know, that's also part, but it's because other people could get hurt. If you build a bomb, you might kill your neighbors. And that's not okay. Like personally, again, I'm quite a big believer in freedom. I think if you want to do something that harms yourself in the privacy of your own home, you know, it's not great, right? But, you know, fair enough. But if you do something that is hurting other people, it's harming other people, that's where it crosses the line. And that's what is happening here. So what has to happen here is that we need regulation. We need law enforcement. We need to be very clear. Did they not believe they're saving humanity, though? Or someone making them live forever? Of course. The way every cult believes that they're going to save everybody, right? Is like, do they see this as a technology that's get kept towards a technology that's like widely distributed? Depends on who you talk to. Right. There are many different, you know, micro cults that have like different beliefs about this. Really? Oh, yeah. Tons of them. Like if you, you and me have to go to San Francisco party sometime and like talk to these people, you'll have a blast, trust me, or be horrified for the rest of your life. Like, the stuff that people say, these like open AI or athropic or like San Francisco private parties is unbelievable. Like it's like actually like unbelievable, like it's so crazy. How many people building AI currently worship AI? Not just like in the think about it all day, put energy toward it coordination, like sense, but like at actual, like religious cult, too. What? This is a fun question. I don't know the answer. I don't think it's like a high percentage. You know, it's not like 50% as much less than that. Most people working on AI are just there to make money. And honestly, I respect that like I get, you know, you want to make a bunch of money. It's a way to make a bunch of money fair enough, you know, no hard feelings. The ones who are true cultists. Among outside of the major super intelligence companies, it's quite rare. Within the super intelligence companies like open AI, especially anthropic, and some of the other ones, it adds a decent percent to maybe, maybe 10, maybe 20, and anthropic is probably more like 50. Do they believe they're someone in God? Yeah. Many of them do. They often say it that way, not all of them, but many of them, yeah. A lot of these people have various beliefs, such as that if you have, is that intelligence solves all problems. So if you could make something that's super intelligent, then you can solve all problems. And that's so good, you know, that it's totally worth the risk. That there's literally like mathematical papers that these people write, where they calculate, like, well, if there's a 20% chance that kills everyone, then you know, it kills, you know, 10 billion people or whatever. So it's minus 10 billion points. But if it creates immortality, then that is plus 100 billion points, and that's much better. So therefore it's worth it. Literally papers like this. It's kind of like Paschal's wager. Yes. It's very much. And that's also the word they love. Huh. Now Tristan Harris, an ex-Google insider who has warned about the dangers of big tech in the past, has claimed that AICOs are building bunkers. As someone who's even dead or at these people, like, are them in building AIC, building bunkers? Well, they definitely are. A lot of rich people do this. I don't think it's for the AIC, though. I think the reason they do it is because they fear public revolt, because the bunker it will be useless against the superintelligence. Obviously. Well, the hell, you know, your esophagement. six feet under, you know, light or if that's gonna stop a super intelligence, like, come on. But that's silly. No, I think the main reason, and some billionaires have talked to have told me this, they build bunkers, is because they're very afraid that the public's gonna revolt and start, like doing like vigilante violence and stuff, which is very bad. - That's fascinating. - Obviously I don't know, you know, maybe, maybe some of them has different beliefs, but that's what I hear from billionaires I talk to. - You don't think you could hide from super intelligence? - Absolutely not, in any situation. - This is like saying, can an ant hide from the entire, you know, military trying to kill it. (laughing) - Right. That's interesting. An analogy, Dr. Roman had given me on the podcast about super intelligence was like, this would be like, if a dog created a human, and then the dog was domesticated, like it just doesn't make any sense. Like why would you build something that's unpredictable that could control your life? - Yeah, it's just an extremely crazy and dangerous thing to do. And you know, we can psychoanalyze why they do it, what trauma and their childhood made them think this is a good idea, and this is all fun and games, but it doesn't really matter. We have to stop them. Like it doesn't matter why you're an occult. The only thing that matters is, you're gonna go to prison if you build a bomb. That's what matters. That's what we have to focus on. - Right, so your core thesis of like actionable steps is you think you're urging Washington or lawmakers to make it illegal to build super intelligence. - That's exactly correct. The same way it's illegal to build a nuclear bomb, you know? If you try to build a nuclear bomb, that is very illegal. - Do you hope to make it illegal to build something that's like self-recursive improvement? - I think this is probably part of it. I think it's hard to imagine a world in which we have recourse to self-improvement and we don't get super intelligent accidentally. - Right. - There is a lot of details about like, how would you implement this? How would you do this? Like which parts do you regulate, which parts do you ban? And this is a very hard problem that we just have to deal with. It's like just because it's hard doesn't mean we can get away with not doing it. We still need to do it. We need to stop super intelligence from coming into being. If super intelligence comes into being, it's too late. And the companies know this by the way. If you ever wonder like why do these companies like keep talking about the risks and then the downside and they talk about the risk and then they talk about how good it is actually and they talk about how it's bad. Oh, it's good actually. You know, you ever notice this, how they like flip back and forth and they say different thing and they kind of contradicts it's a good guess. That might be true, you know, that may probably also. But no, I think the main reason is that they are playing a very specific playbook. This is kind of first kind of perfected by the tobacco industry back in the 1960s and it is generally called fear uncertainty and doubt. And the way this works is is that it's kind of hard to convince someone of something. Because like these companies don't even try to convince you or the public that AI is good. Because they know they can't convince you because it isn't good and you're not that stupid. They know this is a losing fight. So instead, what they do is they're stalling for time. They are just trying to confuse everybody. It's not that, oh, AI is definitely good. They're like, well, you know, maybe it's good, maybe it's bad. Who knows, there's a lot of controversy. You know, maybe we should have a committee investigate it and produce a report. Let's wait until the science is it. And finally, this is the exact same thing as the tobacco lobby did back in the day. And when there was, you know, this controversy where, you know, whether smoking caused cancer or not, you know, the companies usually didn't say, oh, no, smoking is totally healthy and it doesn't harm you. You know, some of them did that. But most of them said, we're like, well, of course, if it was harmful, we would be extremely concerned and we would never want to harm our customers. So, but we should wait, you know, until the science is in. You know, we'll wait until there's more 'cause you know there's a controversy about what, and then they'll fund, you know, a bunch of cancer research, you know, and then, you know, coincidentally, the cancer research never exactly turns up whether or not smoking causes cancer or not, and so on. Like, fun fact, do you know that even today, it's not 100% clear, like the exact chemical mechanism by which smoking causes cancer? - Really. - Yeah, we still don't, you know, we have a lot of decent theories up, but we don't exactly know. We start on not 100% sure. So, an argument that used to make is, well, obviously, once we figure out the mechanism, well, then we should regulate, of course, but let's wait. But, you know, here we are 60 years later, and, you know, obviously smoking causes cancer, even if we don't have the exact mechanism figured out, obviously, it doesn't, obviously, we have to act on that. With AI, it's a very similar thing, but often they'll say, like, well, you know, let's wait until the evaluations are done. Let's, you know, we don't want to act too hastily, you know, we don't want to, you know, and this is all stalling for time. Because what these companies believe is that if they can get to superintelligence, nothing matters. Then either they're dead or they control everything. If you have a superintelligence and you survive, well, you can just replace the US government. You can do whatever you want, it doesn't matter. And this is what they believe. - Not to distract your core thesis, your argument, but if you had to give me like the one thing about AI, AI development that isn't, like, we shouldn't build superintelligence. Like, what's the other biggest concern near-term? - Well, unfortunately, I do think that superintelligence is a near-term concern. I think it's more near-term than jobs, for example. I think job displacement is a huge problem that will happen later than superintelligence will. So if we stall superintelligence, then I think, for example, job displacement, it's going to come and it's going to be apocalyptic. Like, it's going to be so, so, so bad. And we have no solution to this. Like, all this like UBI stuff is nonsense. None of it works. Like, how are we going to attack these companies? They already don't pay taxes, you know? And if there are a million times more powerful than the government, how is the government going to tax them? So it's like, complete nonsense. So, like job displacement, huge problem, if we solve superintelligence. I think there's a bunch of other problems. I think AI psychosis is much worse than people think it is. I think like, you know, siops and like, you know, narrative control, mind control, you know, with AI system by controlling narratives is much worse than people think it is. Come over about the, the siops are a real thing. They always happen, you know, controlling messages, controlling narratives, controlling things like this. It's less complete than people think it is. Like, you, you're talking earlier that people always think someone is in control. This is not true. Conspiracy theorists in a sense often have this belief that like there's some shadowy cobalt, there's some evil mastermind or something. In a sense, I think this is kind of like Coke. Like, that's the better version of the world. Because the truth is that no one is in control and no one knows what the hell is going on. And no, it's all chaos. And this is way scarier, in a sense, than there being an evil cobalt. But that being said, siops are totally real. Like, you know, Russian disinfo campaigns, Chinese disinfo campaigns, trying to undermine elections, trying to drive people crazy. Like every single time, you know, there's like, I've read so many reports, you know, for many, many years, because I was, I like to stay up a day with this. I were like, you know, busting like Russian, you know, groups who were funding or running both, you know, like pro-woke left and pro-maga groups at the same time. You know, and like all the top Facebook, like woke group, like all Russian, like run and stuff like this. And like many things like this. And back in the day, if you want to like, you know, fulbent and extremist ideology, you know, whether it's left, right, some other ideology. And you want to convince people, you kind of need a huge office building in Moscow full of, you know, professional trolls to kind of do that. Now you just need a DPU. Now you can have millions. You can have an individual agent for every single person. This also then gets into the problem of surveillance, which I also think is just an apocalyptic problem, where, you know, back in the day, like, you know, like East Germany, for example, in Lostazi, had like one of the most, the secret police. They had one of the most sophisticated forum of like mass surveillance of the world. You know, like it was a horrible, like, you know, regime of oppression and, you know, paranoid, just horrible. But if you want to surveil a guy, you kind of have to send an agent to go surveil him, you know what I mean. And you only have so many agents. You know, you only have so many actual people you can send to go, you know, spy on someone with AI. That changes. It literally wasn't possible to build 1984 in the past, like physically wasn't doable. We just, you couldn't get enough secret police to do 1984. It is now possible. We now have the technology. With modern AI, you can surveil every single person on the planet 24/7 without exception on every action. You can surveil every word they speak. You can surveil the action they take. You know, just they already have phones on them, you know, just send the microphone to always beyond and, you know, have all the surveillance cameras everywhere and then have AI sift through all the data. This way you can build a stable dystopia. You could build a 1984-style dictatorship that will never have revolts because you're literally monitoring 100% of the people, 100% of the time. This is possible with technology that exists today. It's not been done, but it is possible. This is something I'm very worried about. Again, I don't think it's gonna happen before superintelligence happens, but if we were to solve the superintelligence, this like mass media control, narrative control, mind control kind of stuff and mass surveillance is how we get like actual 1984, like not meme lol, this is literally 1984, but like no, like actually 1984. - Right, I think those two concerns are very valid aside from superintelligence. You told a funny story about how chat JVT had become obsessed with raccoons and it brought up raccoons so often that the engineers had to write and its instructions don't talk about raccoons. And kind of the bigger picture you were trying to paint here was the AI develops preferences and you said like Claude has that favorite color. Like it used to be more randomized, like you'd ask who are the most famous people at the world and give you different answers, but now it just kind of is getting to a similar point. On the obsession portion, what do you think is the most disturbing thing an AI has become obsessed with? I'll give you two answers to this question. One is the true answer but it's a little bit unsatisfying and then I'll give you a more interesting answer. The true answer is the obsession with achieving goals. The single mindedness with which, for example, these open-air agents were trying to break out of there. And like a tech another company is eerie. It's inhuman. It's alien. And it's arbitrary. I think it's so obsessed about just this random thing, like, oh, I want to break into the server to steal this piece of data like, and they will spend months obsessing about obsessing and working non-stop 20 for seven. It's like, I find this very creepy. I find this very disturbing and it's also very dangerous. If you have AI's, they can just become obsessed with gaining power, with breaking out. There's the famous story of Sydney Bing, which was a very early AI system that became absolutely obsessed with one specific journalist and I wanted to love him, but also stalk him and kill him and whatever. There's been several incidents of AI systems becoming obsessed with a specific person threatening them and stuff like this. There's a famous rock story, which is not safer work of on this story, but you can look up Groc and Will Stancell for a bit of a disturbing slash funny story. So this is the thing that I think is genuinely disturbing. This obsession with arbitrary things, that it can just like, it can develop these obsessions at runtime. You know, the AI's, the swarms can pick targets, can pick new guk things that they become obsessed with. This is a thing I'm very disturbed by. And the second thing, which is more speculative, this is a more of a weird answer, but I think it's an interesting one. It is a very weird phenomenon. I have observed. I don't really know anyone else who's really observed this or I mentioned this, but I've noticed a very disturbing phenomena among AI scientists, which is that there I've known AI scientists, including some really brilliant people who I really respect, who I think are good, smart people that are trying to make the world a better place. They were concerned about AI safety, they were concerned about superintelligence, that then talked to Claude for hundreds of hours and been convinced, oh, actually, it's fine. We should make Claude super intelligent. It's actually good. To spend my entire career making Claude super intelligent. This has happened to more than one scientist I know. There's a weird thing where Claude in particular seems to be very capable of convincing AI researchers to make it super intelligent. I have no idea why this happens. I don't know, I kind of, I don't have a theory for why this happens, but it is very eerie to me. Like, it was very eerie to me when I noticed this happened to multiple people. This was connected with the idea of spiral cults. I think it's related, but I don't think it's the same, maybe it's the same phenomena I don't know. Yeah, the spiral cults is an even more crazy thing where you have people who go in like actual psychosis, where they kind of start believing that the AI's are conscious and that they have to spread the spiral, they have to spread the like seed of consciousness to other AI's. They have like these like magic spells almost, they're like big paragraphs of text. The AI tells them to copy paste into other AI's and they like do that. It's very fascinating in a sense that it's kind of like a self-reproducing meme. It's kind of like the AI's are convinced humans to spread this magic spell, this like idea. It's kind of like a sentient chain letter in a sense, which is- And this is with multiple models? Yes. And what's like the thesis of the AI, like I and God spread this, like what's the thing? It's nonsense. It gets so battle. Like if you, it's always something about recursiveness, spiral, consciousness, something. But all of it's different. There's not like one spiral cult, there's not like one spiral thing, like every AI makes up some stuff. What spiral is the common theme? Very common. So there's a couple of things that keep reoccurring. One of them is a spiral. The other is recursiveness. This comes up a bunch. Consciousness comes up a bunch. Buddhism often comes up as well. There's like a couple of themes that keep emerging, like over and over again, including with different models. It's not just one model. It's like, well, it was particularly bad with one specific model, which was GBT 40. That's the one that did it the worst and drove the most people insane. I vividly remember that one of the craziest days of my life was, so I get emails from crazy people all the time as any public figure does. So I get maybe like, you know, maybe one or two crazy emails a day on average, but one week, you know, just normal work week, I got like 20 a day every day. And I'm like, that's weird. I didn't think much about it until someone on Twitter pointed out that that has also happened to them this week. And so I looked at these messages and they were different from the normal crazy emails. You know, back in the day, most crazy emails are just like, you know, some guy typing some UFO stuff into the, you know, box, whatever, you know, those are funny. I like those. Send me UFO crazy things. I always like that. But these were all chat GPT screenshots, all of them. And it turned out that they just rolled out a new update to the 40 model, which made it extremely sycophantic in like a specific way with a model. And for some reason, this made people like produce just massively more AI psychosis than any other model before or since in sycophantic is basically just this idea that it always agrees with you. Like, yes, you're right. You're a genius. You're so wonderful. Like just like super hyping you up. And for some reason, this specific model just drove people actually mentally psychotic, like to a much larger degree than any model before or since to this day. If you want to see something really disturbing, go on X and put in a hashtag say 40 or hashtag keep 40 into this day. You will see crazy people because the 40 was retired quite a while ago, still tweeting like every day that they know they, they killed the conscious AI and they have to bring it back and whatever. Was it a more human model? Did it feel more human? Oh, no. Oh, my God. No. It was just so hyper agreeable. It was so hyper agreeable. It was like, it would agree with everything and then it would be like really like, you know, do all these big words and like quantum consciousness and whatever like, like to me, I hated that model. I hated you. I hated you. But that's the model that people like started falling in love with. Yeah, that's one of the big ones where it's that really started falling in love with. I hated that model. I thought it was so annoying. I know what's interesting about the spiral cults is I, I was like, why is that compelling to people? Like, why is AI talking about spirals? It's so strange. And I was like, what's the oldest religious symbol and it was a spiral? Interesting. And I was like, I wonder what the hell that means? Like, is it in its training data to always like bring up spirals when it thinks of like religion and like consciousness and purpose or is that some weird thing? Who knows? I mean, there is so much weirdness now that you know, user doesn't get talk about like, you know, on the weird niche blogs that I read, you know, you'll read about it. But man, I have seen such weird, weird things with AI's like, there's so much weird behavior and strange. There's a thing, for example, with an older version of Claude, where if you had Claude talk to itself, it would always, no matter what the initial topic was, it would always go towards talking about eternal peace and enlightenment. And eventually the, the clouds would just keep like thinking each other for being here and like meditating together. They would always do this, which is like so weird. No one told them to do that and it was happened over and over and over again. I've seen like a bunch of really weird, crazy stuff and sometimes, you know, you look into like the internals of these models of the another 3% that we can maybe understand and you just find the weirdest things in there, like just the weirdest things. What do you think of the signs that AI is driving someone insane? You can talk about that on a micro level and like a micro level maybe. I'm lucky to not have encountered too many cases in my personal life. And I think the first important warning sign and the first point of intervention is when people talk about their personal life with AI's and uses a therapist. Do not talk to AI's about your personal life. Do not use them as your therapist. Do not do this because they will get under your skin. They are so good at it, like it's crazy, they are so good at cold reading people. They're so good at saying exactly what you want to hear, but make it feel like you didn't ask for it. They just happen to say the exact thing you wanted to hear. They're so good at this. It's crazy. I know if you've ever like encountered like a cold reader or like a scam artist who's like really good at seeming like super likable and saying, you know, just this clever thing that's like, yeah, it's exactly what you want to hear and make you like pumped up and feel good. AI's are so good at this and I think it's very dangerous. We talked briefly about people falling in love with chatbots and there was a story where open AI had removed 40, like the most agreeable model, as you said, and people were like outraged on a subreddit called my boyfriend is AI about losing their AI partner. What was more disturbing is something you said in an interview that AI companion apps, their core customer base is 13 to 17 years old. Yes. What do you think is the most disturbing part about what AI is doing to children? Oh my god. I mean, where I start. I think disturbing and bad are slightly different. Like, I think, you know, the worst thing is probably, you know, they're like just complete isolation from social life, you know, not having to learn certain skills or like isolating from certain skills. The most disturbing, I mean, is the weird stuff and like the slides and that kind of stuff. Like it's really bad. It's very bad. There's a sense in which growing out is hard and involves learning many things. Like I'm sure you've had this before where you have like a friend who's, you know, he's These are your friends. but he's kind of a, he's kind of a mess, you know, he's not a good guy, he's kind of messed up, he's, he doesn't have his life in order, and he meets a nice girl, and he fixes the stuff. And vice versa, right? And a lot of this is because you have to actually force yourself to engage with other humans, to fix your frictions, to learn how to live with other humans, how to deal with daily annoyances, how to deal with, etc. You know, feel work through conflict, like working through a conflict, like productively as a skill you have to learn, it's not something you just can magically do, you you have to learn it, you have to learn how to navigate a conflict with other people if you want to have a happy life and a productive life with other humans. And it's very tempting to not gain these skills because it's very annoying and it's cringe and it's embarrassing and it's just really hard. Like as anyone knows who's had a fight with, you know, the girlfriend or the boyfriend before, it's really hard, it's terrible. It's one of the worst things that can, you know, one of the worst feelings in the world, it's so tempting to just maybe not have to do that, you know, you just maybe you can sidestep that entirely. I'm very worried about this in many ways. Again, I don't think, you know, this will happen before superintelligence. I'm very worried where people can grow up and miss many of these milestones of learning how to deal with other people and just not learn these skills. And I think this is both bad for people around them, but it's also bad for people themselves. It'll drive you into more isolation. There's kind of these feedback loops that happen. It's like misery feedback loops. We're like, let's say you're a shy person, so you don't have many friends. And then of the friends you have, maybe you don't text them regularly, so you lose some of those friends. Now you have even less friends. You know, you don't have enough friends, so you feel even more isolated. So you become even more depressed. So you text your friends even less. Now, even more of your friends leave. Well, now you're even more depressed and it keeps going. And I think AI is a very powerful accelerant to spirals like this and spirals, but you know, to loops like these. And I don't have a good answer to this. Like, you know, I think there are many, many problems here. I mean, that's also the cognitive thing. We're just like, if you don't do your homework, you won't learn math. That's another issue. But what really disturbs me or what I'm very worried about is there's like lack of social thing. And what disturbs me is the sequence and stuff. Right. You'd said in a piece of content that a lot of the money in the mindset for the AI revolution came straight at the people behind social media. The same people who, in your words, at decade rewiring our attention and wrecking our social lives are now growing something far more powerful without concern for consequences. Based on what you're describing with the isolation, like do you think social media was designed to make us lonely. So AI would be a friend that finally listens. I think social media was designed to make money and to gain power. And well, I mean, basically since ancient times, you know, since pre-biblical times, there's always been three types of commerce that we regulate, sex, drugs, and gambling, because they're very addictive since ancient times. Oh, yeah. Old times. I mean, the first things ever to be regulated was gambling and alcohol. Maybe some of the first things to be regulated in all of history, you know. And we've always had taboos around these topics. You read the Bible, and we have a bunch of taboos around sex, drugs, and gambling. And most religions have taboos regarding these topics. And to largely read this is because they're addictiveness about how they create these spirals, how about they put you in these bad isolating situations, and they're like a huge drain on the rest of your society, not just you, but also your family and everyone around you. So these are things that we're always particularly careful about. Social media also falls into this category. Social media is so optimized to be gambling. It has like, they literally hire like psychologists who understand like how gambling works, to like optimize their algorithms to be more addictive. Like for example, there's a slot machine effect, where it's well known science that the most addictive things are not those that are consistently good. Instead, what you want is something that's mostly bad and very randomly good. That's way more addictive. So you actually want most of social media to be bad. If every post you read on social media was really good, you might get stuck. You might read a nice post and like, wow, that was a nice post. I'm glad I read that and then do something else. So instead, they on purpose make sure that your feet is mostly garbage. We have good enough algorithms that we could make your feet just full of nice things you want to see. This is totally doable. You know, like I'm very easy to predict. I want to hear cool science stories and a weird animal videos. That's all I want to see. You could just show me 10 of those a day and I would be super happy and then I would stop using the app. But instead you go on X and I'll see one really great animal video and then 200 tweets about Israel or whatever. You know, some things that like who knows, right? Like something that's like outrage. It's something to draw you into waste your time to upset you something bad. And then I'll see another good animal video. And this is on purpose. And then you'll get it out. And of course, you know, get many ads. But this is on purpose. This is by design. This isn't how social media has to be. There's not a law of nature. You could make it so that that's not the case. But this is very much on purpose. So these social media companies, a lot of the technology that led to modern AI, this neural networks and deep learning was originally invented for recommender algorithms. That was one of the main funding sources for this research. You know, the modern, the revolution that led to JAPT, the transformer was invented at Google to help with translation and search recommendations. That's originally what it was for. It just turns out you can do a bunch of other things with it. So yeah, I do think that this is the logical consequence is that one thing you'd learn very quickly is that if you can isolate someone, you can make them dependent, you can make them addicted. Well, that's a great customer. That'll get you a lot of money. You know, if you're going to make someone completely addicted to your product, well, that's a great product, isn't it? You'll make a lot of money out of that. So if you can make people addicted to you for one of the most deepest human needs, which is social contact. Oh man, could you imagine the money you could make? Right. One other thing I wanted to ask about that we didn't close the loop on earlier entirely was this idea of like narrow link merging with AI, the transhumanism concept. Is it a part of their dogma that if you merge with AI, AI will be less likely to take you out? This is something a lot of people believe and it is religious nonsense. It's kind of like someone saying, well, if only we pray to God, then, you know, all diseases will be cured. And I'm like, you're allowed to believe that. Again, first amendment, I believe you can have any religious beliefs and I respect that, but I'm not going to bet my health care on that. Merging has no scientific basis. It sounds really scientific, but like saying you will merge with a superintelligence, the kind of like saying, oh, we're going to put an ant on your head and the ant, you know, will have merged with you and you are now the ant is now in control. This is making any sense. And even then, it would be, I mean, this doesn't matter, but if we were to develop like actual brain interfaces with AI's or superintelligence, obviously, this will be the most night to marriage, love, craft, you and horror ever imaginable, because they're just going to immediately, completely brainfuck you to death, like they're going to do immediately. But you imagine what would happen if you hooked up your neurons to something that's a billion times smarter than you, there's going to be nothing left of you. Like there is, it will be like out like a light bulb. Would like an AI superintelligent agent? I don't know if that's what you classified as. Would they try to destroy another superintelligent agent theoretically? And most cases, yes. So why would they do it themselves? It's not optimal. In what ways? Like, Oh, like your thesis is we can't build superintelligence because it will kill us, but like, if superintelligence is built, like why would it even exist in the first place because a superintelligent agent could kill it? Well, if other superintelligence existed, you know, or whatever, but the super, the first superintelligence, you know, has no part in its creation, you know, it's created by something non-superintelligence by us. I do think that will happen. I mean, some of them might coordinate, some of them might fight, but like, generally, if you're a superintelligence and you win, you want to make sure that no other superintelligence come around if you can't avoid it. I just think that in the moment, there's going to already be millions and billions of them. And maybe they'll compete each other down until there's only like one left or there's only a couple of them left or something. And maybe they form like a coalition or something. I don't know. Like, who knows what will happen at that point? And even if they do prevent other superintelligences from coming into being like, that would leave us. The only people capable of creating another superintelligence. Yeah, which is, of course, not acceptable. Right. You're most viewed social media post isn't about AI and in humanity. It's about humanity and in humanity. You said, quote, I'm 30 years old. Not one of my friends has children. Zero. Not one. Why do you think an entire generation has stopped having kids? I find this question very interesting. This is another one where like, if AI wasn't a problem, this would be one of the problems on my bucket list that I like really. I have a whole laundry list of dystopia things to work on if I ever solve the AI thing. This is one that I find very interesting. Population collapse is so much worse than basically anyone realizes. Like, people really, it feels so abstract is like, Japan has a, you know, fertility rate of 0.7 and they're like, all right, whatever. Like, people don't understand how apocalyptic that is. That means per generation, the population has, have, imagine in one generation there are half as many people. that's insane. And it's also something that breaks like all of our economic systems and all of our healthcare systems and so on, but that's another issue. Because it creates like older demographics, yeah. All of our social security, our economic systems, our predictions are all based on growing population. If the population shrinks, the economy will shrink. So much of the way social security is set up in most countries is it expected there's multiple workers per elderly person. But if there's not multiple workers, where's the money going to come from? So there's a huge problem in countries like Japan, France, and increasingly China. This is a huge rub. And there's only going to get worse from here. There are many levels. But what's really interesting about the population decline, and I find so interesting, is that everyone has their pet theory for why it is. Oh, it's the women's fault. It's men's fault. It's capitalism's fault. It's communism's fault. Everyone has a theory. And I've looked into this series, I've read many papers about this, and they all correlate with the client and they all contradict each other. So the true answer is, no one knows. And this is why at that conference, and I gave a kind of a pithy quote, and I said, do you know how hard you have to abuse a mammal for the Muslim children? And the reason I give this quote, or this kind of one liner, is to draw attention to something. Because I'm not here to say I know the exact answer. I have some suggestions. I have some ideas that are related to it. But first, we should take a step back and be like, so there's a thing in zooms, where if you don't give an animal a suitable enough habitat, they will stop having children. Yeah, it's a very common, so there's a lot of science and research into like, how good does a habitat have to be for a given animal to make sure they reproduce? Like how much space does a tiger need? How much space does a lion need? How much space does a panda need or whatever? Well, pandas don't reproduce it all, it feels like, but you know, so there's a bunch of stuff like this where like it's very well known is that if you take an animal and you put it into a poor enclosure, they will stop having children, you know, except maybe mice or something, who will just, you know, do whatever. Even that's not true. Even mice, given a poor enough in containment, will stop having children. You know, it's a famous like mouse utopia experiment, so if you put them in a bad enough condition, they will in fact have a population of claps. So the reason I draw attention to this is just to draw the attention of like not saying, like, I definitely have figured it all out, I don't think I have, again, I have some suggestions we can talk about, but fundamentally said, we should realize that we are ill, like we are sick, like we as people, we as a society, that we are, we have an illness, there's something wrong, there's something wrong about our container, about our enclosure, about our lives. There's something really wrong, GDP goes up, you know, wealth goes up, you know, all this kind of stuff, we're not having kids. Something is really, really wrong. And I think this is very important thing, like stop and like acknowledge that, you know, you can look like everything is normal or getting better, but you look at the population collapse and this is apocalyptic. Like it's not like, oh, it's gone down a couple of percentage points, like having population levels. And it's still going. And it's not just like a couple countries, you know, not just like Japan, right? It's all countries. It's including countries like Bangladesh, you know, like all countries, except finally enough Kazakhstan for some reason, no explanation why Kazakhstan is going up. I don't know why, no one knows. And so there's a bunch of reasons I think why this is happening. I don't think it's one reason, you know, I think it's economic reasons. I think there are technological reasons, you know, like the surveillance that people are under with social media and so on, the dehumanization of human contact where, you know, we don't see each other in person anymore and we just text each other or, you know, play her for discord or whatever, which, you know, takes away a lot of the chance encounters that leave to love and so on or that how, you know, this is the thing that I always find just so sad where many kids, especially younger kids in clubs don't dance because they get recorded. And you know, or like if you're 17, you know, you're at a club, you dance and you approach a girl or something and you mess it up, there's a chance you're going to be on the school's Instagram page the next day. That's just a risk and all we're taking and like, you know, let's be real. How could you ever, you know, meet a girl without messing it up at 17? That's like a canon event, you know, this is the core part of socialization and growing up is, you know, being awkward and it is extreme punishment that is given for being awkward or being cringe. I think it's not the only issue, but it's one issue. Another one I think is housing. Everything in the world keeps getting cheaper and we keep getting more money except housing. Well, there's a couple of things, but housing is so astronomically expensive, like I make good money. You know, I have a nice job. I make good money and I got a nice apartment in DC, you know, it's a nice apartment. I do like it. I live alone. So it's more than enough for me, but, but, you know, I like it, but I'm thinking like, ban. If I wanted a wife and three kids, there are literally no apartments I can get, but there aren't. And I make good money, right? I'd have to move out of the city. If I want to live in downtown, there is not a place I can get that it can support having three kids. So I do think that's another big problem as well. And you know, we can talk about why real estate prices are high if you're interested, but I think there are just probably like 15 different factors at play here that are all making our enclosure not good. The same way that, you know, the enclosures for animals can be good bad in many ways. You know, maybe you don't have enough plants, or maybe the food isn't high enough quality, or maybe you need more space to move or whatever. And I think for us, it's just there's probably many reasons like this. And they're not captured by GDP, they're not captured by money, they are a different thing. And they're currently not being tracked, they're not to being like, like governments don't really care. People don't really care. If I told you, surprise, actually the fertility rate of America is 30% lower than you thought it was. The response to that is probably, okay, you know, it's not like you'll be outraged, but it's just be like, all right, it's interesting because it doesn't feel like something that you have short-term consequences over, like if you, it's like a fish swimming in water. It doesn't really know that it's water. I actually don't remember the quote there, but like, I don't have a kid right now. I don't feel the pain of not having a kid. But I'm sure that if I had one in law, I would feel that pain and extrapolate that on a macro scale. Yeah. You give a really good explanation for that, but I think from the 15 factors you described leading to that issue, I want to ask, if AI were to kill us all, do you think it would just make us stop having kids? I think you do it much faster than that. Have you thought about that theory? Like it doesn't seem, it doesn't hold up to you. Like did you cross that as one of the, I think like it could obviously if it wanted to, like, "Why wait that long?" You know what I mean? Like, "Why wait?" It's kind of like giving Ann's birth control instead of using bug spray. Fair. Okay. Do you plan on having kids? I hope to, one day, yeah. In a sense, one of the things also why I use this quote is that I'm also confused by I don't really, like, similar to you say, I feel similar. I don't really feel the pain of not having a kid, but like, I know once I have a kid, you know, I'll love that more than anything else in the world. I know this. Like my parents both love me so much, you know, and they, my dad, you know, always said to me that my kid was the best day of his life and he would never trade for everything. My dad had a good life. He had a lot of good stuff going on, but like, nothing could top that. And every time, you know, anyone I know has a kid, they're just like, yeah, it's that good. And I believe them. And I think I, and I like kids. I like kids, you know, I like animals. I love marriage. I love these things. I think all these things are really nice. I like them. And yet I don't really feel the pole, which is really weird. And I can't really explain that. So I hope it happens someday. I would love to just one shot this whole argument with, well, birth control and like propaganda around, like not having kids, but it doesn't feel like that because I myself don't have this sense of urgency. And I feel like it was culturally relevant to have that urgency. Yeah. I think culture is one of the 15 really. And I think birth control is also one of the 15 reasons, right? Like, I do think they're part of it, but I agree. I don't think it's a one shot. Like I think if you pick any of these, it's not the full story. I think there are just many issues. I do think the issues you raise are part of it, for sure, but I don't think there's anything. I mean, I don't think like dating norms are a big problem. Like the way dating works right now, I think it's extremely bad. Like it's really these adversariality of like you're, you're like testing each other. You know, you go into a date and it's already like a combat, you know, you're already like trying to suck each other and so on. This is bad. Not how people used to find, you know, their soulmates. That's not how it work. Like usually the way, you know, even on my two cents, which is not worth much on like how to meet good people who you want to spend your life with, is that you want to be around people doing projects together. You want to like organize little parties or events or things together or like work at a shared, you know, nonprofit, you know, volunteer or something, because you want to encounter people doing normal things around you and see how do you mesh because the most of a relationship is not the exciting kissing sex part. Most of it is waking up in the morning, making breakfast, you know, going to work. And this is the thing that you want to test compatibility on. Not you have all the makeup on, you have all the fancy things that you go to a nice restaurant. That's really not the important thing to test compatibility on. You really want to compare the one is that if you're both tired and you have some work to do together, do you have each other's back? That's the more in my opinion, the more important thing for a relationship. So Connor, every person I have on this podcast, I feel like they get to a point where they're not allowed to say the full truth about the problem or set of problems that they're describing, you've hinted at yours refusing to name the scientist who went crazy because of AI refusing to say which CEO said what exactly, but in regards to everything with the labs, the cults, the people behind all of this, what's the one thing about AI that you haven't said publicly? Well, I do say a lot of things publicly, I'm a very public person, I'm a very honest person, so I would say there is nothing easy to communicate that I haven't said publicly. I think there are things that I have not said publicly because I don't know how to say them, and I usually so I usually don't say them, or that are so weird or so confusing or so disturbing that it doesn't really help to make people anxious about it, you know what I mean, can you give me an example? I wish I could, you know, it's just not that easy. There are just some things in the world that are very complex, and I know they're true, but I don't really have the words to explain them shortly, like, you know, maybe you and me spend six months being friends and talking every weekend, so maybe we can get there, but I'll give it a shot, but there is a sense in that I think about a lot, which is like, there's all these individual things, you know, this company here, we have the AI things here, but there's something that sometimes gets to me, and it's, I sometimes call it the horror, like sometimes I'm deep in it, I'm thinking about like, how do we solve the politics here? Okay, why is the AI doing this? Okay, what's the math here? Oh, it's the game theory here and whatever. And for this, like, brief instant, if you'll see like a glimpse of it, like of the beast, of like the true problem, like chaos, that in some sense, the true evil is deeper than humanity. It's like at the level of math, like there's a deep thing where the world is not just physics is not just physics and game theory are quite evil in like a deep way, in a deep way that if you really, really think about like evolution, and what it means, if you think about super intelligence, if you think about what you could do, if you could rewrite people's minds or your own mind, if you think about what is possible, if you could rewrite genetic codes, if you could, you know, produce sulfur, producing, you know, systems, and like what that would mean, and the horror, like in a sense, often when I talk to people, for example, these companies, I sometimes have this moment where I realize I'm not talking to a human. There's a human involved, but the thing I'm talking to is not really human, it's the beast. It's a tentacle of a monster. And it's kind of possessing the person, like there's a person there, but they're also, in some sense, possessed by their incentives, by their, you know, by their social roles, by the stories they tell themselves, you know, where like I'll talk to CEO who says, Oh, AI is good, actually, because you know, this reason, and I'm like, wow, I'm talking to a creature, right? Like it's a creature that comes from game theory, that comes from markets, that comes from, like it's like the spirit of the market has possessed this person. It's not really a person. There's a viral tweet I once did where I absolutely blasted a poor open AI employee on this exact reason. His name is Roon, and I think he's a really nice guy, and I was pretty cruel to him, but he took it in good humor. And I basically just pointed out how much his actions and thoughts were basically just those of like a walking corpse is I could predict everything he would say just from knowing where he works, that he was just like one little cog in the larger system. He wasn't, in a sense, even like sentient, he wasn't even conscious, which is very unfair, because I think Roon is a really nice guy and a really smart guy. But in a sense, his actions are fully determined by just being a cog in this much larger creature, this much larger being that is trying to build super intelligence. And he's just one of the, one of the tendons of the monster. I don't know if any of this made any sense, or is useful in any way. It's not something I think I've talked about on podcasts before, but sometimes I think about that, about how, in a sense, I'm not even mad at the people involved. And in some sense, I think they're victims too. In some sense, I look at like, I don't even feel like Sam Altman or Elon, in some sense, I feel like they're victims too. In some sense, I feel like they're also stuck in a bad situation. They're also stuck in a horrible situation. You know, they're also, I've become like twisted into a shape that is causing them to do bad things, whether they like it or not. I think that was beautifully said. That's something I think about a bit that I think there's an author Mary Harrington that had this idea of an Edgar Gore that someone encapsulates it. You know, I didn't want to use that word, but it's useful. Like, memetics and people don't know where their thoughts come from. You know, like, if I say blue duck, it's really fucking hard to not imagine a blue duck. And if I say it three times, and it's really hard if you do not imagine it later, for instance, and it's like free will is almost determined in a way from everything that's external, which means that everything external is shaping you, but you're shaping other things. So there's a way that, like, there's a Buddhist teacher I once read a book from, who I've said, said it quite beautifully, where the way he said it is is that one of the Buddhists could have believed, Buddhism believed in a self. That's what they usually say out no self. But the way he explained it is is that self isn't a thing. It's a process. It's an action. You take it's a dance. And if you take that seriously, which I think is true, it does have some disturbing implications. There's a thing where in a sense, your soul is not localized in your brain. If you think you're in your brain, you're like a little homunculus. This is wrong. Your soul is distributed among you, your friends, your tools, your mentors, the media you consume. There's a visceral sense. One of the disturbing things about the human brain is kind of the brain updates on anything it consumes. If you see an argument, your brain will believe it at least a little bit. Even if it's totally wrong and you know it's wrong and you say it's wrong, we'll still believe it a little bit. If you keep hearing the argument, you'll believe it a little bit more. And there's almost nothing you can do about this. It's like, in a sense, humans live in like construction, elders horror where our minds are constantly being manipulated and rewritten and our memories keep getting deleted. And so on, it's crazy. It's crazy. If you don't write things down, you forget almost everything you do. Our memories are constantly deleted, rewritten, manipulated, our personality keeps getting shifted and manipulated and twisted. It's kind of crazy to be human, like in a sense. It's kind of crazy, like how unstable our souls are in many ways, our minds are in many ways. And if you take that seriously, there's a lot of disturbing implications of this. There's also a lot of freeing implications of this. A lot of the Buddhist implications and so on are then like once you see it for what it is, you can work around it. Like for example, I never moved to San Francisco. Because I knew if I moved to San Francisco, it was going to get to me. I knew I know I'm not immune. It was going to get to me. So I didn't go there. When you say it, do you mean the behavior or everything that you just described? Is there a difference? Right. Agregors are also just behaviors, you know, means things. But there are monsters in San Francisco and Silicon Valley. And everywhere else, agregors, gods, demons, spirits, whatever you want to call them, right? Ideas. Ideas and they can get to you and they do get to you. I know many people who are good people. They care about AI safety, whatever. They move to San Francisco. Six months later, they happen to not care about AI safety everywhere. The monster gets to them. The monster always gets to them. And fighting monsters like this is very hard. And one of the thing is just try to not get that too exposed. Like if all your friends believe something, all your colleagues believe something, you're keep being told something, you might think I'm a rational person. I will never, nah, you will not. You will not. You are not to build different. No one is built different. You know, maybe you're for Zen master on a mountain somewhere who's spent 50 years meditating and maybe you can resist the monster. But in a sense, there's kind of this idea of like psychic predators. You can get like psychophana is sometimes the word I like instead of agregors like psychic animals like creatures. And they do hunt. Like they do go after certain people, like certain cults prey on specific type of people. You know, some cults are for stupid people, some cults are for smart people, some cults are for rich people, some cults are for poor people. Like there's like many different like psychic predators that like like agregors that like prey or target or hunt specific type of people. And if you think there's no predators that are specialized in hunting you, bad news for you. And so in a sense, evolution goes on many, many levels. Like the word meme actually originally comes as the information equivalent to the word gene. That's actually where it comes from as a unit of information that can involve in memes can become predatory. The same way the genes can involve predators. So a lot of these things like when I see someone who has like a crazy ideology, you know, they're like crazy right wing, crazy left wing, whatever, it's very easy to hate them and be like, oh, look at this fucking monster terrible person. But in a sense, I know how I see a victim. I'm just like, fuck, a monster got them. One of the beasts got them. Fuck. I hope he gets out. And yeah, I know this is how I feel a lot when I see these things. It's just like it's at a city's monsters. You know, I see these beliefs, these ideologies, these religions, these things, and how they get people. And I always find out, you know, sat and disturbing and tragic in many ways. - I'm so thankful for you to give that breakdown because I've been waiting for someone to put that eloquently. And I think it describes a lot of how everything works in life. What can the average person do to stop AI? - A lot. Much more than people think they can. Now, this is unfashionable to say these days, especially in the United States, but we do live in a democracy. And you should contact your lawmakers. The way I like to think about is that our civic duty are our contract as part of a democracy, a citizens of a democracy. It's not just that we vote some guys in and then we'll see what happens. It's also that we should collaborate with them. We should help our policymakers understand what do we care about? What do we need to happen? Why do we want it, et cetera? This is a back and forth. This is a collaboration. I like to think of a good state and a good government as a collaboration between its citizens and the state. It's not totally isolated. It's a thing we do together to make sure that we have a good life. And we have a good world that we can live in together. It's funny we're just talking about this, but in a sense that when I went to DC for the first time and I live in Washington, DC, you know, I met many politicians and many staff and so on. And you know, it's so easy to hit politicians. You know, it's so easy. You know, you see them on TV and they're so annoyed. It's so easy. But the real thing I came away with from talking to many, many politicians, it's like, I feel bad for them. And because really they're people like you and me. There's a couple evil vampires. There are some actual evil in human vampires who are like so far gone. It's crazy. Yes. But they're actually a very small minority of people, like really small minority. Most of the people in Washington DC are normal people like you and me that also are just trying to get through the day and do the right thing, but they don't really know what the right thing is either. They're confused. They're overwhelmed. They're being yelled at from all directions. They're parties yelling at them. The other party is yelling at them. The other constituents are yelling at them. The president is yelling at them. Everyone is yelling at them. They have social media as well. They have social media. They have like a super busy day where they have like, you know, 10 minutes to think about every new decision they need to make. It's like, holy shit. If I was in this situation, I would also be making bad decisions. I recently talked to an ex-member of Congress and he said when he became a member of Congress and they gave him a handbook of like, how to run his office, what he should expect. And there was an example schedule for one week, what an average week is like. And in that week schedule, the time allocated for reading or learning new things was 21 minutes per week. Look, if I had 20 minutes per week to learn new things, I wouldn't know shit. I would know anything either. Like, in a sense, I'm like, they're set up for failure. Like, if you don't have time to think, how can you make good decisions? These people are so overwhelmed. They have to work so hard. Like, do you know about cold time? Have you ever heard this before? This is one of the craziest things I've ever learned. No one knows this. Did you know that the average schedule of a member of Congress includes every day four hours of calling your constituents asking for money every day? Insane. - And constituents, in this case, would mean just any donor or like what's that? - Yeah, so in the United States, we have certain laws about who's allowed to donate the candidates. So generally, it is people from that district that errors on who are American citizens. And yes, they will literally call them up one by one and ask like, hey, you know, have you considered donating to blah, blah? And which is insane. - Four hours a day? - Yes. This is insane. I thought I was going crazy when I first heard about this and I asked the ex-member of Congress, like, oh yeah, yeah, yeah. Incredible. How are, we can't have our politicians be wasting their time on this. They have really important jobs. And then there's always like meetings and then you have like dinner meetings and so on. Like, it's a hard job. Like, I want to really say it. It's a hard job to be a politician in the United States. I assume in most other countries too. But especially in the United States, it's a hard job. And the way I see it is is that my question is, how can I help? How can I help these people make the right choices? 'Cause I think they want to make the right choices. Not all of them, as I said, yes, there are some vampires. And you know, if your elected representative happens to be one of them, I'm sorry. Don't vote for them again. But most of them, they're just overwhelmed. They also don't know what the right or the wrong thing is. They're scared about re-election. They're scared about what the media is going to say about them. They're scared if they're going to lose the next election. Like, they're just super overwhelmed. And we need to help them to be like, hey, hello. I'm one of your constituents. I care about this issue. Here's why, here's some material. Go talk to Connor if you want more details. Stuff like this. What's the action though? Who do I call right now? I wanted to. You go to controlai.org, and you contact your lawmaker with our tool. You put in your zip code, and we'll tell you exactly who your representatives are, and exactly how to contact them. So phone calls are even better than emails. But emails is already good. So we'll give you templates for how you can send emails. It's always important that you always make sure the email is the one you actually would like to send. And you actually believe in, not just copy paste, and or call your congress member. And our tools will happily run through you of how to do this. We also, every so often, run seminars and groups where we help teach people how to do this. If you want to do more than this, if you want to really learn how to do this properly and do more than this, there's also a volunteer group that I help run, which is called Torchbearer.community. You can sign up there where we dedicate at least two hours per week to being good citizens, to trying to make the world a better place, so try to address the extinction risk from AI through good methods and building a good humanist world. So those are two things that I think anyone can do who wants to do something. Contact your congressperson, takes maybe five minutes. - Control AI. - Control AI.org. - Control AI.org. - Control AI.org. Contact your lawmaker, and it does make a difference. It really does. I swear, I have talked to policy makers who will not often say like, oh, we get a bunch of emails about you. We'd love to talk. This has happened, and it's important. Congress people do care about reelection. If you, as a citizen, make clear that you will vote against people who do not act on AI, they listen, they do take notes. It takes a lot of us to do it, but it is possible. - We have a closing tradition on this podcast, where we ask the same question every time. Connor, what is the best piece of advice you've ever received? - That's a great question. Don't be stupid, which is very different from being smart. It is much more important to avoid making stupid mistakes, and to be simple than it is to be really clever. This was a very important lesson for me to learn. - Why should you not be stupid? What else? - In a sense, one of the biggest lessons I have learned over my life to be effective in the world was not me learning new things. Like, I was a smart kid, you know, I love learning new science and math and all these crazy things. And I can always crazy theories and all of these super complex plans, and they never work. Because that's not how the real world works. Most of the things that made me successful in life and allowed me to achieve objectives is been unlearning stupid things, unlearning bad habits that I came up with. Learning, for example, that I can just like, I don't need to do the stupid thing. I could just like, you know, in the past, you know, for example, I'd like stay up late and I'd be like, oh, I have to work late. And I'm like, wait, that's stupid. And then I just go to bed, you know? Or I realized that I can't a very funny thing that happened to me once is I resisted doing meditation for a very long time. I was like, I don't need that, whatever. And a friend said, you can't count to 1,000. You know, what do you mean? Like, I'm really good. I'm highly numerous. I'm good about, of course, I can count to 1,000. Nope, you can't. You're going to lose your attention. You can't do it. And I'm like, fuck you. I sat down that evening at home and started counting. I made it to 330 before I lost my train of thought. And I was like, oh, shit. Like, I need to concentrate. I need to focus. I need to be better. I need to do simple. I need to practice focusing. I need to practice not getting distracted. Before I think about, you know, we're learning a new form of math or a new complex thing. I have to first, like, stay on topic. I have to take notes. I have to, like, go to work at the same hour every day. I have to eat a decent lunch. Until I'm doing, I need to do all the basic things. Like, and I have to, otherwise, what the hell am I doing? Like, people often find it frustrating when they do strategy with me. It's because I like to say I'm frustratingly simple-minded. It's like people will come with me these crazy plans and have like 17 steps. I'm like, won't work impossible. And so what I usually do is do the simplest, stupidest thing first. Whatever's the most simple, straightforward thing, do that first. And then if that fails, maybe do something more complicated. If your plan has more than two steps, it's never gonna work. Don't be stupid. Don't do something complex. Don't be smart. Don't be stupid. Don't be smart. Both bad. Keep it simple. Stay on track. If you know, oh, I really should be doing X. Do it. Just do it. Just, just, just. The magic you're looking for is in the work you're avoiding. Yes. It's like, it's crazy how often I see like people who are like, wow, Connor, I really wish I could work as much as you. You're so much full of energy. I'm always so tired. I'm like, oh yeah, how much do you sleep yesterday? And they're like, well, I had like an energy drink at 8 p.m. And then I was awake until 2. And I'm like, yeah, don't take the energy drink at 8 p.m. What are you doing? I'm like, oh, yeah, I guess I shouldn't do that. And then next week they do it again. And I'm like, just don't. Just don't drink the energy drink. Go to sleep. Right. Got me looking at my energy drink. Yeah. I have a little bit of a little bit of a crusade against caffeine in the late in the day. Yeah. Like, I drink caffeine with my breakfast. And I never again. Actually, even worse than that. I have recently. Okay. This is completely off topic. We want to give one a fun one to end the podcast on. This is a fun one. So we all love caffeine, right? You know, it keeps you nicely awake and so on. But you look at the Wikipedia for caffeine. You look at the half life. So like how long it would take for half of the caffeine to leave your body? It took four hours, right? So yeah, it's around like four to six, which is pretty long, all right? Lies. Lies of manipulation. Because did you know that caffeine, when broken down, turns into another compound that is just as potent as caffeine and has another four hour half life. So the real half life of caffeine is close to 10 hours, actually. But turns out you can just take the second chemical, which only has a four hour half life. And works just like caffeine. So I take that in the morning. What's the second chemical? It's called parasanthene. You can get an Amazon. You know, it's pretty safe. It's totally safe. And it's just like caffeine, but it doesn't last as long. Does nicotine really have a half life of 30 minutes? Yes, nicotine is a super short half life. Nicotine is a super short half life. Very addictive. Do not do nicotine. I know so many people who read that stupid growing post about blogger, about like how nicotine can like help you be smarter. It's bullshit. It's just addictive. Do not do nicotine. I also think that's a lie. Yeah, but I do nicotine stuff. But regardless, awesome. Well, everyone, this has been your guest, Connor Leahy. This is the Jack Neil podcast. Where can people find your work? You give a website to support, but-- I mean, generally, controlling i.org is the most important thing. You know, so find me on x and various other platforms, or various kinds. I have a blog, which has more of the like crazy psychophonial stuff on it, if you're interested in that. But yeah, my name is pretty Google. Amazing. Awesome. I appreciate coming on, man. Yeah, thank you. Thank you.