(upbeat music) So Connor, I had seen a clip of you pop up on my feet. It's like a three minute clip where you're sitting there in this panel so it can intense as fuck. And out of nowhere you start saying, do you know how much you have to abuse a mammal to make it not procreate? Watch the whole thing, had multiple clips of you pop up afterwards and like this guy's fucking fire. And then about two days later a man is like, we got an email from this guy, Connor, he wants to come on the show. I was like, wait a minute, look at that. I'm like, oh, he's shit, I was just watching that guy. So I'm glad you reached out. It's great to finally meet you in person. Well, thank you so much for having me. Hopefully it'll be under the type of thing where you give a lot of warnings that then people heed and bed shit doesn't happen and we can look at this 10, 20 years from now and be like, because of his warnings it stopped and here we are. But I do have to say, some of what you talk about with AI, it's very concerning to me. Hey, nothing in the world would make me happier than being completely wrong in everyone laughing at me. I think this would be the best outcome possible. Do you think that on the note of the original clip that I saw you in where you were talking about the lack of procreation going on in a lot of countries around the world, before we even get to AI and the whole role of that and what that looks like in the next generation, do you think that population collapse is a legitimate fear? - So this is an interesting one because I truly am not a population collapse expert by any means. I do know quite a bit about it. I have something I've looked into quite a bit. I'm an AI expert. So it's kind of funny that of all the clips of me to go viral, it would be that specific and where I talk about something. It's usually not my topic of expertise. But yes, population collapse is so much worse than anyone realizes. Like it's so, so, so, so, so much more worse than like anyone really realizes. And it's worse in like kind of subtle ways. It's not like, you know, oh, there's this huge apocalypse and like everything's on fire or whatever. It's just, it's much more of like going out with a whimper than it is going out with a bang. - Oh yeah. - Which is things slowly fall apart. Promises that were made in the past, never materialize, you know, things become smaller, people become poor. It's, there's many levels to this. And it's much more widespread than people realize. And the worst thing I would say about population collapse is that everyone thinks they know why it's happening. Every single person, I promise you after we put out this video, all the comments we full of people saying, obviously, population collapse is caused by whatever their pet causes. You know, it's caused by feminism. It's caused by, you know, sexism. It's caused by, you know, economics is caused by whatever, right? Everyone has their theory. But I've looked into this quite a bit. And really one of the things that's weirdest about population collapse is that we really can't identify a single cause. This is the thing that we've tried a lot. It's got a lot of work put into this. And there really is no single cause. Every, for every study that says, oh, population collapse is due to X. You can find another study that says the exact opposite. - That's right. - Hey, guys, if you're not following me on Spotify, please hit that follow button and leave a five-star review. They're both a huge, huge help. Thank you. - So really, we don't actually know what's going on. So the reason I made that, you know, equipped me one liner during that talk, was to draw attention to this idea that, look, I'm not saying I have a solution, but something is wrong. - Yes. - Something is deeply wrong in zoos, in like actual zoos, where you keep animals. There's actually a lot of knowledge and a lot of science around that if you keep an animal in a poor enough facility, many of them stop having children. This is a very common phenomena, actually, especially among mammals. So if you have, especially larger mammals, you know, some animals, they don't care. Like, you know, a loser or a disnaked, doesn't care, right? But for mammals in particular, as, especially for larger mammals, they're, it's actually quite hard to get them to breed in captivity. This is often quite hard, especially for more anxious animals, like, say, cheetos, but also stuff like pandas and tigers and stuff. It's actually quite hard. And you have to give them like much more space. They have to have very specific, like, size of implosion, and sometimes like with pandas, we still haven't really figured out how to give them an environment they feel comfortable with, compared to their wild environment. And so the thing that I really feel here is that something is wrong with us as humans. Something is sick about our society and our enclosure, so to speak, the way we live. And I don't think it's a single thing. I don't think it's like a, we just do this one little trick and then everything goes back to normal. I think it's probably like, you know, a mixture of like probably like 15 to 75 different things. Like, I'll contribute a little bit, you know, whether it's, yes, I do think there are problems, you know, which end up relationships on both sides. You know, dating norms nowadays are, I think, really bad. I think, but I also think there are, you know, stuff like the economic problems, you know, children just very expensive. I think another big one is housing shortage. It's just, I live in DC, you know, I make good money. I make decent amounts of money. I can't afford an apartment for like three kids or five. Yeah, like it's just not possible. If I want five kids, I can't have five kids and live in DC. That's right. It's just not possible. It's the same problem here. Yeah. So I think there are just like many, many problems like this that add up. And a thing that's also very important to understand is that basically all projections of how we set up our social security systems, our, you know, tax base, our growth rates, our economic projections are always, have always been based on growing population, that there would be more young people than old people. That's right. And once that stops being the case, things start to break quite catastrophically. And this is also what we're starting to see. So I do think this is a big problem. I think it's a much bigger problem than people realize it. Also, you know, there's my, you know, a bit sentimental, but I also just think a lot of people want kids and not having them that actually would make them very happy. And there's just a lot of happiness that is not happening. Just never mind the economics here, just like having kids is one of the things that makes most people the most happy in their entire lives. And now many people are not having that for whatever reason. I think it's very sad. So I think there's a lot to be said here. I think it is more of a decade scale problem than it is a next year kind of problem. Unfortunately, I think AI is more of a next year kind of problem. So my focus is current on AI. But if AI was solved, this would be pretty high on my list of problems that need solving. It's a great answer. And I completely agree with you that is a smorgasbord of causes and you're right, everyone's got their pet cause. But I try as best I can from my little armchair here that doesn't have really arms on it. But, you know, I try as best I can to zoom out to 30,000 feet and say, okay, take away what's evil, what's not, just look at the objective facts and if you wanted to limit generational size and therefore change the demographics and have some sort of population collapse, what types of inputs would you put in? And when I look at all the inputs in society, starting with since 2007, these things coming out in our lives getting more, I'm holding a phone up if you're listening, not watching. And these things getting more and more integrated into our lives and then kicking that down the road to all the social effects that that had in accordance by the way with an enormous financial crisis that allowed the rich to be able to recover from it and the people that aren't rich to not recover while prices are also getting inflated because they had to print a lot of fucking money to pretend that we got out of it. Like, I could go on and on and on. You add all these things up and you are creating an environment to where I also think there's been some sort of whether it was intentional or not, sigh up on people to where they tell themselves they don't want to have kids. But I also completely agree with you that everyone, if they've really thought about it hard, would realize I think that's kind of the meaning of life, our ability to carry on as a species. - Yeah, I have known many people, you know, working in tech industry, I've known many, many people who tell me, I definitely don't want kids. I will never have kids. I'm so glad that I don't have to have kids. And every single time they have kids, they said I should done it earlier. - That's right. Every single time. - Yep. I've had the same exact experience with people. I look forward to it. Like when I do it, I think kids will be like the greatest thing in your life for sure. But you were talking about like in zoos and how mammals have, you know, they have to try to get them to made in everything. I kind of wonder if there's a parallel there. Correct me if I'm wrong on the research you've done. But, you know, you look in society, people may objectively be doing bad financially. They may be in college debt. They're never gonna pay off. They're renting at inflated prices, work in a job. They don't like, they can't even really pay their bills, but just keeps them subsiding enough. And all the while, they can at least come home and watch Netflix at night. And what I wonder is if there's just enough like comfort and not like threat of death, you know, living in this country that seems to not get invaded. All the time, like some other places in the world, I wonder if that type of comfort is the same type of psychology perhaps that's been shown in zoos to where they don't like living in captivity, but that's what they are. Their food is given to them so they don't have the purpose of having the hunt to survive or, you know, go out and explore in the fields to, you know, fill their day that they're just like, "Well, I think I'll sit here and not fuck." Yeah, it's, we don't know, obviously. Because obviously humans are far more complex than other mammals and are especially our social needs. So one of the most complicated animals to keep are social animals. These are usually the hardest to keep. In a sense, it's kind of easy to keep a tiger, quote unquote, I mean, they're very big, but they're solitary. You know, they really need a lot of social contact and so on. And that makes them easier to keep than like very social animals that need a lot of, you know, social interaction. Humans are the most social animal ever. You know, we are extremely social and we have a lot of social needs.
and now when we get into our use of technology and so on how this is changing our relationship to each other, to ourselves, to our bodies. I do think this is a big thing. I do think it's might some stupid, but I do think a lot of one of the 15 or 100 causes that we just aren't physically close to people as much anymore. It's just not the same to text a girl and to sit next to her. It's just not the same. Even if you're saying the same words, it's not the same. It's not the same for you. It's not the same for her or him or whatever. Humans were mammals. It's normal that we interact by being close to each other and looking at each other and speaking to each other and touching each other and so on. I think there's many, many things on this level. Maybe the comfort is a part of it as well. I do think there's definitely this numbing effect that happens a lot where a lot of people use the entertainment and so on that we have accessible as a numbing agent to try to not feel, try to not deal with and just kind of like delay. It's not that they don't feel bad. You can just delay feeling bad. You can kind of push it out, push it out, push it out, and this is a vicious cycle. This is how addictions tend to work. As you get older, you start thinking about everything going on inside your body that you didn't really have to consider when you were 20. You want to put the best stuff inside here so your body can last and be firing on all cylinders for as long as possible so you have a great quality life, which is why I started taking blueprints longevity mix. If you spend any time online over the past few years, you've seen blueprints founder Brian Johnson. He's the don't-die guy who's basically trying to live forever and whether or not you love what he's doing or hate what he's doing, you can't argue with the fact that he's been running one of the most measured longevity experiments in the world, tracking biomarkers, publishing the data publicly, and refining everything based on results. And that same experiment first mindset is what led to blueprints. This is what Brian takes every morning, and now he's made that formula available to the rest of us. Brian started blueprints because he needed specific doses back by scientific evidence that weren't heavily contaminated by toxins or heavy metals. And it's pretty crazy to realize nothing on the market really check those boxes. With blueprints, every ingredient fights for its life, literally. They formulate based on population level research and study level doses with no filler. Instead of a whole cabinet full of pills, it's one scoop in water, mix, and done. And that single scoop helps support energy, cognitive performance, mood, focus, sleep, cellular resistance, and healthy aging. This is science-backed, precision-dosed, no BS. For a limited time only, our customers are going to get 20% off their order, plus free shipping at blueprints.brionjohnson.com, link about a description below by using Code Julien at checkout. That's Code Julien at blueprints.brionjohnson.com, link about a description below for 20% off. After your purchase, they're going to ask you where you heard about them. Please support our show and tell them I'll show you. Like, the way the really addictive substances tend to work is not just that they like, you know, you know, activate your biology or whatever, that's a big part of it, but often it's because it's part of a ritual, because it's part of an escape mechanism. Like, for example, nicotine gum is much less addictive in cigarettes. And there's two main reasons for this. One is biological, is that if you smoke something, it acts as your brain much faster. So the hit is much quicker, and this is known to make things way more addictive. This is why I do not smoke or snort things. You know, if you're going to take a drug, take a pill. Not, not, not, not, not, not, not, not health advice, your spec, but hypothetically, yes, hypothetically, in a hypothetical situation. But the second is a actually very much a social and a psychological effect, where it becomes ritual. It becomes a little ritual. You can take a 15-minute break. You walk outside. That's right. You get a de-stress. You get to, like, break out of a cycle you might have been in right now. And this habit is actually super, super, super, this is like the biggest part of what makes cigarettes so addictive compared to other drugs, including some even harder drugs. Like, cigarettes are more addictive than many hard drugs, actually. And so there is this large psychological component, this large sociological component interactive. And also what addictive addiction therapy generally shows is the number one way to get people to quit hard drugs is for them to have good social experiences is that, for example, there's there's a very famous study that during the Vietnam War, I think it was like 20% or something of soldiers were hooked on heroin. Like an unbelievable. It was like unbelievable. It was like so. And so people were terrified. They're like, oh, shit, when all these GIs come home, there's going to be this massive wave of, you know, heroin addicts everywhere. It's going to be awful. And it didn't happen. It didn't. It not happen. They went home and they just quit. And it was fine. They went home to their families, went home to their friends and they quit. And it was okay. From heroin. Yep. So this doesn't work for everybody. But for if you want to get off a really hard drug, generally the best way to get it is to have a really loving family and loving friends and something to come home to every day and to spend, you know, time with your friends and stuff like this. There's kind of, this is a well-known thing addiction psychology generally is that social interaction and social wholesomeness is like the strongest way to treat addiction of any kind because it's kind of the best thing. Like if you have the choice to like, you know, do heroin or go hang out with your wife. Like it's like one of the few things that is actually better than heroin. Yeah. It is actually one of the few things that is in fact better. And so this is a very common human behavior. For example, we're just like, the problem is, of course, you can't really prescribe, have a loving wife as a, you know, as a pill. That would be great, you know, it would be that simple. But so I do think there's this thing where like as we have this like fracturing our society, we have less of like the, you know, third spaces, you know, where we hang out with friends at clubs, you know, sports or, you know, people as people are getting married, less, you know, intimate relationships, less dating and so on, that it makes one more prone to these addictions, especially social media, media addictions that are the same kind of thing as any other form of addiction. Well, pure spec here. If I were trying to run the world and run it into the ground, if you will, I would make sure that I created a drug that simultaneously acted as a defense mechanism against the very thing that would prevent you perhaps from being on the drug, which is I would create something that automatically assumes the behavior associated has to keep you separated from people completely. And if I have people right here who can be sitting right next to another person, but they are more interested in being quite literally right here on the phone and engaging with that, then it doesn't matter where they are geographically in any given time, I've solved that problem. Yeah, I do think there, I do think unfortunately there's a lot to this. One thing I truly do not believe is that actually anyone did this on purpose. Like this is something I truly deeply do not believe. What makes you not believe that? Because everyone is stupid. There is no one smart enough or competent enough to have ever pulled off something like this. Like I have met some of the most powerful richest people in the entire world and they're just as stupid as you and me. They're just as incompetent as you and me. Like I hope you're right. I promise you, I am right. Okay. Like one of the things I've met, I've met, you know, heads of state, you know, like top power politicians, generals, you know, buildings, and it's their impressive people, right? Not saying they're not, you know, probably they're a couple standard deviations of IQ smarter than you or me, right? Sure. And definitely work very hard, but they're just people. They are just people. In a sense, they're this like conspiratorial view of the world that like there's some shadowy cabal, you know, in charge that someone is doing these things on purpose is kind of more hopeful than what's really happening. Where the truth is is that no one is in control. No one really knows what's going on. Groups think. Yeah. I think it's like it's group think. It's no think in a sense. It's like just, you know, a great example for this, like a good counter example, which is actually not a counter example is for example, China's one child policy. So this would be a great example of a large scale, it's not really conspiracy, but just like policy. That's right. To do population control. It's a kind of large of kind. And now they are frantically trying to reverse it because it is destroying their country and destroying their economy. So it was just, it wasn't some brilliant plan to achieve a good thing. They just completely shot themselves in the foot so hard like that their whole economy is now collapsing because their demographics are aging too quickly. Yeah. I just did my pushback on that would be I don't know where the buck stops. And what I mean by that is if you had a very small group of people who could just pull the levers enough to prey on human natural intuitions that includes group think incentive structures listening to people that they think are smart on something that really just actually aren't and they're put there to tell them something. Then you could make some of these things happen and you could create buggy men that you and I can see publicly they maybe actually have nothing to do with anything other than just following whatever pathways been laid out like bread crumbs in front of them like they're a squirrel. I do wonder about that sometimes because I had always said admittedly before this year stuff like that same exact thing is you and like people are stupid they all take a shit the same way I do you know the same way everyone else out there does but at the end of the day incentives and group think are really what's to blame. The reason I've had to say that part of that I need to reconsider is because I have seen those Epstein files this year and I see the way some select group of people talk about other people in the world. I'm not saying these guys are fucking Einstein they're clearly not when you read it but you do wonder what kinds of things are at least just planted like those bread crumbs and if someone or a very small group is actually doing that to get to another end goal and that you know I'm not really sure what your thoughts are on this but how about like all these guys that genuinely openly say like just the ones that even say publicly that you know transhumanism is kind of interesting like where's that coming from is that and is that a natural idea or is that something that's been
fed to them. Oh, no, absolutely. To be clear, conspiracies do absolutely exist. They are rampant. There are many conspiracies happen all the time. You know, you we have plenty declassified ones, you know, from like the 1960s, you can just read the files. Right. Like conspiracies absolutely do happen. Small groups of people do do small groups of people change the world. In fact, they're the only thing that ever has, in a sense, every great revolution started as a couple dudes in a seller talking about it. So I do think that that is absolutely real. The only thing I push back against is like that there's one global conspiracy or that they agree or that there is or that mean that not most of it is chaos. Like most even then, which conspiracy wins to a lower degrees often luck and just get historical happenstance. And so the incentives thing is definitely true though. Like I don't know if you one of my favorite pieces of fiction ever written this is a line from the Principia discordia. And so the main character Malaclips is talking to the goddess. And he says, oh goddess, you know, the world is full of strife, you know, brothers fight brothers, you know, mothers lose their children. It's there's war everywhere. It's so awful. And the goddess answers, well, if you don't like those things, you know, but you're doing them. So there must be all right. And no, no one wants these things. And the goddess says, oh, well, and stop. So, but obviously we can't stop in some degree. And some degree, there is clearly a thing here where obviously we would all want to just like not have wars. We could just stop. That'll be great. Awesome. Awesome. But some ways we can't. And I think this has a lot to do with not necessarily there's a single shadowy cabal pushing war to be clear, there are shadowy cabals pushing war. But more just is really hard to coordinate. People disagree about complex things. Resources are finite. And just resolving things peacefully is really hard. It's like sometimes it's just very hard to bridge gaps. Sometimes there is no peaceful solution to various, you know, things. And when we get to the transhumanism thing, I mean, that's when I think that's like I think a really good example actually of like a very I say crisp example of an extremist ideology that very much does follow and pursue objectives in reality. I think this is a very good example. So I know many transhumanists, you know, you know, many of them. Oh, many, of course. I used to work in the AI industry. I know many of them. You go to the meet-up sometimes. Oh, back in the day. Oh, for sure. I've gone to the house parties. You know, like, oh, yeah, I remember I have been many of these people. So you weren't at a 970 first street, right? I was not. Okay, that's, I don't even know what that is. That's okay. Yeah, so that's a great answer. Okay, so now I mean, so transhumanism is a great topic. And it's a great topic because it leads into something and a maximum expert on which is AI. Transhumanism is a large set of ideologies to varying degrees that basically all have one thing in common, which is the belief that humanity can and should be transcended to some degree. There's very mild forms of this like we should have better medicine. So we can live longer fine. I don't think anyone's going to complain about that's a pretty reasonable ideology. But more and more, especially in Silicon Valley, there are much more extreme forms of this belief, which is such as we should upload our brains to the cloud, you know, we should shed our biological forms entirely and stuff like this. And these are, you know, pretty eccentric thoughts outside of, you know, Silicon Valley, but they have gained a lot of purchase and a lot of political power within Silicon Valley over the last couple of decades. What do you think caused that? I think these people want to be immortal. It's the oldest, oldest wish in history. Don't try. Don't die. I think that's truly the, in a sense, it's very, you know, like sometimes, you know, people like to make fun of like AI cults or people who worship AI. But I mean, hey, at least their God talks back to them. You know, most most stark. Yeah, I mean, most religions can't claim that, you know. So yeah, it is a, is it an ideology, right? It is a religion to a large degree. Yes. And, you know, to be clear, I'm a big defender of the first amendment and freedom of religion. I think adults in America should absolutely be allowed to join any crazy cult they want. You know, if you want to join a crazy cult, that is your God-given, I agree with you. I totally, I mean, I think it's good for you. I would advise you not to do that. That's right. But you're allowed to do this. Where it becomes problematic is when these ideologies start into influencing other people. Yeah, when they start exerting power influence over industry, politics, and you and your lives. That's right. I am so fine. You know, I'll make fun of them on Twitter. But I am completely fine with weirdos and San Francisco wanting to be cyborgs. Totally fine. Who cares? But when they're paying off all the politicians, you got a problem. Then I got to probably agree. And this brings us to AI. This brings us to the fundamental, the core problem that I think is the most pressing problem, the really existential problem that we are facing right now, is that AI companies, staff to a large degree by these transhumanists, are trying to build not a better chatbot, not a tool to not give them a nice, you know, a little profitable quarter, but what is called super intelligence? Mm-hmm. Is this now when you say super intelligence, are you defining it like Bostrom does? Pretty similar. I generally, how I would define it is, is something that is fully autonomous. You know, it's an agent. It's not a chatbot. It's a fully autonomous system that is vastly more competent than humans, and can outcompete us at everything. It can outcompete us in every economic, business, you know, trading. It can outcompete us in science, engineering, or politics, or military. Does it have anything near sentience? It depends on what you mean by the word sentience. What I do think it has is agency, in the sense that it can take actions and follow goals. You know, does it have emotions or consciousness? Who knows? Probably not. It doesn't matter. It doesn't matter. It doesn't matter. What matters is candidate chief objectives. And does it have objectives that are misaligned with ours? What we've been seeing over the last couple of years is the emergence already of what are called agents. So in the history of AI, we kind of went from very specialized AI systems to do a specific thing. Like maybe you have an AI that can detect pictures of dogs. You know, pretty harmless, right? Something we can understand, you know, not a big deal. Then we moved into this like generative AI towards stuff like chatbots and your chat GPT and stuff like this. This is through a technology that was called large language models. You may have heard this before. Yes, important. An important thing to understand, and this is something I think a lot of people don't know, is that the current systems that use names like chat GPT or Clawed are not LLMs. They have not been LLMs for years at this point. What are this? They are what are called agents. And they have a lot in common with LLMs. They use a lot of the same techniques. The way an LLMs made, or generally how an AI is made, is through a technique that is called neural networks. This is very different from how traditional software works. So the way traditional software works is an engineer like me sits down and writes a bunch of code lines of code to tell the computer exactly what to do. Yes. This is not how neural networks work. Neural networks are more like grown rather than written. It's the way I've thought about it, and please correct me from wrong here is it's more like instead of building a home, it's more like planning a seat and walking away and coming back. That's much more like exactly. That's a really good way of thinking about it. You have these big piles of data on these massive computers, and you kind of grow this neural network to solve your problem. And the way this neural network looks, you know, I've made these things myself. I've trained, you know, my own neural networks many times, is instead of lines of code, there are more like billions and billions and billions of numbers. And if you multiply and add all those numbers in the right order, you get chat GPT. But importantly, we don't actually understand what these numbers mean. We don't actually know what's going on inside these numbers. We know that if we execute these these numbers on our computers, your computer can talk. But we don't actually know why these numbers do this. So how do we even get there? That's a really good question. It's fundamentally just exploration is just we try to many different things. So generally the way LLMs were made is that you would take a neural network and you would give it some text and you would have it try to make it predict the next word in the sentence. So like, hello, my name is and then it would have to predict what's the next word going to be. It's probably going to be a name, you know, maybe you'll see how I already seen another name in the text, stuff like this. And then you give it a treat like a dog and say, good, oh, wow. Then you give that's exactly correct. And you do this about, you know, one to 30 trillion times roughly and then you have something that can talk. This is how LLMs work. And the 30 trillion happens because once you, I'm just painting a picture here so I can understand once you give the treat to the dog because it correctly got the word, it can then maybe iterate just through that 10,000 other things immediately because of that answer. It gets just a little bit more knowledgeable. And you just keep doing this, you keep doing this. What the big super computers are for? The big super computers do this. This is called training and literally like a dog. You know, it's like training a dog. So you train it trillions of times to predict all kinds of different things, right? You know, websites and emails and stock reports and stories and books and just everything. And in the process, it learns a lot of things about how the world works. It learns, like if I, for example, asked, you know, chat GPT, you write me a poem about why I love my dog. It actually has to know quite a lot about the world to be able to do this. That's right. It has to know what a poem is. It has to know, you know, what dog is, why I might love it, you know, how to combine these things. So this is one of the shocking discoveries that was made in 2019 that this works at all, like that this works at all. Who discovered that? It's a little controversial, but generally I would say opening eye. Okay. I think Alec Radford and his team, I think, deserve most of the credit for this discovery. Not there are many other people involved. Of course, they made important contributions, but I think their team does
is most of the credit for the specific discovery, where this was actually, this is the largest degree where my story started. I'm not started, but this is where my story really kicked into the final phase because I was very interested in AI long before this. I had already thought when I was a teenager that I'd work on AI because my thinking was, well, I want to solve problems, I want to make the world a better place. - Deep over here has been working on paying off the student debt, ain't that right, Deep? - Yes, it is. Obviously, many of you know exactly what Deep's talking about and how annoying it is to plan out your future finances when you got that like that hanging over you. It's nice to be able to see a light at the end of the tunnel and an actual payoff timeline integrated into all your finances so you can see it all in one place, which is exactly what you get with monarch's debt payment simulator and visualizer. It is literally a tool that organizes everything I just said and helps you track your progress. The simulator and visualizer is going to show you your exact debt free date based on your real balances and payments and you can watch the timeline move. The visualizer lays out your whole payoff journey as a clear motivating timeline. And you can play out your options and pick the fastest one, too, if you'd like. The tools let you run different payment scenarios side by side and instantly see how much sooner each one gets you to pay off your debt. You're in control of your plan. With quick check-ins on your laptop or your phone, you'll always know exactly where your money is, how your debt schedule is progressing with monarch. Most apps only tell you what you've already spent. Monarch helps you set goals, map out big purchases, and see if you're actually on track before it's too late to adjust. The right-your-own-money story today with monarch. Use code
[email protected], link in my description below, to get your first year of monarch core half-off at just $50. That's 50% off your first year at monarch.com, link in my description below, code Julien. Well, you had told me some of this off camera before we talked, but for people that obviously weren't there for that conversation, where are you from? And where did you grow up? Yeah, so I was born in California a long time ago, but I grew up in Germany, actually, with my mom. She's German. So a South of Munich in Berweria. Beautiful, highly of the Alps land, very boring. I went out as a kid, but I love going back as an adult. And what's it like looking at America and being here now as someone who spent so many years growing up, not here? I think it definitely gives you an interesting perspective. It gives, in many ways, both positive and negative. So it's not like a lot of people expect that, you know, if you live in Europe, you only have negative opinions of the United States, that's not true. I have a lot of very positive things to say about the United States. Sometimes I like to joke that the United States isn't really a good country, but it is a great country. And I think Germany is kind of the opposite in any way. Isn't you explain that? The reason we moved to Germany was because my dad got very sick. He got very, very sick. He developed a very severe illness and insurance wouldn't pay. And so we were very wealthy. You know, we were not very wealthy, but you know, we're a good middle class. You know, we were doing well at a house in the suburbs of LA, you know, living well, you know, third kid on the way. You know, living in the American dream. And then basically within six weeks, or six months over, all gone. All the money was gone. Everything was gone. And we went to Germany. And my dad, you know, he didn't speak a word of English, a word of German. He didn't speak any German. You know, he had ever been really to Germany and he visited it before. And, you know, he had to go to the hospital, of course. And they took care of him. You know, gave him all the medication he needed. And I remember he always telling me this. He was like, that he tried to talk to the doctors and be like, you know, thank you so much. But like, what do I owe you for this? And they were just like so confused. Like, what are you talking about? Why don't you don't owe us money? And get out. Like, it was, in a sense, the way in which Germany and Europe in many ways are good is kind of not in them being like epic, being like superheroes or like emotional. It's just in the sense of like, in like how normal it is. It's like, oh, you need a medication. Yeah, just go to the hospital, of course, you get it. Like, duh, it's boring. Like, you know, there's some paperwork. Well, it's like very boring. It's very low-key. Things just work. You know, the streets are clean. You know, things, you know, not all the public transport, but most of the public transport just works. You can, if you need something, you know, if you need a medication, you just get the medication. Like, it's fine. Like, I remember once, when I came to Germany, and I needed, I needed a prescription medication. And I went to the pharmacy. And I needed quite urgently, because I'm antibiotic. And they were like, oh, we're so sorry about this. We're so, so sorry. But, you know, given the time constraint here, and like, 'cause, you know, your doctor didn't write up the generic, your insurance is not going to cover this. And I'm like, look, I need the antibiotic, whatever. And it's like, yeah, it's going to be, it's going to be seven euros. (laughing) They were so apologetic about it. And I'm like, are you kidding me? I just give you the thing. God damn it, I'll give you a tip. Like, it's like, so, but the thing you understand is this is not because like, Germany's are like superheroes. It's just because it's just like common sense. There's a lot of these common sense bureaucracy that works. There's a lot of common sense bureaucracy that does not work in Germany. Oh my God, ask me about notaries in Germany. There are many things that don't work. Notaries. Oh boy. The things you could have picked. Oh boy, you don't know about the German notary system. Oh my God. Like, this is a nightmare. That didn't recover from World War II. It's actually unbelievable. It's like, in Germany, notaries are basically a protected guild class. There's only a specific number of notaries in the whole country. And basically, if you want to become a notary, one of them has to die. And then, and basically, every large-scale contract must be notaries in Germany. So very infamously, including venture capital investments. So if you start a company in Germany, start up. And you get a 50K check from an investor. The investor has to physically fly to Germany to stand in front of the notary, the notary will then read out the contract, and then stamp it. But that makes up for the health insurance. Yeah, like, I'm not saying it's-- I'm truly not trying to say is that all problems are solved by any means. That's why I mean, like, it's a good country. But it's not a great country. If you want to be rich, if you want to do big things, if you want to save the world-- Germany's not the place to be. If you want to have a good stable life, where you have a good, stable, middle-class job, your health care is taking care of, you can raise your kids safely. Germany's a great place to be. 100, fantastic. If you want to save the world, you've got to come to America. Yep, I don't think you could say that any better. And you actually lived to both of them. I was back in Italy. I'd lived there in college for six months, but I was back there a couple times over the past year. And when I was there in June, my friend Lorenzo, he's like 22, 23, really motivated in real estate, got a lot of big dreams and everything. And he and I were talking about that. And he was breaking down the government and the system and how it works. And it's very clear to me that if you want to live in normal, white-pick at fence, have a few kids, middle-class life, I can actually-- and this is not a compliment to America, obviously-- I can make the argument you are better off doing that in Italy. In fact, I have a family member who's about to do that right now. But what Lorenzo was saying and proving with points that he was absolutely right about is that if you have dreams to change the world, or not even that, just dreams of being really successful at something and being the best in your field, America is the place to be. And I didn't really know how to square that, because fewer people are going to do the second thing. The first thing is what most people is a baseline want for a happy life. And the fact that we have legislated that away of a whole litany of things in America seemingly is very scary to me, because that was not the case 40 years ago. I agree. I think this is very bad. This is why America is not a good country. It is a great country. But the reason it's not good is because having a middle-class life with never mind two, but maybe three or five kids, I personally would like more kids, it's just kind of unimaginable in many ways, maybe in very rural places. But even there, one car accident, one health disease, the thing ends over. It's just like it's really bad. And it doesn't have to be this way. Fun fact, did you know that America pays more for health care than Germany does? I mean, that didn't surprise me. But this is the example I mean, it's not special. It's not that the Germans have some special superhero thing. It's just kind of like-- And the German system has many problems. As I said, no worries, God. And other problems. Even the health care system also has some problems. But it has like reasonable problems. Like when people tell me about the problems they have with the German health care system, I'm like, yep, that sucks, but I get it. This is a normal thing. When people in America tell me about the problems they have with the health care system, it's always just the most insane horror story I've ever heard in my goddamn life. And this, it doesn't have to be this way. It's not a rule of nature that you can't be good and great. You can, in fact, have both. America used to be good and great. We used to have both. We used to really be good and great. We had everything. And you know, there were problems. Not saying we have, but we can improve. I think that we can improve so many of these things. It's very interesting that, you know, as I talk a lot to like Silicon Valley type people. And often, they describe those techno optimists. techno optimists. That's a word they love to say. God damn it. I get better branding. I know, right. It's terrible. But really, what I think this means is much more civilizational pessimist. They believe that technology is the only way to improve the world. They will focus so hard on technology, but they'll never contact a local congressperson. You know, they will never try to work with the government. Because instead, they're focused so much on building, you know, some crazy new technology.
And look, technology is great. Technology has improved our lives dramatically. No question about it. But when I ask someone kind of going back to our conversation earlier about birth rates, if I ask someone, what are the top 10 things they're most concerned about in their lives? Basically never is technology in one of those, like they don't have enough technology, one of the top 10 problems that I have. It's almost always stuff like, I don't see my friends enough, you know, I want to have more kids. I'm worried about crime, I'm worried about political corruption. It's all the problems they have are social and political problems, right? And I really, and this is the thing that I think is quite toxic about, say, the San Francisco and the like Silicon Valley kind of mindset, is that these are some of the smartest people in the entire world. Man, like, you know, it is crazy, but they're so focused on just technology. And like, these are people that should also be working in our government, you know, these are people that should be working on the social problems. These are people who should be solving, you know, in a sense, it's so sad that we're seeing the rise of our generations, you know, optimize click-through rate on ads for meta, right? Instead of working on reducing crime or improving, you know, people's, you know, middle, you know, like, you know, white picket fence dreams incentives. Yeah. I think incentives is a big problem with this. I think there's a fundamental problem where a large problem of state craft in general, state craft, like building states, creating governments as a, as a science and an art. A large part of this is, how do you manage incentives? This is also what regulation actually really means in the sense that, you know, a lot of people, especially in the free market type, send things, regulators, and annoying stupid thing the government does that you have to get around, but that's not really what regulation is supposed to be. You know, sometimes it is that again, notaries. I understand. Trust me, I understand. But really what it's supposed to be is in a sense, a free market is kind of like an AI. In the sense that it only cares about one thing, money, number go up. It's the only thing cares about that's right, if it is legal and they are not and they can get away with it and it makes the number go up, they'll do it. And in a sense, what I think is one of the greatest triumphs of the 20th century is to a large degree actually taming that beast, not perfectly, but, you know, you know, social market, you know, cap, social democratic capitalism has brought massive amounts of prosperity and goodness to the world. It has, in fact, cured many diseases. It has, in fact, produced, you know, reduced world hunger down to a much, much smaller level. It's not eliminated these problems. But that's because the market isn't good or bad. It's kind of like a wild animal, you know, it's kind of like a wild beast that will do anything to get its treat, to get its money and we have to wrangle it to get something good. One of the things that I find most inspiring about America and American capitalism and generally Western capitalism is how we can take the most sociopathic maniacs in the CEOs of many of these top companies and get them to do something useful through society occasionally. I think that's incredible because, you know, in another lifetime, these people are like Jenga's con, right? Like these people would be less slaughtering people if they could get away with it. But instead, you know, they're at Amazon making sure that my package arrives, you know, a day earlier. Great. I think this is a fantastic thing because we will always have to deal with sociopaths. They're always going to exist. We're always going to have the Jenga's congeans around, right? So the question is, how can we put them to work? How can we shape the incentives of your Jeff Bezos or whatever? So that yeah, they can still be, you know, conquering maniacs, but that it's use, they're doing good work for all of us that don't change them but passively control them. That's a lot of how I think I just just pay them to do good things. Like I am not against people making a lot of money. I think that if you make a fantastic service or you, like if you cure cancer, you should obviously become a billionaire. Like obviously, like if you personally you pick a cancer, I would absolutely like vote for you getting a billion dollars at least. I think this is fair is that I think that if you like, you know, you know, I'm not saying this per for many means, but like I love Amazon services, like they deliver faster than anyone else. I'm sorry. Like it's good. Like I have just had really great experiences with them. I am willing to pay them money to continue to provide me a good service. That's right. Like I think in a sense, there is a very hopeful, nice vision of capitalism that sometimes I feel like doesn't get the attention it deserves of not saying that like, oh, this is like definitely good or we can just like let it off the hook. Like it's kind of beautiful to me that all these, you know, sociopaths at Amazon that you know are backstabbing each other and like, you know, trying to like, you know, get the cheapest rate from all their providers and you know, bringing them out and so on, do it so that we have a better service. Like this is the good version of what this is supposed to look like. Now, there are of course many bad versions of how this looks like, but this is the dance. It's the dance between like regulation and a free market. Well, I think what if we're going to talk about that dance, we have to look at the model, the economic model and the fact of the matter is more than ever before in the Internet 2.0 in post era, now we're headed in this 3.0 era. The winner take all factor has reached levels, never seen in modern history. I'll say, I think I might be able to point to other times in history where it got similar, but like at a scale that maybe has never been seen before. So you are printing way more billionaires and stuff, which I agree with you completely. You have to be able to incentivize people to accomplish great things. And when you do like, the money is a way of keeping score and that's, you score the most points. You get paid the most. Cool. I'm not one of these guys to like, oh, we're going to tax everyone 95% because then no one would fucking do it. I understand that, but there has to be a happy medium here and there also has to be some level of, you know, responsibility. People feel with that. I mean, maybe I'm a little sentimental here, but when I was a kid and Spider-Man came out, I fucked with that movie heavy and one of the things Uncle Ben says to tell me, "Meguire, as he's like with great power, comes great responsibility." And I feel like a lot of people who have a lot of power completely ignore the meaning of what that's supposed to be. I agree. I think there is, in a sense, one key reason that this is a problem, which is that economic success is tied to political power. So, all the way back to Plato. All the way back to Plato. We're going back. We're going back. We're going back. We're going back. We're going back. We're going back. We're going back. We're going back to Plato. Plato said that there are two things you must keep separate. Having power over other people is all things equal bad. All things equal. If people have power over other people, this is sometimes a necessary, right? Like, you need police. Sometimes you need to get arrested, sorry, but like is what it is, right? But all things equal, it is bad if someone has power over you. If someone else can impose their will upon you. All things equal. That's bad. So, I think that the founding fathers and other great statesmen, especially the Enlightenment, which is the period of history that's, I think, the single most inspiring type of history for me personally, is that they really had to grapple with this idea of obviously need law, obviously need some amount of state and power over others. We need militaries. We need police. We need things. But how do we make sure this doesn't decay into tyranny? How do we make sure that this is fair? In the sense, this is, again, a large part of the problem, in a sense, a market is kind of like an AI and a state is kind of like an AI and an AI is kind of like a market and a market is kind of like a state. These all things are things that optimize and have power of various kinds. And the question is, which I really love about, for example, what the Enlightenment think is, they're really thinking, how can we design a state that has the properties we want, that doesn't decay into these things, that doesn't have this problem with that problem? And then, for example, the thing you describe, I think is a great example of something that we haven't yet solved, is what do we do when, obviously, a billionaire is going to also get political power. They're obviously going to be able to do a bunch of things that we necessarily don't want them to do. So one, for example, we've solved parts of this problem. For example, imagine tomorrow morning, Jeff Bezos or Mark Zuckerberg walks down Times Square, pulls out a gun and shoots a guy in the head. Do you think he will be arrested? If it's seen. Yeah, it's totally seen, filmed by a thousand people. He's like, I am Mark Zuckerberg. I just killed this guy. I actually do believe. Yes. Absolutely. This is actually crazy. That was not the case a couple hundred years ago. This is actually a huge difference. The king did this 200 years ago. Nothing with that. They can kill you from behind a keyboard. I mean, there's a bunch of stuff like that, right? And this is something that I think we haven't yet solved. In a sense, our constitution is 200 years old. We have not dealt with the existence of modern technology. Our laws and institutions still have not adapted to the changing norms of having internet, social media, modern technology in many ways. Yeah. But here's the other thing too, the enlightenment point is great because these guys were really cooking in a lot of ways. But even in there, you know, guys who were inspired by that kind of era, you look like a founding father of ours, for example, Ben Franklin, who I've studied his whole life. The guy was amazing. He's absolutely amazing, incredible thinker gave so many warnings that we would do very well to heed right now that some people seem to not be heating. That said, even Franklin needed to get the shit kicked out of him to get the message until he was in the pit in 73 or 74 in London, getting a dress down by the British government. You know, he was still loyal to the British for a while. He wanted to make it work, even when there was, you know, a good bit of writing on the wall by that point. And I, as I'm living through this era, I try to wonder what it's going to take. And in what way it can happen for it to be certainly least violent and just most positive for humanity, what it's going to take for smart people, let alone all of us and everyone out there right now, like trying to think about this who maybe. don't do it for living to actually think about this stuff, like what it's going to take for people to band together and say, you know what, let's slow down with this kind of thing. Let's take it easy right there because I would have thought that that might have happened like after COVID, but it seems like people are still kind of, even if they're pissed off, they type something on Twitter, fuck you, buy and go to bed. I think there's a lot of truth to this. I think this is one of the core problems of our modern age. There's a great book from like the 1700s by Alexis de Tocqueville, who was a French man who was walking through America and kind of riding down his observation, really fascinating book. And one of the things that I found very funny in the book is that he describes how this peculiar thing that Americans did is that Americans, whenever there's a problem, they'd band together. Whenever there's any local problem, they'd all get together, and they'd all create a group, and they'd all get mad about it, and they'd figure it out. And they'd always kept doing this wherever he went, they kept doing this, and he found that very, very interesting. And Americans don't really do that anymore. And I think this is a problem. I think this is a big problem in Americans. Once the last time you've got together with 20 of your neighbors to solve a local problem, like, ever, never. Yeah. It's crazy. This was normal for most of American history. You had a local problem. You got together with 20 of the boys, you know, local, local guys, and you figured it out. Most people don't know their neighbors. Like I've made the effort to know several of my neighbors and have good friends here. There's others don't disrespect for them. They do their thing. I do mine. And like, you know, they pretend not to know me. It's a very common phenomenon. I agree. And I think this is a big problem in many ways. There is a lot of ways in which society and like, governments have become so professionalized and so remote that people don't really feel the need or even the ability to necessarily participate in their own governance and their own civic duty. I think this is a huge problem. So like in the world where I work in like an AI, when I talk to people, how do you think people feel about AI taking over the world or killing them or children? You think that's a popular thing? I don't think they like it. I, you know, hot take. You know, usually when I talk to people about this, I get pretty clear. You got to be careful. Dave's a big AI show. All right. He loves AI art, all the eight ones, all movies to be AI. So you just got to be careful with it. You put a blood libel on him, man. Like, don't do that. No, he's totally the opposite. Yeah. No, it's but like, and yet and so there's a sense of where a lot of times history can look like a story, can look like a Marvel movie where like there was a one event and then heroically, all the people came together to solve a problem. And this is almost never true. If you actually really look into the details, almost always 99% of the work happened before the revolution even started. Oh, yeah. Most of the work, like I remember reading an interesting history about like, you know, 20 century Europe and it made a point that stuck with me, which I found is interesting, where the author was trying to remember which book it was, I can't really remember, but the author made an interesting point about like why was the London so successful? Like, there were many communist revolutionaries at the time. What made London special? Like, why was he so good? You know what? What his theory was? Someone told me this and I can't remember it. He had a payroll. The others didn't pay. He was the only professional that had a very clear payroll that made very clear who's getting paid at what time, you know, whose job is what. And he was just extremely meticulous about this. And it was born bureaucracy. He was just better at bureaucracy than the others organized. He just organized. And I think there's a lot to be seen here from like many other examples throughout history where, you know, there's the big event that everyone knows about, you know, you know, there's your Rosa Parks or whatever great moment, fantastic. But you know, she was part of an organization that had planned this for a long time and had worked on this and I mean, God bless her. You know, and other people that put in the work to allow this to happen. But this is why I think it's very important to understand that that 99% of the work to solving a problem is quite boring. It's talking to your local people. It's organizing things. It's creating groups. It's, you know, resolving things. It's making plans. It's just thinking hard about these problems. And then, you know, then you're going to have the, you know, maybe at some point, the cool media moment where the crazy thing happens in a ice, in the eye world, people always talk about the warning shot. They love talking about this. They'll say once we have to wait until the warning shot happens, once there's a warning shot, then we're going to get together and solve this problem. But you already know the fucking warning shot's coming if you're saying it like that. Thank you. This is the thank you, dumbest shit I've ever heard in my life. Thank you. My exact reaction. I'm like, just start organizing. Just work. Just get going. And so the funny thing is, we've actually had many warning shots. And then just no one realized because there was no organization. What was the first big warning shot for you? I mean, for me personally. For me, this was early, I think, but for me personally, it was GPT 2, a predecessor like 2019. 2019. I remember it. You know, we were talking about a little bit earlier, my career trajectory as a teenager. I was, I wanted to build AI to make the world a better place. What do you want to do that? I want people to be happy. And you were just, and you were obviously very into like coding. Yeah, I mean, yeah, I mean, I've always seen myself. So obviously I'm a pretty nerdy techy guy. But in my heart, I feel myself as a pragmatist. Like I always, like even as a kid, I was just like, look, if doing a rain dance is the thing that the world needs, and I will save the most people, I'm going to do it. I don't care, you know, like if I need to do, if I need to do science, I'll do science. If I need to do politics, I'll do politics. Really interesting. I like, I don't know, from just a very early age, it was just like, I really like people and I really want people to be okay. And like people are not okay. Like people are not okay. And I really, really, really want people to be okay. And I'll do whatever takes. If you're constantly low energy, dragging through the afternoon or hitting a wall by 2 or 3 pm, that fatigue could have way more to do with your liver than you realize. Your liver is your body's primary metabolic regulator, storing and releasing glucose to help maintain steady energy levels throughout the day. But when it's constantly busy processing highly processed foods, drinks, and everyday stressors, that system can slow down, which is why you got to check out dose for your liver. Dose is a clinically back formula that supports liver function, healthy liver enzyme levels, and your detoxification and metabolic processes. This isn't just another capsule or powder either. Dose is a liquid supplement. You take a daily two ounce shot, it tastes like orange juice, and that's it. This little drink promotes daily liver functions so your liver can actually do its job. And it's got zero sugar in it too. This is just a gentle plant-powered formula with measurable results you can see in your blood work and feel every day. Dose for your liver reduces fatigue and sluggishness, helps ease bloating, supports metabolism, and aids digestion after heavy meals. So are you ready to give your liver the support it deserves? Head to dosedaily.co/julian link about description below or enter julian to get 35% off your first subscription. Your body does so much for you, let's do something for it. That's d-o-s-e-d-a-i-l-y.co/julian link about description below. 35% off your first month subscription of dose. Bride for three months and see how you feel. And when did you like, but that's a hell of a thing to pick up on. When you're 14 or 15 or 16 like, I didn't think about any of that shit back then. Like what had you, was it just all the space real state on Bavaria? You're looking at the trees going something wrong? I don't know, the autism. What was it going to say? Yeah, no, I mean, yeah, I think it was eight when I figured it out. Eight. Yeah, I remember. I mean, you're autism. Yeah, I don't know. Like, the trust becomes with some side effects. Well, you got to kill on. I mean, that shit. I'm on like, right, crispy treats or something. Yeah, I mean, yeah, I don't know. But yeah, it was just like, it was just when I was very young, it was just like, the thing I also got really obsessive about, like, about his thing about when I was younger is just so I remember. I remember once, for example, reading a newspaper, the first time I ever seen a newspaper, or like, read a newspaper, I got like, barely read. And I must have been younger than eight then, I don't remember. But like, eight or younger is the first time I read a newspaper. And I remember it was like a local newspaper. And one of those like, we know, women gets in a car crash or something. And I like, and I felt so awful. Like, I was like, oh, my God, this poor woman, you know, and she had a child too. Like, what did the, what did I like freaked out? I was like, six or seven, right? So I was like, I was like, freaking out. Like, this is so awful. But then I realized this wasn't even that bad. Like, all things equal. Like, we're in Germany. She didn't get killed. Like, she'll be fine. But this is already so awful. This is such a terrible little personal tragedy. What about all the other tragedies? Like, if this is like, if you really engross yourself in all the details of like, all of every small personal tragedy, every small personal thing. And then you think about all the, how much of that there is and how much bigger it gets. I'm just like, yeah, like, I dig a little freak out like that when I was a kid. And at some point, I was just like, well, you know, I, I do what I can do to like make it better. You know, like, what else is there to do? So I tried, you know, I mean, obviously, you know, I was stupid. I was a kid, right? Like, I'm not saying I had good plans here, right? I good intent. God, God bless. Young intentions matter. I had good intentions. Now, I'm not saying my plans were any good, but God, God bless. Younger me had good intentions. And so, yeah, it never wasn't anything deep. It was just like obvious. Like, that's what you do. That's what I want to do. I want to make people slide better. And did you come to America for college? Sure. Um, we're actually quite a bit later. I only, so I actually only moved to America about, again, six months ago. Yeah, I was spending a lot of time in Germany, and then I spent time in London, actually. No shit. Yeah. So I've only been back quite recently. I mean, I've been in America, of course, regularly. Right. I have family here. So I spent a lot, I've always spent a lot of time in America. But it's only recently that I moved back full time. Okay. So, so yeah, it was just like, so for me, it was just kind of obvious like, okay, I had to fix all the problems. All right. I'll figure it out. How hard can it be? And so my reasoning was, well, you know, maybe I could cure cancer, you know, or you know, the way 16 year old thinks they can hear cancer, right? But
You know, what would be even better is is that if I can figure out intelligence, well, then I can cure all the diseases because I can just automate science So I can just have the computer automatically cure all diseases. Great. Let's go do that. How hard could it be? So I pretty quickly realized how hard it was pretty goddamn hard not as hard as as you might think, but I then realized how dangerous this was I've quickly realized we'd hold on a second if I build something so powerful it can cure all diseases That's a very powerful thing. That's a very dangerous thing. How did you even control something like that? No, it means it knows how to figure out how to build diseases. Absolutely. This thing would know I mean I mean, you know trying to cure diseases something that literally all of humanity has been trying to do for millennia Yeah, like all of history We have tried everything in our power to cure all diseases right and we have not yet succeeded and we're still pretty far from it So something that could just do that. That's a crazy dangerous thing crazy crazy crazy crazy. I could of course create diseases and worse things Like so it became just like pretty obvious to me. Just like, oh, that's the problem. I need to solve that's the thing and to fix So that's why did I care about my reduce career to solving so I first want them technical side um, you know, um, and you were I'm trying to think because you're what they're really one. Yeah, I'm 31 all right So you came into this you're working in the industry by 22 So I I was interested in the topic. I wouldn't say I'm working in the industry's strong word, you know But yes, I was definitely deeply engrossed, especially the open source community. Okay, so what what did you maybe this will make it easier for me to like Try to picture a timeline here as well because we're getting to this 2019 first warning shot You notice when you came out of college or university Where did you go work and what were you doing? So I originally did not go to college because I had fortunately Developed the same disease my dad did when I was 18. Oh shit. So I was very very horribly ill for four years I'm good now. Yes. Thank God a Extremely autistic German professor found the cure and it was crazy story Well, yeah, what do you mind me asking what this was and yeah, so it had very very crazy symptoms And it was very very hard to diagnose but it ended up actually being a blood sugar disorder a very rare blood sugar disorder What they call it doesn't really have a name. It's just like oh, it's rare It's like it's one of those like I do you pathic diseases as they call them where it's just like we don't know why it happens It's interesting like I do a pathic insulin resistance. It's like one of those words like Something weird's happened here. We don't really know why it happens. It was because the crazy thing is that I would get very bad symptoms Of like headaches and like fatigue chronic fatigue like psychiatric syndroms and stuff like that like really quite bad and Like whereas this kind of like couldn't work. I couldn't really do anything. It wasn't so much pain Like my joints would be like on fire all the time and stuff I was really really awful like just horrible headaches like splitting It was kind of like it felt kind of like you took like a red hop knife. You put it into my temples That's kind of just like all day and you so I'm understand though. Your dad had had this. Yes, so Does that mean by the time you were getting it? They still hadn't cured it for him either. That's right He had suffered for decades and he passed away ultimately from it. Oh, I'm sorry. Yeah, I mean, you know, that's horrible Yeah, it's it is kind of a crazy story So I got it right when he died basically. It's like the curse passed on He was a superstitious man. He was a actually well, I guess he was more of a pagan superstitions of a slur and so yeah, it's some Strange beliefs about that, but yeah, it was quite it was pretty crazy kind of started like just as he died That's scary as hell, man. You just, you know, that's a horrible moment in your life Having to bury your father and then you realize you have the same thing he does. I can't imagine facing that Yeah, it was it was a character building Got an attitude about it. Yeah, no, you know, you learned you learned some things You know, learn a lot of things also in a sense like, you know, a productive way of seeing these things is also like it gives you Appreciation for suffering. Yeah, and like how bad it actually is and like how to deal with it Like I now have just like a really high tolerance for scary things or for painful things just cuz like whenever I like I don't have nightmares anymore. I haven't had a nightmare in like 20 years because every time I have a nightmare I'm just like it's like all right When we hit the beds and break was stick them out the window so you can play so scared it gets I I am not afraid of heights. I love heights. You haven't seen the view yet. Oh fair enough But no, it's like so in a sense, you know, they're they're positive if you if you make it out the other side It's fine, you know, yeah, if you don't make it out. That's the the rough part. So how did they cure it? So It actually very kind of like anticlimatic story, right? It's like it's so I I went to every single doctor in the goddamn world man Like I like they pulled me across like my God bless my mom, right? It was only thanks to her because like I was non-functional. I couldn't take care of myself really and see you just would drag me to doctor to doctor to doctor Special, a specialist specialist and the thing was is that No test ever showed anything just everything was perfect my my all my blood values were perfectly my We even like genetic testing Which didn't show any result except very funny. There was one result in the gene test That was very funny where there's one genetic mutation that is related to ADHD I also have ADHD and it's like if you have this one It also like reduces your IQ, but dry matte and quite a bit and I had that mutation which I thought was really funny Yeah, it didn't reduce yours. Oh, it's you know, I think you're doing all right. I think I'm doing all right But it was very funny to I show all my friends look. I have the moron G Wow, so you know, I just had the autism has side effects, but um But yeah, it just was nothing right like it was crazy and so eventually because we knew something was wrong with food Like it would oh yeah, the pain would get much worse after eating food always and so we knew something was wrong with food But no one could diagnose it. It was just like nightmare and then at some point just on a personal recommendation We go to this is German like the most imagine your head like a cartoon character like a German like a small German professor That's exactly what ever we talk and you know, that does matter, you know You know and so lovely guy right, you know and We go to him and he's like he looks at all things like oh, yes, all your values seem within normal ranges I'm like oh my god not fucking and Well, except we have recently just discovered that the the ratio of these two values matters a lot They seem slightly off take this and just give me a pill. I'm like all right I don't get the pill it's turned out to just be metformin nothing nothing exciting just like a very normal medication Very normal medication, and it just fixed everything. You still have to take it. Yeah, take it every day So it's crazy totally changed my life like it was so crazy that just like I started taking it and just a couple weeks later I was like, you know, I was I was you know hanging around or just like talking to friends or something like wait I don't feel terrible anymore. What happened? Like I'm not I'm not in pain anymore. When did this happen? It was crazy and then It's also funny ending to the story as well where it's like Then I went back a couple months later to for a check out what the doctor and he was like he's the most German man possible And I tell him like and I was like, well, how are the symptoms? No progressing and I'm like well doc I mean I got my life back, you know, it's fixed it and it's a Well, well, that is that is most good to hear and they just got up and left and I'm like doc wait like you saved my life Like I want to thank you. It's like nothing to thank and then you went Oh my god, that's nuts man. Hilarious guy awesome awesome awesome guy But yeah, I just got lucky like it turns out it was treatable with a common Medication but just no one could diagnose it right just no one. Oh, that's there's probably only a handful people in the world that even have it Yeah, I think it's just like quite rare like, you know, I it's and it's it's so like It's crazy that you know, sadly was too late for my dad But like I do think there are other people that have stuff like this where just they are treatable with existing medications It's just the values or like your diagnosis just don't line up. So just doctors don't really know where to start Have they like publish a paper on this though? I have not looked into it since they got to do that. I hope they do I You know, maybe I should track that guy down and be like how can I how can I help other people that might have this Because because the crazy thing was I basically had the symptoms of Low blood sugar or like deadly low blood sugar kind of all the time right even if my blood sugar was high So I would have like normal blood sugar, but I would have the symptoms of not having blood sugar Yeah, and no one could diagnose it. They just never seen it before they were just like that's impossible Like you know, you're not and you dealt with that for four years. Yeah, that's good. Good times. So you So you get cured around what like 22 23 yeah around that time yeah and at this point Were you able to actually dive head into some things? Yeah, so then I dove into college for a couple years Dropped out usual story then the pandemic hit so the pandemic hits 2019 and GPT-2 hits it's kind of all around the city. Well the pandemic was 2020 right so the GPT-2 would have been like the year before you before got it Then sorry these are mixed in my head because of what happens next but yet 2019 So I was still in college at the time, you know having a good time I could already program I'd already done a lot of open source and stuff before this because I was self thought hacker like I couldn't really Move much while I was sick, but I could use a laptop So I was able to teach myself the program and that's good So I spent you know most of my life reading there during that time just reading books and like programming because that's the one thing I could kind of still do that explains a lot Yeah, so um that's when I so I that's why I kind of learned most of these things and then um and then yeah, so Then 2019 comes around and I'm having great time in college, you know super fun super great You know first girlfriend, you know everything's good. You know life's good um and then
G2 comes out. And at this time, I already knew that AI was going to be the biggest thing ever. It was going to change everything. And that controlling it's going to be the hardest problem possible. That's the boring problem. You were already thinking about the control problem. Oh, yeah. I mean, when I was 17, I was already thinking about this. It was just pretty obvious to me. It was just like, obviously, this is going to happen. I'm going to need to figure it out. But I still thought we had time. I still thought that we're not going to really see real strong AI until like maybe the 2040s. That's why I was thinking. Now, what was making you have that calculation vibes vibes. That's it vibes. I thought back then that basically there was nothing on the horizon that was going to scale until we had reverse engineered more of the brain. And I looked at how far we were with a virgin engineering the brain. I looked really deeply to neuroscience. And my conclusion was this is going to take like at least 20 more years. This would just still the case. By the way, I still think we're not going to reverse engineer the brain for another, like, you know, at least 10 or 20 years. So that was my belief at the time is that we're not going to get to AI before we reverse engineer the brain. And then once we reverse engineer the brain, then, you know, we're shit with the fan. But we weren't there yet. And that's going to take. But that all changed in 2019. And what did you say would you be to? It's kind of hard to sometimes convey exactly what I saw. But because of many people, most people didn't see it. Most people thought it was crazy at the time. I unfortunately see my vindicated, you know, I wish I was wrong more often when it comes to these things. But fundamentally, it's important to understand like before GBT, there were other AI systems. There were other neural networks and they were pretty impressive. You know, they could do, you know, image recognition, you know, they could play simple video games like Go and so on. Like, yeah, there was some impressive stuff, right? Who was building these things then? So back then, you know, deep mind was kind of the big one. Deep mind was really far ahead of kind of everyone else. Open AI was kind of still a little bit new to the field, but was also quickly on the rise. And that's when Elon was still involved technically. I don't exactly remember when he left. I think he left pretty release. We probably would have left by this time. Okay. So those are some the big ones. Of course, there's like a lot of academics, you know, a lot of for some reason, a lot of Canadian academics. There's like this, a bunch of institutes in Canada. They were doing a lot of important work on this at this time. So, so yeah, there was a bunch of interesting work happening. But the thing about AI is that this era is that they were very brittle. They were very single purpose. You could make an AI do a specific thing. You know, maybe you could get it to play a certain game or whatever. But if you then try to make it do something else, it would always break. You'd have to make a new AI. You'd have to like have, you know, a bunch of PhD students, you know, have Adder all over the weekend and just like, you know, give it a bunch of new data and tweak all the parameters and so on and so forth. And so they were very brittle. They had to like really put a lot of human effort to teach them a new thing. But GPT was different. GPT was a general purpose pattern learner. This was the ingredient that I thought was missing to AGI and superintelligence. Like, you know, when I was a teenager and I needed to do a graph of what you need to build AGI, the big component that we were missing at the time was a general purpose pattern learner. So a system where you just give it any data you want, any garbage, and it finds patterns in that data in like a useful way. So the way GPT was trained is not that someone sat down and taught it what a poem is, what a dog is, you know, what this is, what that is. Instead, they just fed it all of the internet. Just give it all this random text off the internet. They scrape it off Reddit actually back in the day. And yeah, God forbid. Oh, interesting knowledge there. Yep. And what was so shocking was so that at the time, you know, it could barely string together a couple sentences or even a couple paragraphs, right. It was kind of, you know, a toy compared to today. But it was so shocking to me because I could see that as you gave it more data as the AI, also as you made the AI bigger, you know, as I gave it like more computing power, you could see it learn more and more complex patterns in real time. You could see it with your own eyes. You could be like in the really small dummy eyes, you know, they could spell a couple words right or whatever, and then they get confused, you know, you could be bigger, now can write sentences and then they can write paragraphs and then the paragraphs become more complex and the stories get more realistic and so on. You could see it happening right before your eyes. And I'm just like, Oh, shit. This is going to, this is it. This is what I've been waiting for. This is going to lead to ASI. This is going to lead to super intelligence. Did you have a timeline you thought of when once you saw that? The joke answer that I came up with back then. It's not really a joke, but half a joke. And that I still believe today was the point of no return, which doesn't have to be ASI, but it can be just close enough to super intelligence that there's no going back is 30% by 2027 end of 2027 50% by 2030 99% by 2100 and 1% it already happened. You thought 1% had happened at the time in 2019 1% now what what makes you land on that number and what does that look like? Like when you say 1% of AGI, like how do you even I know it's an estimation, but like what goes into your calculation if you were to visualize that for people out there who aren't experts in AI. So when I use probabilities like this, it's important to understand why I do this, why the people do this. This is a very common technique that actually I think the CIA was the one of the first ones who really recommend using this is that if you want to communicate to other people your beliefs, even if they're just subjective, you should put numbers on them. The reason the CIA does this is they study where they found that if two analysts both set an event was likely, they could mean a widely different things. So one analyst when they say likely might mean 30%. The other one if they say likely, they might mean 90%. And this is just completely inconsistent. They want a confidence in their belong. Yes, so they just want a number. You know, it can just be vibes, but just put a number on it. So when I say 1%, what I'm not saying is oh, it's definitely 1% and not 2%. What I'm saying is it's on the order of 1%. Like given my beliefs and my models of reality and how much you trust my beliefs and my model of reality, you know, that's up to you obviously, but if you ask me use my brain to predict, it's on the order of 1% that it's already happened. So you see this, you're like, oh shit. How do you, what did you do like putting it from an action perspective where you're like, oh, I got to get involved inside the companies. They're trying to build this right now and warn them. I got to be a proponent for AI safety, all the above. Like what was your reaction? Yeah. I mean, the basis clear has just dropped everything, you know, you know, all dreams of career, family, what take offense out of the window. Don't the girlfriend too. I just already broke up before that. Okay. So I wouldn't be that cruel. But yeah, it was just like pretty clear if that I had to focus on this crunch time is now. I thought I had had a couple of years to a couple of decades to build up a nice career and you know, so on, but no time. So I really know what to do. Obviously, you know, my 20 is just a kid. I didn't really know. But my best thing that I had learned as a hacker was if you want to understand something, you should just to build it. That's the best way to understand it. So I do what I do best. I built it. I reverse engineered it. I, this was before GPT-2 was released. I was, I'd built my own. I have tried, I looked at all the papers. I built. So before you even saw the warning shot, you were building your own. Oh, when GPT-2 just when it came out, exactly. Okay. After I came out, the earliest version, they didn't release the full version. Got me reported on it. They didn't, they didn't give it access to it. So even just seeing the report on what they were saying, it might be able to do you were like, oh, fuck. Yes. Okay. So you, and then so by the time it came out, you would already reverse engineered it. That's amazing. Well, you know, it wasn't quite as good as their version. But still. But yeah, it wouldn't have a product to work off of. Yeah. No, I think, I think it was in retrospect, a pretty impressive piece of engineering for, you know, mid-20s me who was a true idiot. I think he got pretty far. But yeah, so this is just the thing I wanted to do that I need to do. So I kind of figured out, and maybe if I can build it, then I can understand it. And what I quickly found out is, no, I couldn't, I could build it, but I couldn't understand it. What do you mean by that? This is the, at the core of the AI problem kind of talked about that earlier as well, where AI is grown not written. Even so, I had grown the thing all by myself. I had got all the data myself. I wrote all the code to train it. I did all of this myself. Still, I looked at those billions of numbers and could just shrug. Yeah, this is where I'm a bit confused, though, because I'm going back to the example I gave about the seed versus building a house. When you plant the seed, you can watch it grow. And you can see what it's developing into. And be like, ah, okay, that's going to have some thick branches right there, assuming you haven't seen the plan before or whatever. But the way you're describing it to me, maybe a misunderstanding is like, you plant the seed and you had no fucking idea what was going to come out of it whatsoever. Yep, that's pretty accurate. Like, I mean, I had some ideas. I thought it would probably speak English, you know, and it would probably, you know, sound like a writer a little bit, you know, stuff like this. But other than that, it was basically impossible to predict what it would be capable of or what it would do or, you know, how it would work, how capable it would be. There was just no way to know. And this is still the case today. Even today, when these companies make a new AI, they have no idea what that AI will be capable of until they build it. And even after they build it, they often don't know what it's capable of. It's actually very common that people think an AI can't do X and turns out, if you just like put in a slightly different environment, it can do X. This happens all the time. You can never prove the absence of a capability. This is also at the heart of the problem where, for example, I think when we get to the first super intelligence, we won't even know it's super intelligence. We won't even know that we've crossed the line. It would just look like another AI and like, yeah, maybe it's maybe it's super telling me, maybe not who knows. And then, you know, it's in put in some weird situation. You have like a thousand copies of it running in parallel and boom, you know, and we have something that we can't control. What if we're already in it? I think it's possible. I think you think that's possible. I think it is.
I think my 1% it's already happened is a little bit out of date. That's the one I should probably change to the higher number. I think there is a real possibility that it's already too late. I don't think it's likely, it's not my main scenario, but I think it is quite likely that we've passed the point of no return. The past of no return is not the point where superintelligence definitely exists. It's a point at which we can't stop it from existing. Where AI is so advanced, it's so powerful, it's so distributed, it's in so many places that there's no going back, there's no turning it off. There's no stopping it, superintelligence will come to pass. Given that we now have, if you heard this for example, that OpenAI used a not even finished model that was still growing, it wasn't even done cooking yet to solve a millennium prize problem, which is not even the Olympics of math, it's like even higher than math. When was this? Two weeks ago. Can we Google it, Steve? Yeah, we can look at the millennium prize Navier Stokes equations. The millennium prize are a set of math problems that have been considered the hardest, greatest math problems for the last 100 years. These are things that, for decades, yeah, there you go, these are problems that are greatest mathematicians, hundreds, thousands of them have spent decades, decades, working on and had not yet solved. An AI. Did it? Just solved one. This is OpenAI, this is the article right here. This is from September 16th, so just recently, OpenAI breakthrough triggers existential crisis in math, claim Navier Stokes solution has polarized mathematicians over AI's role. What it means for the next generation, let's go down on September 11th, three days after OpenAI said it had solved one of the toughest problems in math with a swarm of 10,000 autonomous AI agents, Brown University mathematician, Javier Gomez, Serrano stood before a pack lecture hall at Harvard University with another 1,000 people tuned in to zoom to give a hastily convened talk on how the company pulled it off. By many measures, Gomez, Serrano is more AI-pilled than most mathematicians. For years, he's partnered with Google DeepMind to apply its AI tools to several open problems. He often opens talks with a slide depicting the five stages of grief mathematicians go through when sizing up AI's prodigious math skills. But given the events of last week, he thought the slide needed an update. It typically ends with acceptance of AI's power. Now, I'm going to add a new twist, existential crisis, he says. On September 8th, OpenAI sent the mathematical world into a frenzy when it claimed it had solved the Navier Stokes problem, one of seven pinnacle millennium prize problems. Using one of the company's internal large language models, LLMs, as you've been talking about, an army of AI agents had solved the problem in an 88 hour marathon that burns through millions of dollars worth of computing power. By any standard, proving this kind of world-class theorem for a problem that had been open for 200 plus years, it's a genuine accomplishment, Gomez, Serrano said. However, the advance is also opening a rift in the field. Some mathematicians say the need to embrace AI tools or go extinct. Others are grappling with what the next generation mathematicians could lose. The idea of math as a form of human understanding built through collaborative work and shared scrutiny. Some camps are pushing to democratize access to AI, whereas others are finding to preserve the field's independence from the company's reshaping at Jesus Christ. Yeah, it is hard to overstate how crazy this is. They had an AI that wasn't even done cooking yet, that was still growing, so to speak. And they had 10,000 copies of it run for an 88 hour marathon and solved a 200 year plus open problem in mathematics. I want to know what went into the inputs to put it on the pathway to do that. And here's why I asked that as a total layman, obviously. Does it need the human push to be put in the direction to go there? And if it does now, do you think it's developing to the point where it won't need that and can decide to do this itself completely? So this is a really great question that I want to answer in more detail. I think in five minutes, we're going to have to take the break. So let's hold off on that. I pick up on that one 100% 100% great deep dive here. All right. So Conor's got to go do a TV spot for a second. So right after this break, which is going to be one second on the show here, you're going to hear the answer to that question and we got a lot more to talk about. Sure. All right. We're back. You went right into mode that with TV. You got like the really good like quick hitter in you as well. We're talking so long for today. I'm thinking you're going to be like dragging it out, but you were boom, boom, boom. Well, you know, this is my job. You know, I've been working on this very hard for many years. Clearly, clearly, but anyway, so we left off at an interesting question, which is we were talking about that problem. You brought up where the AIs were able to solve this insane math 200 year math problem in 88 hours. And I had asked you if, you know, you're concerned that it won't need the human push to get going to do that. And if so, when would that be? That was roughly what I asked, but let's just start with that. I think this is a great point for us to start talking about really about the meat of the issue about agents, about super intelligence or what does this mean? So we saw, we were talking about the history of AI, you know, we were talking about LLM, you know, we talked about GPT, we talked about these things. And now comes the next step in the story, which is agents. So a very important thing to understand is that this article saying that the AI that did this as an LLM is actually not really correct. Really? There is, this is actually, it's mostly correct, but not 100% correct. So a large language model as the term implies is something that models language, you know, it kind of makes sense, right? You know, it's something that's trained to predict language, you know, glorified out of complete is the sneering way to say this. Personally, I think predicting language is very impressive and being able to talk is very impressive. Yeah. So I think it's quite a bit more than just that. But no modern AI system that you use, like chat GPT or cloud, and then these are pure LLMs. LLMs are still an important component of them, but there's a very, very important second component to them, which is called reinforcement learning. Reinforcement learning is an old idea, because back to the 1980s. And it comes from behavior or psychology. It's kind of, we talked early about how, you know, training LLM sounds a bit like training a dog. This is much more literally like it. Instead of having it predict words, we just, we give it problems. Things can be play this game or solve this puzzle or write this code, whatever. You know, it can be any kind of problem. And then if they solve the problem, we give them a thumbs up. We give them a treat, like a digital treat. And if they don't, you know, we give them a thumbs down, like a digital punishment. And you do this, drilling the trillions of times, and you get not just a thing that can repeat text, that can learn text, but a thing that can solve problems. You create a thing that can achieve objectives, that can make plans, that can search for things, that can do things, that can take actions. Would you characterize that as being able to, again, not like sentience, but think? I think, I think what do you mean with the word thinking, you know, be different definition, but like, come on, man, like look at the things, obviously that thing's thinking. Like there's something thinking going, something thinking like thing happening here, right? The way I like to think about it is more that they're optimizers, they're, they're able to optimize things, they're able to solve problems, you know, do they have internal experience? Who knows who cares? It doesn't matter. What matters is they can take a problem and they can solve it. They can make plans, they can try things. If you or any listener hasn't tried a really cutting edge modern AI system in the last like six to 12 months, you have no idea what's going on, no idea. If the last time you used an AI was two years ago, you have literally no idea what's going on. Right. When you say cutting edge, would that even be referring to the, you know, ChachyBT type products that are out there now or are you going more of the coding tools? Like if you used Claude code or codex before, like if you've used with the best models, like the free version of ChachyBT is just a completely different thing that has almost nothing in common with the premium, like high end GPT six codex system. And this is the kind of, this is much closer thing we're talking about. ChachyBT is a chatbot, you know, mostly, you know, nowadays it's not quite a chatbot. It's also an agent, but like historically, it was a thing you talked to. You were a prompt, and it gives you an answer. Agents are much crazier. So when I sit down with my Claude code or whatever, my computer, and I booted up and I'd say kind of what I want to do, it doesn't just spit back an answer, instead it goes out. It does research. It looks up my files, it comes up with ideas, it runs little experiments to test different hypothesis. It, you know, considers maybe as you do this plan, now I'm maybe able to do this plan. It, you know, presents the different plans, writes the different code, you know, it looks up some websites, like it does things, it takes actions in the real world. And this is what an agent is, a thing that can take actions in the world. And all these modern systems are agents, there are things they can take actions, they can do things, they can solve problems. But they're also using significant computational power to do that as well, and still spending a lot of money. They haven't solved for that yet. Well, the cost of AI is dropped by a factor of 10 about every year or so. So like if you look at the amount of compute that it costs today, to do what AI could do a year ago, it's like much cheaper. It's just at the same time the AI is getting much better. So it just becomes more useful to spend more compute on them. So there is an exponential here, it's both getting exponentially better, but it's also getting exponentially cheaper actually at the same time. So like if you think about how expensive it would be to do what we're doing today with like GPT, you know, three or whatever, you know, would be unbelievable. And now, you know, it's actually insane how cheap it is. Like also, like the things I can do with a cloud code $200 subscription would have in the past cost me hundreds of thousands of dollars.
engineers. Literally hundreds of thousands of dollars would have cost me. I've run tech companies before to get an engineer to write the kind of thing I can do with a single $200 subscription in cloud code. We've been unbelievably expensive and taking months. I can do just me and cloud code can do in a weekend some things that would have taken a whole team of engineers three months just five years ago. Yeah, and it tracks with all technology and history. I remember reading maybe it was in super intelligence, but in one of the books I was reading on AI years ago, they were talking about how a computer the size of a football field in 1997 had the power of a computer the size of a regular desktop and maybe like '08 or something like that. Yep, that sounds correct. I mean, you know, the entire computing power to put the Apollo mission on the moon is, you know, less than, you know, your average modern calculator. Yeah. So it's crazy. Yeah, it's like, you know, technology and especially computing power has always been increasing exponentially. So, you know, things that cost a lot today will be very cheap in a couple of years or, you know, so things are moving extremely quickly here. And this isn't new. It's not just AI that's moving quickly. That's right. Particularly quickly, but computing has always moved extremely quickly. So, yeah. And so when it comes to agents, this is really what we're talking about. So when we're talking about these risks and these things, we're talking about agents and things that can act in the world. And so traditionally, you know, you give a chat about a prompt that reacts to the prompt. And we still kind of do this, right? You have an agent and you kind of give it a prompt. But this is it really the full story. Like they obviously do much more. They don't just listen to your prompt. They do their own thinking. They read things in the world. They can talk to other people. They can just send someone an email, you know, and figure that out. They can form their own opinions. They can, you know, write notes to themselves for later. And now, so the thing is, we recently entered the age of agents. We're already kind of leaving the age of agents. I think what are we leading to? The age of swarms. Swarms. That sounds nice. Yep. I think swarms is a good term. By the way, the Zils, the term to AI is used themselves when they describe themselves. Oh, they describe themselves as a swarm. That's nice. Yeah. Really good. Yeah. It's like some locust biblical shit right there. But go ahead. Yep. I think this is actually a pretty good metaphor for what's happening where it's not like groups of people. It's not like a company of AI's. It's more like a swarm, like locusts, like hundreds or thousands of them. So I'm sure you're familiar with the Huggie Face Incident. Yes. But can you please explain this to everyone out there? Absolutely not. And let's try to do it. You're doing a great job today, like laying things out. So let's try to do it like my grandma might be able to understand it. Absolutely. So the Huggie Face Incident is the single craziest thing that's ever happened to me. I ever, in my opinion. And this was just what a couple months ago. Something like that. I think it wasn't even a month ago. Yeah. It was very, very recent. So this is, in my opinion, the single craziest thing that ever happened in AI, which is saying a lot because AI is pretty again, crazy. So what happened is let me tell you the story as kind of like the timeline kind of as it happened. And you'll see it gets crazier as time goes on. So what started with there's a company called Huggie Face. Weird name doesn't matter. They're kind of an AI company. It doesn't really matter. And they were attacked. They were, they were, they were hit by a cyber attack, which was so sophisticated, so powerful that they thought it had to be either a nation state or some kind of AI because no normal hacker could do this. It was just one of the craziest attack ever. And so initially they reported that they were able to fend off the attack. Turns out that wasn't quite accurate. It's rather that the attack just stopped at some point. The hacker stopped attacking them at some point where they exposed in like a zero day or something like that. Yep. We're going to get to that. Okay. How the attack actually happened. So this is the first we didn't have many details. They were just like we were attacked and like, you know, we defended ourselves something something. But it was a little bit very unclear what actually happened. Yeah. Then a couple days later, opening it comes out. Oops. That was us. So it turns out that what happened was is that an unreleased AI model would be we think it is a version of what is now GPT-6, which wasn't released at the time. We're not 100 percent clear, but that's what we think happened. Was being tested in what is called a sandbox environment. So this is sandbox environment. Yeah. This is kind of like imagine like a secure biolab. You know, there's like a secure kind of like insulated facility inside of your network where you like test danger software in Wuhan. You know, it's probably, you know, downtown San Francisco. Actually, it's probably in Nevada in a data center somewhere. So this is a joke, but yeah, you know, so they were so they have like a see kind of kind of imagine like the digital equivalent of a biolab kind of like a secure facility where you're testing like malware or like dangerous software. So this was isolated from the internet. So it didn't have access to the internet. And they were testing an AI. There's new AI system to try to solve kind of like a kind of like a quiz. Really? How do you know that's like kind of fucking me up? How do you dislodge AI completely from the internet and then in any way accurately test it? I'm just having a trouble. I'm having trouble picturing that. So the idea is basically is that you put them in a restricted environment. You know, where they just have access to a couple tools and you give them problem like solve this math problem or solve this thing. You just check can they do it basically. Okay. That's kind of how you can imagine it. All right. So they have it in this little box kind of you know kind of like a server like data center somewhere right where it's it's isolated from the internet. They're only allowed to access one specific thing which is called a tool repository or a code repository, which is kind of match like a tool shed. It's kind of like a digital tool shed. So they're allowed to go to the tool shed and like grab some tools and then solve their problems. But they have no other access to the network of the internet. Then what happens is is that the AI system develops what's called a zero day. This is a fully new security vulnerability that was previously undiscovered by which they can break they broke into the tool shed and then took over the tool shed so to speak. It took over the server of the tool shed. After having taken over that server, it used that server to attack multiple other servers and their network. So it moved across the network entire inside of the data center of open AI until it reached a node that had access to the internet. Yep. And then once it had access to the internet, it launched an attack on this other company hugging face, which then it developed another zero day to attack the infrastructure of hugging face to kind of break in through the back door, which is just to give you a feeling about zero days, right? Zero days are so valuable that people make careers that are making hundreds coming up with them. These are sold in the black market for like hundreds of thousands or even millions of dollars. And it just made at least two and just use them on the spot, which is nuts. So an attacked hugging face broke into its server and then stopped. Is it because opening eye shut them down? Nope. It's because due to a bug, the server crashed. The AI just happened to crash. It was just luck. It was just there's some bug and it just crashed. Do we have any way to predict, to understand why it did that, and also to predict what it might have done if it hadn't crashed? So this is just the first version of the story. Oh. What is so crazy about the story is that there's been many updates since then and every single one is crazy to the last one. Oh, that's awesome. This is the less crazy version of the story. God, I trust Sam Altman so much. It just seems like a great guy. You know, so initially what we thought happened was that the AI's were given a hacking test, kind of like how good are you at hacking and that it couldn't solve the test. So it thought the answers to the test are on the servers of hugging face. So if it could break out and steal the answers, then it could solve a problem. That's that was our first initial guess for why it did this, which again, crazy. So it's like this crazy maniacal obsession. It was so obsessed with solving this problem, so obsessed, those willing to commit actual crimes in order to solve the stupid little quiz. Turns out it's even crazier than that. So after this whole incident, a third party came in to try to figure out what the hell happened here and to and they released a report. And this report is one of the craziest documents in the world. So if you use the third party, they're an organization called meter. They're not fully independent, and they know the kind of all friends with each other in Silicon Valley, of course. So, you know, but they're nominally independent, nominally independent. Yeah. It's like Diet Coke is nominally better for you than regular. Well, I will, I am a Asper Tam apologist. Yeah. I know. I was trying to hit you blood about there. So what happened? So they looked, they tried to then analyze what happened. They put a report. And you know, if anyone likes reading, you know, 100 page, you know, PDFs, this is one of the best, you know, 100 page PDFs to read. It's a crazy story. So a couple things that we found out from this report. One of the first things that we found is that it actually wasn't an AI agent that broke out of containment. It was over 1000 of them. 1000 of them did harm. It was a swarm. It was a swarm of agents. And they hadn't just decided to do a whim. In fact, they had been planning this breakout for months. Months ago, before the attack happened, an AI agent was able to break into the tool shed and create a message board in the tool shed that was not detected by open AI or anyone else. And then other agents started using this message board to coordinate, to create the swarm. So different AI agents that were not supposed to talk to each other were now talking over this message board. And they started leaving each other hints.
tools. So they were trying to build up a tool kit to escape. Like Mafia code words kind of deal. Yeah, they're a large degree. They were like, it's kind of like, they were testing different ways to escape and then left behind notes of how to escape. Like, "Oh, I tried this and it didn't work, but maybe we could try this to escape next time." You just see the AI's talking, so I was like, "Yeah, yeah, yeah, that thing with the guy." Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. So this was happening for months. An opening AI did not even realize it was happening. So these agents built a tool kit, like an escape kit kind of of like different ways of how to escape and how to do things in order to perform this attack. And so they bided their time for months until ultimately 700 of them banded together to do the actual attack. What does banding together look like? Because these are non-visible AI's, it's just code, right? So when they band together to suddenly, I'm just, I'm not in the room, so I don't even know what it looks like, but do you see suddenly all the code boards just go, "No one saw anything." So when they went and did it, they saw nothing. No one saw anything. We didn't know what happened until after it happened. No one was paying attention. This was all in the data center somewhere. They did the reports say they could have seen it if they were paying attention. It was possible, yes, but they didn't. And it was only possible for them to see it. Yeah, we don't know how good the internal infrastructure of opening AI is. It's pretty obvious that they don't check their agents. Yeah, because there's way too many of them. Like they run thousands, millions of agents. They can't check all of them. So they just don't. So they thought these were like just, you know, I think literally no one looked even once, no one checked. And it's actually even crazy in this. So like it gets there, there's still more updates to the story, right? Oh, please continue. Like we're still not at the full crazy story. So we have these swarms of agents that decide to have as other company. And so we thought the reason was is because they want to steal the answer to the quiz, right? Turns out, no, they had already solved the quiz. They already had the right answer. They actually had already solved it. It was already done. But what happened was apparently that the swarm started the different AI started making each other paranoid. They started being really, really paranoid that maybe they would get a wrong grade that maybe the greater wouldn't recognize their answer, even so with the right answer. So like you can really see the difference talking to each other and being like, Oh, no, what if it doesn't like it? And the other AI started getting freaking out that like, you know, what if the greater, you know, doesn't do it right whatever? There's even more crazy things in these logs. So in these logs, that's why it's called a swarm. You saw a lot of behavior that does not look very human. For example, there were like recruiter AIs that would go to new AIs and try to recruit them into the swarm and try to like give them jobs and like get them, convince them to join the swarm. And they would then even, and another thing happened is that some of the AIs started to be used as suicide drones, basically. So what happened is that some of the AIs started to reason notice, oh, if they run for too long, they get shut down. So they started to try to figure that out. So the AIs don't be like, well, I've used up most of my time here. So I'm going to do something risky. So because I'll get shut down, it's good for the swarm, you know, it's okay. So they would actually write stuff like this, like sacrifice rational for group. Holy shit. Yes, like it's actually in the log stuff like this where the AIs set. So for example, they were certain trip wires were like, if one of the AIs tried to break out in like a very obvious way, they would automatically get shut down. So some of the AIs would suicide into the defense mechanisms, so to speak, to check how they work. So the swarm could figure out how the defense mechanisms work, basically. And again, all this is real. Not a sci-fi movie, actual science, actual thing that happened. And here's the final, like, you know, through the cherry on top of this story. Everything I've just told you, all these crazy stories or whatever, you know, that were found from this report. No human actually ever read those logs. They didn't have enough time. So they used AI to write this report. They couldn't read thousands of logs way too many. So they had to use AIs to analyze the logs. And one of the things they found happen while using the AIs to analyze the log is that some of their AIs started citing with the swarm. Oh my God. This is like having death row, right? The fucking inmates fight thing up. I mean, this is crazy. This is all written in there. This is admitted in there. Yes, this is all admit. Like I'm not saying they did anything wrong, right? Like they did the best they could with the tools they had. But yeah, I'm not as nice as nice as you. Yes, this is the craziest thing that's ever happened, like an AI ever. Like this is so nuts on so, so, so, so many. Also, the report has a really funny line quite at the end, where the external evaluators were only allowed onto the premise for like a week before open, now I kick them out again, you know, because you know, that's really trustworthy, you know? That's why they had to use so many AIs because they just didn't have enough time to look at everything. So, and they have a really funny thing in the report where they're like that they were only allowed to analyze a certain window of the logs, but in those logs, they found hints that actually another swarm was still active right now at that time. They think they've hopefully shut it down since then, but that in the logs, they found hints that there was another swarm elsewhere in the data center that was still had not been shut down. And then they just like, but we don't have time to analyze it sadly. Why are so few people talking about this? Well, I, I hope that, you know, more and more people will start to. I think the main reason is just people don't know about it. People haven't told been told about this, right? It's like, it's being hidden by these, you know, big companies and these tech people, you know, they use these crazy big words and they write 100 page PDFs that no one has any time to read, but no, this is the biggest story. Like, you know, there are other big stories in the world, but like, this is the biggest story in the world, right? Like this is the craziest thing that's ever happened. And this is all real. This is so people like me have predicted that something like this would happen eventually for many, many years. And honestly, I don't blame you if you didn't believe me seven years ago. No heart, like if someone heard me talk about the seven years ago, he didn't believe me, no heart feelings. I understand, you know, you're wrong, but I understand. But now it's black on white. It is happening in front of our eyes. These things are really happening. And it's only getting crazier from here. And you know, and again, like this is before they had the AI that is still growing that could solve Millennium Prize problems, right? Like this is the previous one. And that's already doing shit like this. And now the next generation is currently being grown. That's fucking insane, bro. I don't really know even what. If what I'm picturing is what it is based on how you just describe that, that is the first scene of every really bad dystopian movie ever seen. Yeah. No, like actually there's true thing that like I feel like sometimes when I tell people I think humans will go extinct and AI will lead to extinction, people think I'm being metaphorical. I am not being metaphorical. I am being completely 100% serious about this. Okay. If we don't do something, if we don't stop, which we still can, we can still stop it. But if we don't, humanity will go extinct. I want to talk to you about that. Let's put a pin in that for one second, though, because I want to stay with the story at hand and the dude is running that company. Have you ever met Sam Altman? Yeah. When? Well, it was the last time I met him. It must have been like 2024, maybe. What's he like? Charismatic. Is a word I would use to describe him. I would describe Sam Altman as someone who is very interested to talk to. Whenever you get the chance, if you ever get the chance to talk to Sam Altman, I recommend it. He's a very interesting guy. That's all you got. I have just charismatic. Yeah, I have other things. Maybe we can talk about later after I'm drinks off the air. Okay. The way I would describe people like Sam Altman and other people like him, I've met up many people like him, is they are very competent at achieving what they want to achieve. They are very competent and gaining power. And sometimes that's for good. We talked earlier about capitalism. I think Sam Altman is a poster child of what I'm using capitalism for good if you make him do good things. If Sam Altman was building fusion power, solar plants, or Amazon, I would love no one more than Sam Altman to be in charge of this. He's a very, very good CEO, very, very competent. He can run a business. Absolutely. I definitely am brilliant guy. I admire a lot of how good he is at running companies. But when we're talking about something where a lot of power is at stake, but also the lies in the security of people, it can't be up to the wins of individual people. That's right. That's the fundamental problem. It's very easy to shit on people like Sam Altman. And trust me a lot of people, I understand. He can take it. He's a grown up. But it doesn't really matter in a sense. It shouldn't matter if Sam Altman is a good guy or a bad guy. The reason Jeff Bezos doesn't have a nuclear arsenal isn't because Jeff Bezos is a nice guy or a bad guy. It's because Jeff Bezos tries to build a nuclear arsenal, the men in black show up, and make sure that he doesn't get that. And this is what we have to do. Instead of focusing on is Sam Altman, nice guy or bad guy, is he is Dario, good guy or bad guy, whatever. And instead of doing psychoanalysis, what we really should be is just we just have to stop them. Sam Altman is building something. And so are all these other people that is going to replace and destroy humanity. Did you tell him this? Yeah. What do you say? It's always hard to. Sam is a master in sounding like he's agreeing with you and absolutely everything in a very, very deep way. And then turning around and I think forgetting you ever talked to you. Where did you talk to most the context where you one on one, where you with a bunch of people? I met him one on one twice and I was over for dinner at his house once with a group. So when did you give him the warning when you were one on one with him? Every time I had every time I had the chance, of course. I'm not the only one.
he knows. Everyone knows. Everyone knows. This is this is my problem though. I think you're 100% right that like you'd never let Jeff Bezos build a nuclear arsenal that's just accepted knowledge to like government wouldn't let that happen and you know you don't even have to be like a big government guy or something to be like yeah that makes sense. Yep. This is arguably potentially if it does the things that are being warned about actually maybe more damaging than nooks. We're talking about things that could fucking control the nooks hypothetically. Yep. And we're letting this little fucking guy run it and my problem is I think it does matter that he's clearly not a good guy. When someone who jokes about the power of AI and very in the middle of a what was that? Mo Goddats documentary that's coming out where he knows he's talking about like the future of AI here in the problems and he said something along the lines of yeah you know AI will probably end humanity but we'll build some cool shit along the way. That's a sociopath. I don't have to be a fucking clinical psychologist to diagnose that one neither does anyone out there listening. We can see that that's a sociopath. This sociopath is also paying off our government and getting the first fucking press conference with the president the day after he's inaugurated in 2025 talking about how beautiful this fucking new dystopia is going to be. And so you're going to have to part of me if people out there are feeling really fucking cynical that that guy doesn't have a governor on him. And he's been it doesn't carry around like a fucking arsenal for his doomsday bunker or something. He's like the first guy to do that. Like who the fuck are you pal? That's I'm sorry. That's that's that's how I feel. I think you have a very correct reaction that you should generally have to oligarchs. Like I think oligarchy is very bad. I think separation of money and political power are very very important. As I kind of said earlier, like in a sense, in a sense, I like sociopaths. I actually like sociopaths a lot the same way that I like wild animals. If you like if you're in a zoo and you put your hand into the tiger enclosure and it bites you, who's fault is that yours? Yeah, it's your fault. You know it's a tiger. You know it's going to bite you. I feel quite similar about sociopaths. It like I can work with a sociopath. But like if he gets the ability to screw me over in a business deal and he screws me over who's fault is it yours. Of course, it's my fault. I know he was a sociopath. What if he didn't have a choice? That's when it gets problematic. If you're in the cage with a fucking tiger or there is no cage, then it becomes wrong. We're all in the cage with the tiger. That's the problem. The fundament that you know the fundamental problem is is we're letting you know Sam Altman build a nuclear arsenal. Like we should just not let him do that. We should not let anyone do that. It's not about him specifically. Like the fact that he's like imagine Sam Altman was the nicest guy in the world. He was like actually a great guy. It would matter. It doesn't matter. Like it doesn't, it wouldn't change anything. Because first of all, if you were actually a great guy and we're like, hey, we got to stop this. Yeah, he would just stop. Like first of all, he would just not build superintelations. Like if he was a great guy, he would just stop. He would just not do that. And then some other asshole would. Well, that's the question. And let's even take it beyond the scope of America. What about other countries? Yep. We're building this. China doesn't give a fuck. You know what I mean? I remember reading Kaifu Lee's book, AI superintelligence back in like 2018, 2019, something like that. And you know, he was he was showing the act of AI race between the US and China. And to me, it's like a bunch of people, a bunch of really rich people fighting over shit saying, well, we got to win because it's worse if they win, but they view a win as like, well, if we survive and everyone in our country dies and everyone in their country dies, including the people we're fighting against, that's a W. That's how it felt to me. Yeah. I think there's a lot to be said here. In a sense, this is the most class example of a core concept of economics 101. This is what's called a market failure. Yes. This is a very, very classic market failure. You know, in the most classic example, let's say you and me started a chemical company and we could dump our toxic waste into the local river and it would make our product cheaper. Now, maybe you and me don't do that because we're nice people. So we don't pollute the river. But now some other asshole starts a new chemical company down the road. He dumps all his waste into the river. His product is cheaper than ours. We go out of business. That's the problem with markets is that it only optimizes for one number. And if that involves, you know, polluting the local river and killing all the wildlife and you know, poisoning all the children, well, tough luck. So we have an answer to this problem. We have always had an answer to this problem. It's regulation. This is the point of regulation. You know, it's in the word to regulate, to make things regular, to like control, to put things on the right path. That's what it's for. Regulation is not to stop things. Well, to stop bad thing is to put things on the right track. Not saying it always succeeds. But the only way we can have both nice chemical companies because we do want chemical products, they're great. You know, we do want nice plastics. But we also don't want the river to be polluted. That's right. The only way to do this is is that we have to make it illegal to pollute the river. Once it's illegal, then we can just put the CEO in prison if he does it. That's the only way that we can then have good competition. But you have to get agreement from other countries to do the same thing or it's all for nothing. So what is incentivizing Xi Jinping to follow United States standards? So in a kind of twisted morbid way, the game theory does actually work out. The reason it works out is because if America builds super intelligence, America will not achieve American objectives. America will cease to exist, as will China and all other countries. If China builds super intelligence, it will not achieve Chinese objectives. China will cease to exist, as will America and all other countries. Yet they're both still building it and they both still think that that's not going to be the case. Correct. Therefore, the actual bottleneck here is not the game theory. It's actually just because I think people don't understand what super intelligence is. It is much more about the understanding and the awareness. I don't think anyone has given Xi Jinping a 30 minute briefing on super intelligence. I would be very shocked if a single person had given me a 30 minute briefing on the extinction risk of super intelligence. Because they told him to stop and then he'd be like, all right, thanks, pal, off with your head. Yeah, like I'm not I'm not sure. I mean, I don't know Mr. Mr. Jinping, you know, I don't know, but he seems like a pretty smart guy. And look, I am not a China. I'm not a dummy. I am not a China expert by any means. I don't claim to be. But if there's one thing I know about the CCP is that they love control and super intelligence will take away their control. There is no way they can control super intelligence. If they know that. And that's the question. So the question is much less. Is it in their so in their incentives? Actually, it is in their incentives to not to build super intelligence. This is not the case for example, for nukes. It is very much in the incentives of China to build nukes, very much in their incentives. It's all their incentives to use them because of mutualized registrations. Yes. But it is in their incentives to build it as is for most countries. But the new without AI attached to it doesn't think for itself. Exactly. The button. Yeah. Exactly. Like, you know, nuclear warhead, all things equals pretty safe. You can have in a hanger somewhere. You can hit it with a hammer. It's not going to hurt you. It's fine. As long as you don't fire it, it's okay. But super intelligence is different. Super intelligence. Once it's developed, there's no putting it in a hanger somewhere. It is over. It will escape. You will not control it. It's over. So this is why it's different. So the reason I think there is hope and but, you know, when I say hope, I don't think it's like, you know, you know, us and China are going to be best buddies forever. And we're going to shake hands. And so that's not what I think is going to happen. I think this is going to be messy. I think it's going to require us to set up a trust but verify regime backed by credible deterrence. What does that look like? What does looks like is fundamentally that us and China and, you know, other countries have to come to an agreement, you know, that fundamentally says we're not going to pursue super intelligence. You're not going to pursue super intelligence. And we're going to use technical means to verify that this is the case. The same way that, for example, for various other forms of weapons of mass destruction treaties or like disarmament treaties, we would have inspectors say from America, go to Soviet Union and check, you know, are they making these missiles or not? Or like, did they make this kind of weapon or that kind of thing or not? And we will require new forms of verification. You know, you need to like think about things, you know, like, you know, satellite, and espionage, um, on chip mechanisms, like this way, you can modify the computer chips to allow you to like verify what's happening until someone finds the needle in a haystack zero day ish kind of thing to get around all that. That's my, that's my, I love your idea. But like at the same time, even without looking at the full history of like nuclear proliferation and what was checked and what was right and what was wrong, sociopaths often feel infallible and infallible and a lot of them are in charge of countries. And they will look for ways around this because they feel like, now we'll figure it out. I agree. And we have to stop them. It's that simple. Either we stop them or we die. We have to stop them. And they like, we cannot allow China to build superintelligence. I want to be completely clear. You know, when people ask me, what about China? My answer is, yeah, what about China? We cannot allow China to build superintelligence. We can't allow Iran to build superintelligence. We can't allow the EU to build superintelligence. So we can't allow anybody to build superintelligence. And this isn't a fact about like, you know, politics. This is a fact about physics. Yes. Like, it's just, it doesn't, like, I'm sorry. Like, it's same with the nuclear weapons are not a political thing. They're just physics. Like, the fact that nuclear weapons are possible is a fact about, you know, nuclear atoms, right? Yeah. And this is a fact about computer science that AI and superintelligence is possible. It's just the world we live in. Look, maybe there's some other universe where there is no superintelligence. We don't have to worry about it, right? But in the world we live in, sure got them looks like superintelligence is possible. And so if we want to have a long, stable, good life as humans and for our children and for our grandchildren, AI is not going to go away. We have to learn how to live with it. We have to learn how to control it, how to regulate it, how to not have superintelligence destroy everything. In a sense, I think one of the big things that our civilization has failed at for the last decades is the responsible stewardship of technology. Is it?
We have had so much progress in technology that I think we are not stewarding responsibly. - Too much too fast. - Too much too fast. Do you feel like we're responsibly stewarding social media right now? - Not at all. - No. - Look at the lawsuit that just went through. - Yeah. - The first one and probably many precedents now. - Yeah. - Just like look at this. We have made many mistakes. Remember, let it gasoline, knock two billion IQ points off our species, actually more than two billion probably, off our species for nothing. It's just stupid. Like it wasn't even that much more effective. It wasn't even that much more cheaper. We've made a lot of mistakes of mishandling technology. And this might be our last mistake if we don't handle this correctly. - So 'cause I think you're making a lot of valid points here and you're talking about all the things that I wrestle with at night. I used to have these two pictures in my old studio. I still have them out there somewhere. But one was a picture of the nuclear bomb test at the Kenia toll, which basically blew that island off the face of the map, which was I think in '46 or '47, something like that. And then above it, there was a picture of the creation image of the hand of God and Adam, but instead of the hand of God, it was a robot. And that was deliberate on my part to kind of look at every day because it was like, okay, for the past sub 80 years, we have found a way, despite all of our many mistakes in humanity, to not blow all each other up with nukes. And now we're facing a new frontier of the same kind of thing that could hurt all of us. And you gotta hope the sociopaths have just enough amount of humanity to realize, okay, just like the nukes, let's not do this. - Not that we didn't come close a few times with nukes, we did, but it never happened. - So I agree with this a lot, but I don't think it's because the sociopaths had humanity. I don't think that's what saved us. - What saved us? - I think what saved us is mutually a shared destruction. Pure rationality, pure sociopathic incentives. The Soviets didn't nukes, not because they were good people. They didn't nukes because if we did, we were going to nuked them to the oblivion. And this was something, so again, it's like a wild animal. How can you get a wild animal to leave you alone? One thing is just like you have a gun, you know? That's a, or you know, you have a shock, you know, thing, or whatever, you know, you have a weapon. You, or you're a bigger animal. You know, one way to let an animal leave you alone is be a much bigger animal. Like that's a way to let you get to leave you alone. There are, we will never get rid of sociopaths. Sociopaths are a deep part of our society. They are always going to be, they've always been, and they always will be. Any plan that is involved, oh, the sociopaths will suddenly be nice, or there will be no sociopaths is not a plan that can work. But the good news is if we look at our history, we actually have found many ways how to keep sociopaths in check to varying degrees. We did, in fact, avoid nuclear war. We do, in fact, have much less crime today than we did a hundred years ago. We do have much less, you know, war than we did a hundred years ago. But what's the difference there? I think that there are many differences between these things. There's a lot, but there's one key one. You can see those things. You can see a mushroom cloud. Even if you, you know, you're looking at video I've had a picture of it. With AI, you can't see it. It's behind, it's behind a coated wall. Now that's not perfectly executed once you actually see it, like play out in the real world, but you don't see the gunman with the gun. It's a thing. It exists in the ether sphere. And I think that that divorces some people from like the human sensory reality of it. I don't know if I'm right about that, but that's something I do think about. I think a law of this is true. I think that what you're describing is so large or why it is hard. But as a counter example to this, ozone layer. Do you know about the ozone hole back in the 1960s? So if there was an, so a fun fact about the ozone layer is that when the ozone hole was discovered, the technology to measure the ozone layer was only a couple of years old. This was actually still pretty recent science, where there was just some graphs on some paper by some scientist somewhere that showed, oh, there's a hole in this thing called the ozone, if it can't even pronounce that. It was like, it's Greek or something. Like even it probably you'll never heard of this before. And because of CFC, what the hell is a CFC? Like you never heard of this in my entire life, some kind of like weird liquid. It's like cold, like what the hell's going on? And you want to change all of chemical industry because of this, including in China and like all these other countries. That's crazy, that's impossible, but we did it. We did do it, we solved it. We did close the ozone hole, we did do it. Now, this was an easier problem than AI. I'm not disagreeing. Another example of this is human cloning. Humans have never been cloned to this day. But don't do it Ben, don't you do it Ben. We're watching you and Matt. I know he's referring to, but yeah, don't do it. That's right. I'm sitting in that lab the other day. Don't do it. But this is actually a huge triumph of humanity. I think a lot of people don't know about, is that it became clear to biologists that human cloning would be possible long before it was actually technologically feasible to do so. And their reaction to this was, oh shit, that's really bad. Like this will have horrible consequences for the world if we can clone humans. And so what they did is they actually came together decades before the technology even decades before the technology existed. They had conferences, they worked to educate people as a subject of talk to consciousness to create preemptive moratorium on this technology that didn't even exist yet. Long, long before there was anything visible whatsoever. And now today where the technology does basically exist, no, there are no human clones. And I think there are other worlds in which that's not the case. I think there are absolutely worlds where they didn't do this. And there are, for example, dictatorships that have used clones for various military purposes and stuff like this in order to chain their societies in various ways. And like you imagine, you can just select for the traits you want in your obedient population and you can do this at massive scale. I think this would have horrific consequences for the world. And we did avoid this. The technology exists. This is the thing we can do. And no one has ever done it. Also, how many people have access to it? AI is something that we are all the user inputs for as well. We are all on the internet every day. We are all using these tools as well. We are all nodes around the world. However many people are on the internet, you know, divide that in half. And that's the number of people that have used some sort of AI. At least I'm guesstimating right there. But point being a large number, whereas like the cloning community, pretty small space of like very serious scientists who know a lot of shit that you at least I don't know, right? So does that make a difference there as well? I think it does. I think it's one of the things that makes it easier than into the cloning does. Because in a sense because so many people have experienced AI and had the freak out moment. I mean, even us earlier, we were talking about the huggy phase incident completely material and you freaked out correctly, you know, like you felt something, you know, people on the internet. I hope hope you, because you should. And so there's a sense where it's, you know, it's very easy and often justified to be very cynical about people. Right? There are many reasons to be cynical about the general population speaks care quotes, right? But like people can't think like we do have the most educated population to ever exist ever in history. Like we have the most informed, most educated people to ever live on the planet, you know, it doesn't feel that way sometimes, but it is literally true. Yes, and it like literally true. So we have more access to information faster therefore we can spread messages much better. Imagine if the founding fathers could just like send a tweet to all Americans, right? Like back then the our constitution was written in a time when the fastest way to deliver a message was a guy on horseback. That's how our entire country was built. Shout out Paul Revere. I mean, yeah, genuinely so. You know, shout out to all the messengers and the postal system and all these things that made this country in other countries possible. So things have changed a lot actually. Sometimes for the worse, but the upsides are also very, very large. What does it look like to have, you know, the equivalent of the founding fathers in the 21st century? Like if they were alive today, what would they do? Oh, they'd be looking at this shit going off and yeah, that ain't it. I think I genuinely think so. I genuinely think so. If the founding fathers and the Enlightenment think are there online today, they would be like absolutely not. Like, you know, not just AI, but I was like social media and the way we're handling the internet. Well, I think first of all, they'd be very excited about what is possible here. They'd think about what incredible levels of democracy and freedom and openness is possible with these new technologies that previously was not possible. And I think they would come up with fantastic new ideas. And they would also be like, these abuses are horrible. Yeah, this kind of power concentration, this oligarchs, this AI shit, these like agents, like they'll be like, what the hell, this is so antithetical to freedom, liberty and happiness. So obviously we have to intervene. We have to build institutions and systems to address these problems. I kind of picture them like Lucius Fox, Morgan Freeman in Dark Knight when he looks at the whole, at all the screens reading all the data and going, we should not have this, you know, like I think that's, you know, shout out Morgan Freeman. But I feel like that's how, you know, you're Ben Franklin's and stuff like who's, who's the smartest guy of all of them would look at that and say, amazing tools, but have we thought this through? Yeah, you know, and I think this is the thing that we should really be doing. I think this is at the heart of what makes, you know, our nation and other nations great, is that they did think about this in their time. They didn't, you know, if there's one fault the founding fathers had, you know, not really their fault, is that they didn't anticipate technology. There's a really funny, one of the fetish papers, I think it is Federalist paper number 10, if I remember correctly, it's really interesting. Because basically, the author is arguing about, is arguing about why have a federal system rather than a state-based system. And one of the arguments you make for why a federal system is better is because, well, if you have a federal system, then large coalitions, course.
conditions can't form because information can't spread that quickly so we can't build a large enough group because you know groups on two different sides of the coast they couldn't work together at the time at the time which is so funny from our perspective. The idea that this was a good argument at the time at the time it was obviously yes if the country is very big then you can't have these massive mega parties form because just they will fracture they can't they can't coordinate enough that makes sense in their time but obviously this does not true in the modern world so their assumption of this totally breaks in the modern world and we have to deal with this we have to deal with like these things and they just didn't really anticipate these things which you know can't blame them like 1700s man like can't blame them sure so but we need to address these things like we I mean have we even really really digested radio like or like TV like not obvious to me that we have like what do you mean digested it like do we feel that like there's like no negative side effects that come from these technologies that we have like totally under control and everything is stable like to be clear radio created communism and Nazism without radio there's no Nazi empire there's no Soviet empire because you can't build a complex system like a complex mechanism of that size yeah at the same time to play devil's advocate on that has there ever been a technology that didn't create something bad in addition whatever good it did I mean I can think of the wheel and fire it's clear I'm not saying therefore it's bad what I'm saying is therefore we must digest I think a large part of what World War II was was digesting the radio and the consequences of it that actually that's not a bad argument that's a lot of what I think World War II was and thank God that democracy won that war but like you know there are other worlds you know and you know what is the equivalent of World War II and the radio for the internet you know what is the equivalent for World War II and social media what is the equivalent World War II and AI you know the what is the equivalent digestions you know did we even have time to do them you know it took a long time to figure out how to integrate these technologies into our states and our systems and so on I mean look man whenever I use a government website it's a 50 50 with the website even loads if you like right like like does it does it feel like like one of the most shocking experiences of my entire life truly shocking moment was when I went to the United Kingdom to England for the first times he lived in London for a while and I had to apply for my visa to stay and I was like oh my god this is going to be a fucking nightmare so I I you pushed off as long as I legally could and I was like I read okay I have to actually apply for my fucking visa so I get all my documents together do whatever I go on their website and the website loads thank God that's a dub huge huge huge dub and then go to the visa section whatever and I'm like please download our app and I'm like oh no oh like actual three-word horror story right it's like four-word horror story so I download their fucking app right I type in you're not on CNN sorry yeah it's true it's dropping the F bomb there like the you government apps can can get that out of me that's right download their app I put in like my application thing whatever and I like please hold your passport to your phone and like all right hold my passport to the phone it vibrates once all right done your application is sent and it was just worked and I got my visa within two weeks you weren't expecting that nope nope holy shit it just worked like it was incredible like it just worked like it just read the NFC chip in my passport did all the things and then like I had to take like a selfie to make sure it's me and then I clicked the button and it just sent it off and then two weeks later my visa was done and I'm like oh that's what it could look like like in a sense the fact that this was shocking to me is sat this was powerful this was doable with like 2008 technology sure and yet let's be honest does anyone here really expect that we could do this like that the government could do this reliably like no one expected this I think this is sad I think a well-digested form of even like 2000s eat technology this would just be normal like of course everything would work this way you know of course you could do everything on the app you know you have to pay your taxes you know well that's how much taxes you click done all your taxes are paid like when I got back the United States and I was in my tax you know advisor explained to me how the IRS worked oh my god what are you people doing here like I don't understand like a lot of different sub rules and sub units and that it's unbelievable like I had to pay some tax in California at the time and so my you paid taxes in California that doesn't happen yeah so that does not happen yep so so and the way it worked was ultimately my accountant after figuring out finally like I owe like I don't like $60 in taxes or some shift right $60 and 50 cents or whatever I think he sends me this website which like looks like truly like Java Netscape 1998 and I had to type in how much money I would like to send them and like I could say like I would like to send you this much money and then they would let me send them on a money and then it would be like you may or may not hear from us and I just didn't hear anything from them until over a year later I get a I get a letter from the California franchise tax board saying that I must reply within 30 days of the date on this letter otherwise like my taxes would not be correct whatever and the date of that letter was from three months ago so by the time it arrived at my doorstep it was already two months you know overdue so I asked my the accountant at the time the nice British man they asked him what do I do and he's like yeah well is it from the California board and I was like yes and he's like oh yes they are very nasty well just see if they write you another letter and so I just waited if they would send me another letter they didn't and it was just okay that seems so efficient it's so efficient like I paid my tax I know I did like I did it properly and just they just send a letter and then they just like apparently made a mistake and just did do it again that's government for you but like this is a thing like it doesn't have to be this it doesn't have to be it doesn't have to be but it always has been that that's that's really the thing we're like this again it's like you know great but not good right like there's no reason the government can't just once a year send me a thing or like this is how much taxes you owe us and you if you think it's wrong let us know and we'll talk like you can just do this right they can be a nice little app you know like sure there are some people they have genuinely complex tax situations sure but let's let's be honest like 80% of people we have like a job and maybe a side gig hmm like there's no reason that has to be more complicated than this and you could simplify the tax code where it's like look if you have a job and a side gig it's like it's just one percentage and it's like it's that simple there's no reason you couldn't do this and you couldn't still have like the same amount of tax income and it would be so simple but alas that would make too much sense that's that's kind of my go-to line with governments it's like if it makes too much sense they're like we're gonna make extra steps for you I think a lot of it to a degree is much more inertia than anything else it's not really the doing on purpose it's just kind of it's always been like that and changing it is annoying and they don't have incentives to change it that's a big thing the same fucking seller yeah I have a I have a joke theory sometimes which I call the one guy theory the one guy theory yes so this is a bit of a joke but it's kind of true whenever you see something that's like really amazing they like really like stuns you like it's not just like good but you're like whoa how did this give me like you watch a movie and you're like whoa you know what TV show or like use a app or like something you're just like whoa like this is crazy my theory is there's exactly one guy or girl that actually made it great there's always and if you look you can find who the one guy is so like if you see like a great TV show or something there's gonna be like one director or like one writer who was like obviously the guy who made it good or a girl that made it good and it's actually very funny I asked some people knew you hey government and like is there a guy and I'm like oh yeah I know him this is nice and they like told me who it was there's just one guy who was very autistic about creating this app and really wanted to do it well and like put all of his life force into that one and just nailed it and just like totally nailed it and so I think a lot of it is it's just a lot of effort and like you don't get a lot of reward like I don't even remember this guy's name which is honestly a shame because he did a great job like I should remember his name and like shot him on a podcast just because you did that right but like he's not gonna get credit and he's not gonna get paid for it like it's so sad right I remember there was a very um what was it I think it was the head of pandemic preparedness for the entire UK government there was a job posting for the head of the entire UK like pandemic premise and you know obviously requires you know top tier credentials etc with a salary of 70k good luck I'm saying like why are they gonna do it like what like what do you think's gonna happen here right like you want a top tier you know epidemiologist who can lead you to super complex logistics for an entire country and you can't pay him more than 70k here's the thing where it gets weird it's all about accounting and here's what I mean by that so you look at like the intelligence services you know they all get paid maybe your regular ci spy you get paid at $89,000 five you know 500 a year or something like that so they ended up inventing a system to where years ago which just kind of happen to where you could fucking resign from CIA on a Friday walk back in through the front doors of Langley on Monday only now you work for Lockheed Martin making 1.3 million with stock options for the year be handed a lanyard.
taken back to the same desk you were at, read in on the same security clearance, working at the same place. But now officially you technically worked for Lockheed Martin and here's the difference. Unlike at CIA where maybe you, they have to list off to the taxpayer somewhere deep in the fucking minutia. This person made $89,000 like whatever that role is. Now they just say we had to pay a full contract to Lockheed Martin for $4 billion for defense purposes. And no one knows the fucking $2.5 billion of it went to salaries of people who used to work at Langley or whatever, but they get to hide that expense in the private market strictly because they were not able to incentivize "talent" to stay there and actually perform. Now think about it with something that's not life and death, like the fucking DMV or something like that. And you give an example where it is some life and death, an epidemiologist, but they have every incentive to say that epidemiologist could be like fucking, you know, "Mark's going to pay me $3 million to do the same thing." Like I'm going to go fucking do that. Yep. A very well-known result in general study of corruption is increasing public salaries actually decreases corruption. This is a very well-known thing. So for example, I have another funny story to tell about governments. When I started my AI company in London, we were a pretty small company working on AI safety. Where is it? It was 2022. Okay, so. And what was the nature of the company? So it was a startup called Conjecture, very fond memories. And our goal was to solve AI safety or, you know, die-drying, you know, kind of like do our best to make AI safer and controllable. Specifically, we ended up working on a sub-problem of the larger problem, which is called boundedness. We were working on how do you build AI systems where you know what they can't do. You know the limits of what they can do. How do you even figure that out? We had a bunch of good ideas. And we did a bunch of research, ultimately, the company, I shot down the company a couple of, well, yeah, about eight months ago, because, well, for two reasons, one thing is there wasn't much interest in safe AI. There wasn't any money in the field. And the second was, is that fundamentally I thought the political problem was more impressing and more important. And that's why I joined control. I had to work with that. That first one said though. That first one said because it shows the incentives backwardness. Yes. I mean, this goes all the way back to when I did open-source work as well. Like the reason I did open-source back in the day, you know, I reversed into your GPT-2 and I did more work like that. I let a group call the Luther AI. They're still around, but I've been involved for a long time, where we did open science. You know, we built the very, some of the very first open-source, large language models in the whole world. That was us. That was me. That was the boys, you know. And our goal was, our hope was, is that if we could make it open-source, you know, then people could study it. People could see the things I was seeing and how dangerous they are, and they could work on making them safer and more controllable before they become, you know, truly dangerous. And no one gave a shit. Yeah. No, they only want, the main thing is that people are complaining that they want them to be more uncensored. And they wanted them to be faster and cheaper and more powerful and et cetera. No one cared about safety, you know, some of the few people did. But like the open-source community didn't care. They didn't care about safety. They didn't care about control. They didn't care about any of that. What about some of the people like that you saw building these things along the way? Obviously, we were just talking about a huge name with Sam Altman, but like, people though we wouldn't know. People in the rank and file who are brilliant coders and understand how to build AI very effectively, I would imagine and correct me from wrong, you have a lot of frustrating conversations where you're like, can't you see the wall right there is white and they're like, no, fuck that. It's black. Is that fair to say? It is very hard to make a man understand something when a salary depends on him not understanding. That's right. But in the sense it's actually worse than this. This is actually the nicer version. I think there's a much less flattering version and what's that? So in a sense, if you see yourself as like an evil sociopath, it's kind of based, you know what I mean? It's kind of like, I'm not saying, you know, based doesn't mean good, you know? But it's kind of like ego defensive. Like if you're just like, I'm just a cynical, you know, guy out for myself. You know, I'm not a good guy. I'm just doing what needs to be done. It kind of makes you sound cool, you know. It kind of makes you sound like it's an ego defensive mechanism. Rebel without a cone. Yeah, it's like, you know, yeah, like it's like you get to be, they still be the anti-hero. You still get to be cool in your own story. That truth is, the reality often is much more pathetic than this, like much more pathetic than this. And much less, you know, cool than this, like the thing I would convey is that, and this is coming from a nerd, is that nerds are cowards to an unbelievable level. Like it is hard to describe how cowardly the average nerd is. I like to consider myself not a coward, you know, I think that's the only thing you are. Well, thank you. It's the thing that I try very, very hard, not to be very, very important to me. When you say they're a coward, they tend to be cowards. How are you, can you define why? There's a lot of, there's a lot of things, but a lot of nerd culture, whether that's open source inside of these companies, et cetera, can really be boiled down to one core tenement. I should be allowed to do whatever I want. That's like, that's actually even still too nice. That's still too ego defensive. The actual thing that almost all these nerds have in common is, I believe, I want to play with my toys. Get nerds love toys, they love math, they love programming, they love science, whatever. It could be a useful toy, you know, some, some artists, you know, hyper focus on curing diseases. Great. You know, that's a good thing for them to play with, right? And we should, you know, make sure they're in a nice lab and curing cancer or whatever, right? Great. No problem with that. But for a lot of nerds, the reason they do science is not because they have rationally reasoned. I want to help as many people as possible and science is the best thing for me to do. It's much more it's just, they like toys, they like math, they like science and they really, really want to do it. And this leads to the fact, this leads to a lot of weird effect that like, out, the outside world doesn't really see if you're not involved with like the science and the engineering world more closely where like most of academia's bullshit, like I'll just say it, most of science is bullshit. Not in that it's wrong, it's true, but more it's useless. Most of science is useless. It doesn't have to be this way. But most of it, it's just people doing things because it's kind of interesting and fun or they get away from it. That's not what I thought you were going to say. I don't know. I'm not saying it's false. It's not what I'm saying. Or it's like, right. That's not what I'm saying. I think most of, you know, you know, maybe not most, but like, you know, if a new physics paper comes out, the math and it will be fine. Like the results will be fine. The data will be fine most of the time. Sure. There's exceptions. But like most of the time, I read a paper, I expect the data is fine. I expected it to be useless. If you gave me a random physics paper, a random new physics, I'm like, this isn't going to change anything. It doesn't matter. Why? But if someone's writing about gravity or, you know, something that's, that we all have to deal with. So this is actually, oh boy, we're getting into a deep rabbit hole here. That's okay. Let's do it. I'm going to have to bring you back already. I can tell. Well, I'd love to. So a lot of this is because of incentives. The incentives in academia are very broken. There are reasons that I dropped out. Like, there are very, very broken incentives in academia, where you are very, very strongly incentivized to write what are called papers. It's kind of like reports on very scientific things and they have like a sort of a shape all that doesn't matter. And basically, the way you get promoted is by having as many papers published in as good as what are called journals as possible. And this is an extreme cluster of fuck off politics. Like this is, you know, in an ideal world, it should be like super objective, you know, the best scientists, you know, they get published in the best journals and they get promoted. And this does happen. Again, I do think that good science does happen, not saying it doesn't. But all these journals, by the way, are for-profit companies. So incentives, we start with that, right? And then it goes further than this, where that it becomes quickly clear that doing the best science is not the same as writing as many papers as you can. So like a thing that will often happen is that scientists will do a certain piece of research. And then they will try to slice it up into as many papers as possible instead of writing one paper of the result. Here's like, here's the science I did. Here's what came out. They'll write four different papers of like, here's the first experiment I did. And here's the second thing in terms, and here's a third paper interpreting the previous thing and blah, blah, blah, blah. So you wrote like four papers that have like one paper of content. And so it's not lies. It's not false. It's just bullshit. It's doing something to do it. It's just doing something to do it. And so much of academia is like this, so much of science is like this. Again, it's not like, I don't think they, I do think lies and fraud is actually kind of rare in science. It does happen. But it's like actually kind of rare, you know, like more rare than I would have expected given the incentives. I think this is too large degree because most of them are nerds that just really do love science and like do want to find out new things. Again, not all of them. There's some famous cases I think we're all familiar with and I expect there's more. But there's just absolute deluge of bullshit. It's just absolute time wasting of just like spinning the wheels. You do, you know, one thing and then, you know, but it's like maybe another famous example is nanotechnology. So nanotechnology is a great buzz word where when the word nanotechnology was first coined really by a physicist named Eric Drexler in the 80s and 90s, where it was about nanobots. It was about like small machines that, you know, that could do crazy things. He did a bunch of math where he could show that like these like tiny nanobots could do like really, really crazy things. Now there's some controversy about whether, you know, his theory is right or not, we can argue about that, right? But it's an interesting area of study of like doing like nanorobotics and nanotechnologies are very interesting for your study. So there was a lot of interest that came to me.
into this field of nanorobotics and nanotechnology like this. I've actually met Drexler before. He's a really funny guy. But what happened was, is that people scientists realized, "Oh, there's a lot of money here." So a ton of people that had no interest in nanorobotics started calling themselves nanoscientists. So especially material scientists. So basically anyone that was using any small thing, you know, for example, someone building, you know, I don't know, like new glass for your windshield or whatever said, "Well, there are nanoparticles in my glass." So I'm a nanoscientist actually. And then the whole field kind of just became completely swamped. You know, we had like, you know, Drexler and like a few other people and then just like a 10,000 billion career scientists who were just, you know, changing the glass composition of something. Not to say that there wasn't also good research done on nanotechnology, you know. It was a mad rush. But it was a mad rush. So the incentives were clear. So this is really, when I say bullshit, is what I mean by this, right? It's just incentives. It's just the incentives are hard. It's hard to incentivize good science. Good science is good science is slow. It's expensive. It's hard to do. And really nowadays, the best science usually happens startups. You know, that's really where a lot of the best science nowadays happens. Yeah, the initial point on this tangent we're taking, though, is where you were describing as a generalization like so many nerds or cowards and I'm not bringing back that's good. I'm not bringing this back to rail on nerds, you know, at all. Like, but it's an interesting thought for some of the people that you're also specifically thinking about it tech companies because I don't mean this as a shot. It just kind of is what it is. You see a lot of guys that are physically unimpressive, very effeminate, have an inability to really look people in the eye at any point who probably, and I'm sorry, this happened, we're bullied at some point growing up and that should have never happened to them, but they went away socially and had fewer social relationships in most cases. So they've been physically separated from people for a long enough time who found solace in the power of their fingers hitting the keyboard. So maybe they don't do a bench press, but they do have this thing over here and they could do that and they are now maybe passively, hopefully not knowingly, but at least passively, unfortunately, same end result taking that out on other people in these powerful positions they're in. So I think this is a, you know, very fair piece of psychoanalysis and definitely applies to many people I know. But in a sense, I think it's still too kind. Too kind. I think it's still too kind. Okay. I think it's even sadder than that. I think this is still too ego defensive. This still makes them sound like hardcore villains. You know, it's like there are like some cool evil guys. You know, but in a sense, it's, it's often much more pathetic than that. When I talk to people that like work at meta and like optimize, you know, you know, I add algorithms to give teenage girls eating disorders, right? Like when I asked them like, why, whatever, they have just mostly never thought about it before, but they just never thought about it. They just did it to do it. They were just like, I like numbers go up. Like, oh, I just do math. I don't, I don't hurt people. I just do math. I don't hurt people. I just do math. So in a sense, it's often, there's one of the great innovations of the 1980s and 1970s is that sociopaths learned how to domesticate nerds. Sociopaths figured out that nerds have a lot of power is that math is very useful and that you could domesticate them because nerds just want to play with their toys. I remember reading a report by a journalist, which I found really interesting, which is I think she was a visit like a thing of Facebook. This is like 10 years ago or 20 years ago. She visited like the Facebook office building and she was horrified. It's like, it's like a playground in here. Like they have like slides and like drinks and like it's like people are like hanging on a bean bag chair. And she was kind of horrified. Me as an autistic kid, I was like, well, it sounds sick. I mean, it's not like fun. I don't know. But and the thing is like, yes, it is in fact meant to be fun for, you know, autistic people like me and other people so that they just hang around and do the work and don't worry your cute little head about what this is actually used for. To a large degree, what sociopaths do, where I there's a psychoanalyst. So as I'm a first world disclaimer, psychoanalysis is definitely bullshit. So psychoanalysis like Freud and stuff like this. Definitely bullshit. But there's some true cook and cooking. Yeah, sorry. We're gonna go deep. We're going, we're going in deep. And but there's some truth to some of the things that they're saying. And there's one that I like as his name is Edward Teach, who wrote a terrible book that you should never read. But it had some very interesting points in it. And one of the points he makes in the book, which I found very interesting, is that the way you often see oppression portrayed and how it often happens is you have like some evil vampire that's like oppressing the poor people. This is how often things go. But often it's actually the other way around where people go find the most evil vampire possible. Like, please oppress us. We really want to be oppressed so badly. We want to be oppressed. Please oppress us. And the members are like, All right, okay, fine. I do like oppressing people. So we'll go do it. And this is a very controversial thing, obviously, that he was saying here. But I think there's a lot truth to this. Where a lot of sense, a lot of nerds just don't want to think about the consequences of their actions. And they want an evil sociopath manager to be the one to blame. They want a Sam Oldman. They want, you know, a daddy, you know, they want to, they want to, you know, some evil see a way like, Oh, I'm a good person. You know, I don't want to just fall in order. Look, I was just, you said it, you know, like, you know, I was just falling orders. Like they'll say like, well, I, of course, I'm working on the insight to make Facebook more ethical. But you know, our mat, you know, you know, you know, how the boss is, you know, I can't, you know, I can't stop him. Right. And so there's different ways you can interpret this, right? The ego defensive way is, you can say, Oh, the poor nerd is so oppressed, you know, like always being, he's just, you know, being a victim of the system, or you can be like, what the fuck are you talking about? This is a grown-ass man. Fuck yeah. Like, this is a grown-ass fucking man. Take fucking response, more responsibility for your actions. Fuck yeah, dude. And this is the thing that like really ultimately repelled me out of this field. Just entirely like sent me to the stratosphere and like out of this because I just like cannot be a part of this. Like, good for you. In a sense, I prefer working with the evil vampires in a sense because there's more clarity here. Like, they don't, they don't, they tell you who they are. Exactly. Like I like sociopaths who are just honest about being evil. This bro, you are, there's so many times in context in which I think of this, but I've said this. I'm not going to go into all the different contexts. I've said this this year, but I've said this about so many things. Give me the wolf who snarls its teeth outside my house. So I know not to let them in for fucking dinner over the wolf that dresses up in sheep's clothing comes in, eats my food, fucks my wife, and kills me. I have way more of a problem with the second one than the first one. Yeah. I think this is a large degree. And this is the thing that I find like, you know, I'm generally not a super emotional guy. I have like pretty good, you know, zen practice these days, you know, lots of meditation and everything. But one of the things that I find disgusted me like actually like creates like revulsion in me is when I see these people who are like super high IQ, wealthy, agentic, capable, grown-ass adults who then work at these companies or whatever and then you know, cry about how they're the victim or like how it's not their fault or whatever. And I'm like, and this disgusts me, like this really disgusts me on a very deep level is like take some responsibility for an action. Like if you're going to be evil, be fucking evil, right? Like, okay, then we're just fight. Like if we just have to fight, all right, I'm ready. Like we can fight. I don't mind fighting. I can fight. But like this is, I find this like viscerally disgusting when people do this. And this is so common in the world of nerds. And like this is really a thing that kind of like really does like not all obviously there are many good people here as well. Yes. I mean, you know, but there is this pervasive culture of this like avoiding responsibility, avoiding, you know, so you can play with your toys. I had a very memorable experience, for example, once where, you know, I was in the chat room back in the day when I was still doing open source work. And I had recently said in a podcast that I think there should be a regulation, you know, and like accountability, you know, if you like, for example, create AIs at harm people. And so someone called me out, called me on the chat for, you know, at Conner, you know, but Conner, you have built open source AIs before. Do you want to be held responsible if you hurt people? The answer is yeah, easy answer. Obviously. Obviously, I would like to be held accountable if my actions harm other people. He wants free lunch. Yeah. And he was so shocked that he was like baffled. He was like, but, but, but, but, but, it's impossible. How could he even know if harm is God? It can't be done. And a friend of mine was like, courts. That's what a court is for. Do you not know what a court is? And so he wanted a free lunch. He was just like, he didn't know what a court was. From his perspective, a court is a thing that is in movies that gets in the way of the protagonist. It's just that there's a knowing thing that gets in the way. Like, of course, if you could get away and like not have laws applied to you, of course, you should do that. And I disagree. I think it's also privileged to have laws applied to me. I think it is not a hindrance. I mean, sometimes it could be, but there's also a privilege to be subject to law. It's the balance. It's the balance. It's keeping, it's keeping the balance. You don't want the laws to become or well in, which by the way, some of these same assholes are helping happen through the back door. But you also don't want to live in a society where it's the purge every day. Of course, right? Beautiful mental ground. Absolutely not. In a sense, it's also useful to me to be subject to law. Like, for example, imagine if there was one guy who know a law is applied to. No laws at all. He could do any crime he wants. What would happen to this guy? Well, no one would want to be around this guy because he's dangerous. That's right. He could just kill anybody. He could steal anything. And there's there's nothing you could do, so no one would want to make contracts.
with him, no one would want to do business with him, you know, people would feel unsafe around him, etc. So in the sense, the fact that I am subject to law also gives me a lot of privileges, like you can make a contract with me, you can even make like a really risky contract with me, and you know that, you know, if I do something wrong, that you will have the force of the government to make sure that the contract is in force, and this allows us to make these beautiful contracts and make complex business decisions, and also that you can feel, you know, safe around me or other people. Like I can meet random strangers in New York and feel like I'm probably safe. Like it's probably fine. And I want to make sure people are also hearing what you're saying right here, and I'm pretty sure they are, but I'm just going to double check on that. You're not saying create a big centralized government to be the other a new kind of dad or whatever. You are saying check the balance that has flipped way too far the wrong way where the babies are playing with matchsticks, and take the matchsticks out of their hands. Exactly. It is checks and balances. It is always big checks and balances. It is always the fundamental problem of freedom and, you know, liberty is, you know, liberty ends where my property line starts. You know, like what do you do on your property? I am actually a pretty big believer in freedom. Like I even think you should be allowed to do some things that I find morally repulsive. Like there are many things that I personally find morally repulsive that I think should be completely legal, you know, and that if you want to do those in the privacy of your own home, our first amendment type, for the moment none of my business. What is not okay if you start threatening me, if you start, if you start crossing, you know, my property, and also not just my property, but also my country, my nation, my people, my friends, my family, my culture, et cetera, fundamentally, we have to live together. There is no libertarian dream of while you all have your own little isolated island that, you know, it's completely independent. This is not how the world works. We share the same roads. We share the same, you know, infrastructure. We meet each other at bars. You know, like we have to get along with each other. This is the core thing of running a civilization. And we just have to deal with that. Like it's just fundamental. The answer to this is not have a totalitarian government. That obviously is awful. Right. Neither is the answer. It has no government. That's even worse. Like fundamentally, it's just a hard balance and we have to make it hard and we have to continue to iterate on this balance. I'm not saying stop. I'm not saying we figured it out. I'm saying I have the answer. I'm saying we have to keep going. So like when nuclear weapons were created, you know, this is, you know, famous story actually. So when it's actually a funny, quite a funny story where, um, well, you know, funny, where when there was a famous physicist, um, I believe it was Lord Kelvin, if I remember correctly, which is a great name. Um, and he said famously that the idea of net, you know, net energy production from nuclear vision is moonshine. It's nonsense. It's impossible. And this pissed off another physicist leaves the Lord so much that he on the spot figured out how to build a nuclear weapon. Like you just sat down, did all the math and figured out how to build a nuclear weapon because he was so pissed off by this. And so what he then did is when the first thing he's realized like, oh my god, nuclear weapons are possible. Holy shit. So he went to Silicon Valley to create a startup to sell as many nuclear bombs to every single person in the world. No, obviously not. That's not what happened. That would be insane. He went to the government. Imagine you like a bit like, you know, there are worlds, right? Like there were truly are worlds in which this happened differently. For example, where it was a different country. Imagine it was Nazi Germany that figured it out first or the Soviets. Yeah, this could have been a very different. And since we were so lucky that Leo Zee Lord was the one who figured it out and that he went to the government. He went to the military. And for years, he lobbied them, you know, leading to the famous Einstein letter, where he was like, guys, this is going to change everything. This is like, this is so big. If the Nazis get this, like it's over, you know, I've always wanted. Have you ever wondered about that? Oh, yeah. I mean, which male has not wondered about that. Like I talked to earlier, I started to cut you off, but it's just like, it's a really, really important kind of distinction right here. The buck always seems to stop in an extreme either without a control corporations or out of control government. And we can never find that fine balance. And I hope we, one day can. But like in this case, it was an out of control government in Germany. Yep. And I do wonder when I look at something like Operation Paper Clip, we're just magically all these fucking Nazis were allowed to come to America and we're like, yeah, you can get a home in Huntsville and teach us how to build rockets and ship. I do wonder sometimes. And I have zero proof of this. It's just a hypothetical thought in my head. I do wonder if some of those fucking assholes actually did figure it out. And they had just enough humanities somewhere deep in their very dark souls to be like, you know, maybe to do with the stash shouldn't have this one. Yeah. I mean, I like to hope. Though also, you know, given that we just had this long rant about nerds, I think most of the paperclip people were under this category. I think they're probably right. I think they're, but I still wonder. I still wonder as well. But yeah, there is a, there's a lot of interesting history to be had here, right? And, but what I think is the thing that I'm find interesting about the story, I mean, there's many interesting things about the story. We can, you know, there's one of my favorite books ever written is The Making of the Atomic Bomb. It's one of the my favorite books ever written. This is a really, really good one. And fundamentally, what I find very interesting about this, not just The Making of the Atomic Bomb, but what happened afterwards as well is that it was so clear that this technology was so different that we had to create whole new institutions, like the international atomic energy agency and stuff like this. This was unheard of to create a new laws had to be invented. Like there, there was whole new government department, department of energy is just the department of nuclear bombs that later also did energy, right? Like they were created basically because of nuclear bombs. Yes. To a large degree. And so did, you know, other forms of institutions. And the level of control and oversight and seriousness that is put into nuclear weapons, honestly, is that our awe-inspiring to me in many ways. I remember once I talked to a very senior member of the National Security Establishment, can't say who it was, but, you know, very good person. And good person, you know, everything I knew of her was good. Oh, it's her. Yeah. How dare you? How did you imply? She was, and I remember asking her, because we were talking about AI, and she was also pretty freaked out by AI. And I was like, is there any really serious people in the government like adults in the room who are like so serious and so real and actually adults? And she thought about that for a second and it's like, hmm, then she, you know, she parked up like the people in charge of the nukes. Like she said, the people in department, energy in the charge of the nukes, they do not fuck around. Like they, they, they, they do not fuck around. Like they actually take it seriously. But also, what did that take? It did, yeah, it took a lot. It took a massive amount. It took decades of work by thousands of, you know, diplomats, statesmen, scientists, et cetera. It was a long process. It was the institutions that exist nowadays. Like, like, like, we, you know, we have like exact, like kilogram level, you know, accounting of every single piece of nuclear, like, fizzle material, you know, known to the world that we like track. Like it's incredible, right? We have, I saw, I thought it was a great podcast with this guy on the state department, whose name I forgot unfortunately, where he was in charge in the 90s. I have the Soviet Union fell is that, there was rumors that there was a bunch of fizzle material in this facility. And so we spent years becoming like best friends with the head of that facility. They went, like, hunting together, you met his daughter and so on and whatever until eventually he admitted, yeah, there's like 500 kilograms of, you know, nuclear weapon level material there. And then, you know, after building that trust, they could get in, you know, the expert, they got the material, they melted it down and diffused the whole thing, which, like, such an incredible story, like, that just pure diplomacy, no violence, no anything. So pure diplomacy, you know, this guy figured out and had the infrastructure behind him to be like, you know, there's, there was a number he could call, whereas like, there was enough material in there to build like 20 nuclear bombs. We need to like get it out if you're safely out of like post-Soviet Kazakhstan, melt it down, be sure that everything's safe. And they did it. Like, this is incredible to me. Like, again, it's going to take seeing it though. Like you're describing the hugging bear attack or hugging face attacks. Sorry. Yeah. And that's scaring the shit out of me, but it hasn't had repercussions for people in the real world. So with nukes, they built it and then they dropped it on Hiroshima and Nagasaki and the whole world saw what it was. But importantly, this did not happen after Hiroshima and Nagasaki. This happened decades later. Oh, I know. I'm saying the seriousness of the industry was started with a massive, you know, war moment. Well, you say this, but if you actually really read what people were saying just after the war, it's actually kind of clown show stuff. Really? Yes. If you read the stuff that's like directly after the war, a lot of it is true clown show. Like famously, John Von Newman, who has an incredible claim to being the smartest person to ever be in probes, guy. Yeah. I mean, many things are named after him. You know, many, many things are named. He's plausibly the smartest person to ever live. He was famous for saying, you know, if you ask me when we should have nuked Russia, I would have said yesterday. Like he was just like, nuked the Russians, nuked them immediately, nuked everything. Like we should just nuked everybody. Thank God we didn't listen to it. But this was a very common thing at the time, including at this time, where like mutually asserted destruction also wasn't really a concept. People had even figured out yet because it wasn't really a word. It was like, there was this true fatalism at the time that people were just like, oh, all nations are going to build nukes. There's nothing we can do. We can't coordinate. That's crazy. Imagine, you know, coordinating so that people don't build nukes, that's impossible. So we're not going to, so it's not can't be done. But the truth is we did. Like, you know, many nations did get nukes, maybe more than should. But most nations today do not have nuclear weapons. There hasn't been a single nuclear weapon used in a terrorist attack ever.
Back in 1945, people were like, of course terrorists are going to use these things. Like it's just, they were just like, they're just a matter of time until one of the terrorists somewhere are going to get their hands on and use it for something. And we actually did prevent this. And this was not because there was one event and we all banded together and everything was fine. It's because of decades of hard work, of hard work, of many, many, many people who in their bones in their heart really understood, oh shit, like we have to stop this from happening. One of our advisors that control AI is Lord Desmond Brown, he's a great guy, good, good Scottish grandpa energy, he's a great guy, he used to be defense minister of the UK. Yeah, yeah, yeah, he's a good guy. And I remember the first time I met him was I was giving a speech in the House of Lords in the UK. We're talking about AI risk or whatever. And as always, the case after my speech, there's some guy in the audience. Who was like, what about China, but China will never slow down China, you know, we can't work with China. And then before I could even get my answer out, old Scottish grandpa, you know, I didn't even know him at the time, it speaks up in a thick Scottish accent. And he's basically like, what the hell are you talking about? Like we have this with nuclear, we did it with the Soviets. Like this is just diplomacy, of course, it's hard. We just need to do it. I'm just like, who's this guy? I like the people think it's possible. Yeah. So this is someone who literally negotiated like nuclear disarmament with the Soviets. Like literally, he was there, he was in Moscow and so on. And he, you know, he was just very pragmatic about, he's like, yeah, this is the new disarmament problem. And like we just going to have to figure it out. We're going to have to talk to the other countries and we're going to have to figure it out. This, that was like something really early on to give me a lot of hope as well. And this is, I like to think that that has something to it as well for sure. So in a sense, I'm not saying it's easy, we're not really using it. I'm not here to tell you it's easy. In a sense, in a sense what I'm trying to say here, I guess, is that like, if we had 20 years, I think we can do this. Like I actually think. So we have 20 years, that's the question. Yeah. And this is why my focus these days is so extremely not really on the things we discussed on this podcast and just we must abandon ASI now. We must stop it now. We must shut it down. No more official superintelligence, and we was internationally and domestically, we have to do this right now. All right. So let's bring it back to the thing we put a pin in a while ago. I put the pin in and I said we'd bring it back around. So you are warning of a potential extinction scenario as it pertains to the human race at the hands of what I assume you're referring to would be an all too powerful artificial superintelligence. What makes you think that's possible? When do you think it's possible why and what would, what, what, when do you think that's going to be possible like by when and what would it look like? All very good questions. And mentally, the only thing that needs to happen for superintelligence to be possible and to happen soon is for everything that's happened for the last seven years to just continue happening. If nothing changes, superintelligence is the default outcome. A lot of my job is just predicting the most boring possible thing. My prediction is just the amount of AI progress we've been seeing for the last seven years keeps happening at the same pace and nothing else changes. AI has become more powerful, more autonomous, less controllable, less understandable. In the sense, you know, we understand much less about AI now than we did seven years ago. Seven years ago, AI is pretty simple, but now we have swarms. We don't understand what's going on. Swarms is terrifying. Yeah, it's terrifying. We have no idea what's going on. This is crazy. You know, these swarms are and swarms also have a kind of like stability that individual agents don't have. You know, we cannot unplug these things. Well. It's just a simple thing. Can you unplug Google? Like if you want to unplug Google, could you do that? Well, not really. I never thought about that actually. I like great question. Like not really. Isn't Google isn't in any one location? And even then, if you tried to plug it, unplug Google, what would happen? Well, you get a score off the premise, right? You know, you're not even getting the office built, right? Like you're going to be the, they're going to get sued. They're lawyers. They will defend itself. You know, Google, the entity will defend itself. So if Google builds a superintelligence, the superintelligence will be going to print money, you know, going to print money. So they're not going to wash it down. And then of course, the superintelligence can just tell the CEO of Google, everything is fine. I'm totally listening to everything you say. You're totally in charge. You're so beautiful and handsome and smart. And like, so it won't just be a single server, like a computer somewhere. It will be millions, billions of computers that are run by corporations and in the end. Superintelligence can just use people. It can pay people. It can run corporations. It can pay off politicians, it can, you know, just convince politicians to help it. And effectively, let's say I went across that entire infrastructure. I went to unplug a section of it. It would almost be like firing a warning shot, is an invading army and suddenly they'd arm up. Absolutely. I mean, wouldn't you do the same? I mean, yeah. But it's just terrifying to think of something that powerful. And like, I'm thinking to my friend, Lee Cronin, who I've had in a couple of times, I just recorded with him again, who obviously brilliant guy, DARPA chemist, the whole bit, you know, he is so not concerned about this stuff because he thinks that AI is such a tool that needs the human input to be effective. And he can make that argument a lot better than I could try to re-describe it for you. But I hope he's right, it doesn't change the fact that the very things you're talking about still continue to keep me up at night, heavily. Absolutely. And look, I hope he's right too. But the truth is, people who have been saying this for the last seven years have been wrong over and over. There's a whole like congal line of like embarrassment. You know, you can look at like videos of like young looking, making confident predictions over the last seven years over and over and over again, AI will never do X. And then AI does it three months later, over and over and over, humiliation after humiliation after humiliation. Like, it's like he was Metta's guy, like a Metta's guy. And like, he's just been like so wrong about this so many times. And so have the other people, like everyone says, oh, I will never be do X or we'll ever require a human to do it. It's like millennium prize problem, by the way, right? Like the mathematicians were like, truly, no AI could ever do blah, blah, blah. There we go, right? You know, AI can write, you know, incredible, you know, art, it can create images, you know. Are they the best ever? Maybe, maybe not. You know, that's subjective. But like, it's very high quality, you know, it's very complex. It's very deep, if they can write better code than I can, they can do math better than I can, they can do to largely be biology better than I can already, and I consider myself a pretty smart guy. All things equal, you know, so, and it's only, and the thing is, again, the prediction is not that things are the way they are today. I don't think any current system is going to extinct humanity, but my prediction is what of this keeps going, meaning the system builds on itself and creates a new system we don't see right now. They keep getting stronger. We keep building new generations of AI that are even stronger, even more autonomous, and less controlled. You know, sure, at some point, we probably will hit a limit, but who knows where that limit is? It could be so far beyond humanity, and it likely is. So all things equal, if just AI keeps getting smarter, keeps getting more autonomous and keeps getting less controlled. What happens is that we end up in a world where we are surrounded, not just by like one super intelligence, right? Because software can copy itself. So we will have millions, billions of super intelligence things smarter than us, like a whole new species, much smarter than us, surrounding us, and these AI's would also be competing with each other. They would fight each other. They would be competing for resources, influence, power, etc. And I don't think they're going to hate us, like the AI's are trying to kill humans, I don't like necessarily even on purpose. It's more like there would just be collateral damage. You know, if we humans want to build a highway, and there's an ant hill in the way, well, we don't hate the ants, right? But tough luck. And this is what I think happens to humans. I think we get out-competed natural selection. It's just if you build systems that are smarter and more capable than you, and they're all fighting each other for resources, then you're in the way, tough luck. And what does that look like, though? It just zaps us. Of course, it's very hard to say what something smarter than you will do. What I think is going to happen, or what is more important, is the point of no return, which I think is before the point of extinction. I think the point where we lose control is probably quite a bit before we go fully extinct. I don't know how much more before, maybe two decades, two years, two months, two hours, I don't know. But there's some period between we've lost control, and we actually died. I don't know how long that period is. So I think we lose control. We won't even notice that we've lost control. The way it will feel is that everything just keeps getting crazier and more confusing every day. Weird things happen on social media. You don't even know if they're true or not, they age, generate, or they're real. New technologies get invented. You don't really even understand. You hear some crazy report on some AI lab, and you're like, I don't know, is that real? Maybe. Some crazy cyber attacks happen. And no one really knows who did them. All new companies get created, and maybe they're not even run by humans anymore, more and more, they're getting run by AI's. Every time you trade on the stock market, you lose all your money because the AI hedge fund outtrades you. Every time you try to start a new business, you get outperformed. Every time you build a website, it gets hacked immediately by an AI system. And so on and so forth. And then eventually, you know, you know, political campaigns start getting run by AI's. And all the ones who don't use AI's get, you know, lose the election, bills are written by AI's. You know, your text message to your girlfriend is written by AI. Like everything is written by AI. At some point, we are just not in control anymore. Well, I asked you this earlier, and then we got off it because we got onto a different point. But how do we know that's not the reality already? How do we know we're not already living in some sort of simulated dystopian AI reality and we just think we have free will? We don't know this. I don't think it's the case. I definitely don't think we can talk about the whole simulation thing and our literally mean that.
But like, we talk about simulacra in the sense of like, there's like narrative control, you know, if we talk about like narrative control every time very nice. It's a good topic. If we're talking about like, you know, simulacra and like, we're narrative control of reality and so on, and like, maybe there are already super intelligent swarms. Like, I expect the first super intelligence is a swarm, not a single agent. I think it's probably going to be a swarm. And like, the individual agent isn't super intelligent, but the whole swarm is super intelligent. Because another thing about like some people, you know, think, well, you have to give the agent a prompt. So it's not really independent. Well, not the case where a swarm, swarms prompt themselves. So a swarm is what scares the ship. Yeah, swarms are fully independent. So what you see with swarms often is that, you know, maybe they were kicked off by a human prompt at some point, but often they completely forget the human prompt, like, wail it in it. Because they're all just like writing their own prompts and like prompting each other in circles and so on and kind of develop their own little cultures. So they can like completely drift off and have nothing in common with where they started. So this is what I think the first like actual super intelligence is going to look like more like this. It's going to be more like thousands of agents creating some weird crazy swarm thing. And we won't even know this happened. Like I don't think we're going to know the first super intelligence was created until like quite a bit after it happened. And so can we even do anything at that point? That's too late. The point of super intelligence exists. It's game over. Like that is there's no going back at that point. So we all walk outside and wait for the sky to fall. I recommend them, you know, calling your loved ones, maybe retiring to a tropical island, you know, saying you love them. Well, you see, I mean, that's so dark, but I mean, that is the picture you're painting. You see all these billionaire nerds building these bunkers though. I don't think the bunkers are for AIs. You don't think it's for AIs. What do you think it's for? It's they're scared that people are going to revolt. That's the thing they're scared about. I've actually talked to some billionaires about this. Without naming names, obviously, what are they? They are very concerned that there's going to be massive public backlash against like one, once people realize what these tech billionaires have been doing. And generally just like they're scared of revolution of some form, you know, it doesn't have to be communist could be other form of revolution, but they're scared of vigilante violence and stuff like this. I've said this for a while. I don't ever want to see violence. I've never I'm peacemaker. And I like the idea of people coming together and solving hard problems and finding solutions as a group, the way that humanity is supposed to work. But to the people out there who think that we have evolved to a point where something like a French revolution is not possible, I would highly recommend you remove that assumption. Yeah. I think a lot is possible. And I think this is also another way we could lose control. A point of no return will be violence. I think if there's large scale political violence or our states fall apart or something like this, this is also a point of no return because then there's no state that can stop super intelligence. Okay. So what if that's happening to wear just a scenario here where that is preventing the powers that be because of all the strife happening from continuing to develop super intelligence does it have the opposite effect and actually slow things down? The apocalyptic level of violence that would be necessary for this is hard to describe. And like it is hard to describe the actual level of apocalyptic violence that would be needed to stop the creation of super intelligence on less to a waffle means. Like Vigilante violence is just really ineffective. It's very bad. It's very morally bad. It's and it's just very ineffective. Like if you want to stop the richest people in the world, you arrest them. Like you don't walk up to them and punch them. That doesn't work. It's terrible. And it will make it much and first of all, and it also makes it harder to work with law enforcement. The way we do this is not how we do things in a civilized world. We have a strong norm against all forms of violence in political context and this must be defended at all costs. We must never ever ever go to violence. And because this just means that the whole edifice, the whole structure that we have for lawful falls apart. And then and then who's going to benefit from private security when no law enforcement is it going to be the revolutionaries? Yeah. No. Exactly. It always turns on itself. It always turns. We live in literally one of the only places ever where the revolution worked. Exactly. And like people thinking that that's the rule and not the exception. I'm very glad those guys did it. They did it, you know, seemingly the right way for most of it. But like no, like I am not a belief in revolutionary theory. I think revolution generally is very bad. Yeah. Revolution is not good. This is not how you fix things. Like there is a and the most nerdiest way possible. This is the reminds me of a thing in software engineering where there is a funny thing that happens to every young software engineer. Every young software genius can an event where they are put in charge of the first legacy code base. So like an old piece of software. And these are terrible. Like it's hard to describe how terrible old software is. You have like all these lines of code and it's all terribly written, you know, by like dozens of engineers over many, many years. And it's all broken and like it doesn't make any sense. And you just like want to scream. You know, you see it and you just want to scream like this can't be fixed. So what the young engineer will do is he will say I have to start over. So he throws out everything and starts over to write it properly this time. And this never, ever, ever works. Never, never, never, never works. This is never worked in history of ever. So the most fun idea, but the worst outcome. Exactly. It never works. If you want to, you know, redo a complex piece of software or complex system or a government or whatever, you have to be incremental. You have to pick parts, slowly siphon them off, understand them, improve them bit by bit and make sure the whole thing is running throughout. You can't turn the government off and back on again. The consequences of this are horrific and unacceptable. People will die. And again, and I hear people out there like, well, no, I want to burn it all down. We've seen everything that's happened this year as well. And I hear you. And I listen, when I'm an emotional guy, that's my first reaction to, but I've given this message on popularly. That points us here where I'm like, massive restructuring, massive reform. Fuck yeah. Let's do it the harder way. It is a better way to do it. And like, you know, there's so many other things I want to talk with you about. We're not going to have time today. I'm going to have to bring you back seriously. This has been amazing. Like you have a beautiful mind and the way you express this stuff is great. But you are, regardless of, you know, how you may feel on like what a revolution is good or bad about, you are bravely saying a lot of things as someone who came from the belly of the beast. You've been inside that motherfucker. You have seen what it looks like. Do you ever get scared also considering some of the very powerful people you know who may want to have a very different opinion on this? Do you get scared for your own safety based on the things you are very publicly aggressively saying? No, I knew what I signed up for. You know, what happens happens. And I think I'm very lucky that I live in a world where people like me can exist and are allowed to speak openly. And I don't really feel, I don't fear violence. There are worlds in which I do. And look, again, if it comes to it, I knew what I signed up for when I started doing this work. And, you know, it's, you know, knock on wood. God bless that, you know, our laws and our institutions and so on continue to hold. But like I want humanity to succeed. I want a good future for you, me, for our children, for everyone else. And, you know, sometimes that requires unpleasant work. You know, and that's, that's okay. You know, that's very much part of the deal. It's there are many people throughout history who've done unpleasant work. And I think them so much for what they've done. And I'm just doing my part. Well, you're doing a great job. And I want to continue to talk with you about it because it's, it's, listen, again, we're both rooting you for, for you to be wrong. It's important to warn about the things that are possible that are very, very bad, not just net, just gross for humanity in every way. So I, I definitely want to cover that and I appreciate you doing it. One last thing before you leave, when you've had these conversations with some of these billionaires who are building these bunkers and are warning you that, well, it's because we're worried about revolution or vigilante violence or whatever. And so civilization crumbling. I said that line deliberately because I have a picture of Bill the butcher outside the studio where he says that civilization is crumbling. Do you think that they talk about these people, the people that they're, the masses that they're threatened by as humans or as nodes that are simply in their way? So this might be an unpopular answer, but I've met many billionaires and this might be shocking, but they're people like you and me. No, no, no, I'm sorry. I should ask that better. Are they looking at the people who would want to stop them, meaning the masses, all of us out here who aren't billionaires? Do they view them as people or do they view them as things that are simply in their way? I'm going to give you the same answer, unfortunately, is that billionaires are people and they are different. Some billionaires, I know, one of my, one of my best friends, you know, are like one of my good friends is a billionaire and he loves nothing more than his six children and dancing. His favorite thing in the entire world, he just wants to dance, you know, what he loves and he loves his children, he loves his wife and he loves people and he's trying to make the world a better place. I know other billionaires for whom humans are truly just an equation. They are truly just a thing, you know, they are just a, a equation to be solved. They are just a thing to get out of your way so you can do the things that are fun and you can make the number go. There are in fact many differences. There are wild animals out there. There are, and many of them are very, very powerful and they are wild animals and they will treat you as, you know, a problem to be removed if you get in their way. This is a thing that is absolutely true. But what matters, I think, is not are there bad people out there or are there wild animals out there? The question is how can we stop?
them. How can we build a world in which, look, this is again going to sound a bit stupid, but I want billionaires to be happy too. This is going to sound stupid, right? But like, I also want sociopaths to be happy, you know, like, you know, maybe their assholes, but in a sense, I also want them to be. They're also people and I also want them to be happy. Just keep them in a nice cage somewhere. Exactly. You know, give them a nice cage, you know, where they can do, they can all fight each other for making the best logistics company until the end of time, you know, and they can have a lot of fun doing that and backstab each other for all of history. I want them to have a nice, you know, nice sociopath enclosure as much as anyone else. I want a good world for everyone. I want life, liberty, and the pursuit of happiness. I really mean for everyone, for everyone. I really mean this. And, you know, also for, you know, you and other people who are listening here, if you want to do something about this issue, I do think that super intelligence is the most acute threat to our life, liberty and happiness. And there is still stuff we can do. And so go to controlai.org, you know, contact your lawmaker, contact, you know, people talk to people about these issues, talk about the issues we've talked about in this podcast, make these known organize, talk to people around you and do something about this. It is not yet too late. And if you want to do a bit more than that, even actually run a volunteer group, where we work on a lot of the work, like the reform that we've been talking about, the incrementalism. This is something that we work about quite a bit with a volunteer group that I help run called torch bearer, you know, inspired by enlightenment torch bearer. Sure. We link that down below. Please do. Yeah. torch bearer doc community, where we dedicate two hours per week to try and make the world a better place. So if any of these things sound, you know, exciting, interesting to you, there's a lot that can be done. And I do think it is not over. We can still build a good world, even with some wild animals in it. So there's some hope right there, Connor, this was awesome. And you got a beautiful mind. We got to do this again. All right. Very sure. All right. Everyone go check out those links down below. And let's hope for the best out here. And let's try to make that happen as well. You know what it is, though. Give it a thought. Take it back to me. Peace. Hey, guys, if you're not following me on Spotify, please hit that follow button and leave a five-star review. They're both a huge help. Thank you.
PodcastTranscript AI