Go back

The Last Days of Group Home Rome feat. Hesse Deni

from TRASHFUTURE

67m 3s

The Last Days of Group Home Rome feat. Hesse Deni

The podcast features a critical examination of the AI safety community, particularly its roots in fan fiction, ideological extremism, and financial speculation. Host Riley and guest Hessa Denny explore how a group centered on figures like Eliezer Yudkowski developed a self-reinforcing culture of irrational belief, using fantasy narratives—especially Harry Potter fan fiction—to frame rationality as a magical, superior force. Despite lacking technical understanding of AI or computer science, members of this community promote extreme doomsday scenarios, such as rogue AI creating paperclips or resurrecting consciousnesses, and advocate for radical lifestyle changes like celibacy, sex parties, or living communally. Their influence is evident in major AI companies like Anthropic and OpenAI, where employees engage in private doomsday discussions and ideological alignment. The culture is described as a blend of effective altruism, financial speculation, and anti-establishment sentiment, often structured around summer camps and online communities that emphasize shared identity over factual accuracy. The podcast draws parallels to cult behavior, noting how the group uses logical-sounding jargon to mask ignorance, promotes sexualized rituals, and creates a sense of exclusivity through shared belief in existential threats. Ultimately, the discussion suggests that the AI safety movement is more a self-sustaining fantasy ecosystem than a grounded response to real technological risks, driven by emotional, social, and ideological needs rather than evidence or technical insight.

Transcription

12641 Words, 69263 Characters

English
Hey everybody, it's Riley, I'm back from traveling and Novus traveling again. We are going to be all together again soon in the next couple of weeks, but in the intervening time. We have friend of the show, Hessa Denny joining us to talk about, I guess, a fan thick community that got way out of control. Hello, pop it. Welcome to the show. Thank you so much for having me. I'm excited to talk about this fascinating community. If it can so be called, but before we get into that, though, there's something I feel I need to share with everybody, meaning Hussein now and the listener community, when hopefully we put this picture in episode description. Maybe it's just the episode art, which is the most Ohio item I believe that has ever been made. If you could please navigate to the first page of the notes. The image is right on top with zero context. Hessa, can you please describe to me the item on the left? Yeah. Okay. So this is a shot glass. The shot glass itself, it's a bit of a complicated structure for a glass. There's a bottom part that's a little thicker, you know, the main glass part is kind of sticking out of the top. And it's a black skull with kind of a gold shimmer to it. It has like an angry face and on the top in yellow script, it says Ohio. The only thing that could make that better, like this is a canonical stupid product. Yeah, this is a product for the profoundly stupid. Yeah, it's a dishwasher friendly. It's dishwasher unfriendly. It bullies the other dishes in the dishwasher. The only thing that could make this thing better is if instead of being written in the Comic Sans adjacent script, the sort of whimsical Ohio, if we could have gotten Ohio and like the monster energy gothic font, that would have made it probably the greatest product ever made. I'm also laughing, like imagining someone robbing a bank but wearing that as a mask and big black skull that does Ohio on the forehead. Well, that would be quite good, right? Because yeah, because you could sort of trick people and sort of like sort of knowing, but yeah, oh, yeah, this guy from Ohio robbed this bank. I mean, what we don't know is actually you actually live in a little place called Chiswick in Northwest Washington, down the fucking ground. Oh my god, it's the Ohio kid. Yeah, give me all your pounds stealing. This man was clearly from Ohio. Leave me alone. Just go back to Akron. We can't hurt anybody. I've never loved to throw away product as much as this piece, this beautiful artifact of stupid culture. Yeah, I don't want to have to be like bullied into returning it by like, you know, people who locally say the British Empire was bad for all of its looting. They'd be like, oh, you're you went to America, you went to Ohio, the heartland of stupid culture and you looted this skull shot class. Yeah. And bottle opener shaped like a gun. You found it in a fraternity burial mound. You dug it up. It was like, damn. Anyway, as much as I would like to spend one full hour or rather like an hour and 10 edited down to an hour, you know, little peak behind the curtain. Talking about my Ohio, my Ohio skull shot class instead for the listeners at home, we did talk about it for an hour or 10, but we edited it down much further. Honestly, there was not a lot more meat than this. But look, look, I think it's probably unavoidable. If you have the interest that we have and the interest that many of our listeners have and an internet connection, it's been unavoidable. This supreme community level crash out of the AI safety less wrong effective altruist constellation. And many of you may have read the allegations made in that AI safety is mainly a sex cult and Berkeley piece. If you haven't, you can't because it's now been taken down. Not gonna repeat all the conclusions made in the piece because not all of them are, let's say, legally airtight. But I think it's directionally correct that the AI safety people, such as the machine intelligence research institute, the various group homes like Light Haven, the bloggers, the posters, the fanfic people are, well, let's just let's let's get into it. But instead of reading a 107 page blog post, there's a smaller version in the Wall Street Journal. But first, I had both of you read the 107 page post and I just want to say now in front of everybody, I'm sorry I did that to you, my actual friends. It's totally fine. I mean, you know, it's not, it's not the worst thing I've ever read for a podcast. I think that would go to Nathan Masry's book that he wrote. I don't know if you guys know who that is, but he started Garfield Eats. He's not well familiar. Yeah, with let's just say I found that book very intergaging. Yeah, I was incredibly intergaged with that book. He talks about how he's, he wasn't actually gay. He just thought he was because he had a gin controlling his body, but it's the best of us. Yeah, honestly, that's more common than you think. Absolutely. But yeah, this, this document, I will say there's a lot of stuff in it. I did skim some of it. I'm going to be honest, but let's just say it's an interesting read. I'm saying I ruined your day, I think. So I printed it out and I enjoyed reading it on the tube and I was shaking my head the entire time to let everyone know on the train. But I disagreed with a lot of it. I disagreed with a lot of the people in there. I hope they took my message too hard. Yeah, but instead of reading a hundred and seven page blog posts, we're going to try synthesize that and not go insane. There is a smaller, a sort of smaller introversion, actually, the Wall Street Journal published because all of a sudden the entire mainstream media is getting very wise to the fact that these AI safety people by which we mean rationalist, effective altruists, a lot of the task real people that nexus of people of motherfuckers we've been talking about for years now. Everybody seems to have woken up to the idea at once that these people are grifters at best and dangerous at worth and not dangerous in the way that the AI they talk about is dangerous, which is dangerous to gorillions of theoretical future people, but dangerous to actual people around them now. So this is from the Wall Street Journal. Inside Berkeley is tallest office building, a small community of artificial intelligence researchers at a co-working space called constellation has been mobilizing to save the world from apocalypse. They enjoy catered vegan meals and bring their laptops to couches with sweeping views of the San Francisco Bay at dinners and happy hours almost every month. They discuss the latest and scariest AI risk. So basically yes, go story house. Yeah, but go story, go story house summer camp, but in an office building. One of my friends works at Google and I went to the Google headquarters like in New York one time and it really is like so funny like all these tech places they just they're obsessed with putting like comfy couches everywhere like in front of windows like oh if you need to sleep you can go to sleep at work at in one of these couch modules you know because yeah what we're doing is we're either saving the world with the burrito delivery app and then that kind of stopped hitting when interest rates went up and then it's okay well now we're saving the world by building the god that will kill it everyone but praying to it right so that it doesn't the resignation this month the journal goes on of an anthropic researcher Jacob Coxon woke up the American public to the shocking notion that run away ii development and humanity again can't but at constellation there are stalwarts of the ai safety community who spent more than a decade obsessing over it and that's the thing right that's the language that i think is kind of why i'm going to pick out here is the word obsessing because that's all they've ever done is obsessed about it it goes on they refine their arguments in bay area group houses and traded predictions at freewheeling conferences and Berkeley the Bahamas they floated ideas like buying islands stock buying iodine pills are moving to electromagnetically shielded bunkers in the desert at one twenty twenty two event the drink menu included a cocktail called death with dignity in reference to an essay arguing humanity was already doomed so in effect what they have done is they have talked and posted for a decade yeah more than a decade like there are less wrong posts from like two thousand eight basically like all these it's and like i also have a question what do you think is in the cocktail death with dignity i mean i can't imagine anything you'd want to be drinking yeah but i know about the rationalist is that the whole method that they all derived from their fan fiction that there's like their bible that we'll talk about in a bit is about taking an off-putting position and then acting very huffy and superior when someone objects so i think it's going to be a lot of a terrible Dutch egg yolk liqueur called bols advocate and and like celery juice and uh wheat grass you know it's a high protein not vegan but it is vegetarian cocktail tastes like muddy grass and if you don't like it you're you do not understand logic like me yeah and the i think also maybe there's like hand sanitizer in it is the alcohol i i do think that if it has to be vegan i don't think it can be the egg liqueur i think maybe it has to be just like vodka and way protein or something i would i wouldn't be surprised if even if it's vegan you've got a guy or two there who are like actually plants can feel things too and we just have to not eat anything except salt because this is the only thing that doesn't have a soul a couple of very - Very thirsty, fellas. - Yeah. Local dooms like those at Constellation have also steered the development of AI itself. They're among the earliest employees at OpenAI and Anthropic, both of which were found on the principle of staving off AI dangers, and they've kept up their influence drawing some staffers and funding from their projects for their network aligned with with effective altruism, including Sam Bankman Freed, who, by the way, bank rolled the vast majority of this, along with, before him, Peter Teal, and of course, a little bit of money from the guy who's everywhere, the evil forest gump Jeffrey Epstein. So the AI safety community's influence has been particularly strong in Anthropic, which is now only cost of a $2 trillion IPO. Employees there are trade doomsday scenarios and how to prepare for them on a private Slack channel, and if come to embrace a model they stick on their laptops, things will never be chill again. But like, if you have to believe that to work in Anthropic, how do you, like, look at the guy whose job is to, like, sell clawed to Deloitte? Yeah. Like, so insane, it's like, I guess, like, it kind of actually aligns weirdly with the effective altruist thing of, like, okay, if you're an effective altruist, step one is to get a job as Tom Wilkinson's character in Michael Clayton, where you're defending a wheat killer in a big trial, so you can get, like, $30 million a year, and then what you do is you donate all that money that you make to AI safety research. It's like, yeah, I'm just going to work for this AI company. It's probably best if I work there instead of someone else. And then I can donate, like, all my salary to a nonprofit that has, like, 50 people in common with this company that is basically, like, writing schizophrenic articles about how to stop AI, you know, like, it's this weird dissonance in their thinking. You have to work so hard, I imagine, to ignore the guy whose job it is to sell clawed to Deloitte. Why does Deloitte always like Talpa for corporate America? You have to work so hard to, like, avoid the guy whose job is to sell clawed to Deloitte, because, like, he's engaged in the other side of the coin, which is the money making. And you don't get that you're part of the marketing. You're part of the marketing for-- because Deloitte's, like, yes, Sherry want to use the death rate to build better consulting deck, I suppose. But the subculture, going back to the journal, the subculture organized around the fear that the AI could kill us all began to coalesce more than a decade before the invention of the underlying technology that enabled it. And this is where we're getting into our actual meat of what we're talking about about these rationalists, so they're called, self-self to identify rationalists. Its intellectual leader is Eliezer Yudkowski, discussed frequently on this show, which the article describes as a Bay Area Autodidact who dropped out of middle school, who by the 2000s was blogging voluminously about cognitive biases and running an institute devoted to warning about the risk of a runaway AI. And the thing about, yeah, that this article doesn't say is that he doesn't know how to code. He doesn't understand computers. He doesn't understand rationalism. He doesn't understand science. It's really-- you can tell that he dropped out of middle school, reading some of the articles on "Less Wrong" that he's written. You found a favorite. Yeah. Yeah. So this is a great one from "Less Wrong." It's called "Many Worlds, One Best Guess." And this is so-- I'm just going to read some of it, OK? If you-- basically the gist of this article is that all the stuff that happens with like quantum particles, that stuff also happens with people. And like you can tell-- Oh, that's cool. Yeah, yeah. It's interesting. Like super positions, waveforms, you know, like all that stuff that, you know, is basically like a quantum particles or many things at once. All that stuff is true of every single human being in the world, which basically you can extrapolate to be like cause and effect actually does not exist. There is no such thing as determinism. Why? You know what happened is he listened to bittersweet symphony, and it was like, damn, that's true, that's true of people. It's-- he understands stuff by analogy, I guess, and then just hugely oversells every single conclusion. Yeah. I'll just read a little bit of the beginning. If you look at many microscopic physical phenomena, a photon, an electron, a hydrogen atom, a laser, and a million other known experimental setups, it's possible to come up with simple laws that seem to govern all small things. So long as you don't ask about gravity, these laws govern the evolution of a highly abstract and mathematical object that I've been calling the amplitude distribution, but which is more widely referred to as the wave function. And it's so funny to be like, actually, I have a different name for it, you know, this scientific thing that we've known about for like decades and decades. And this, I love this part, this is really like where the schizophrenia kicks in. Now there are gruesome questions about the proper generalization that covers all these tiny cases. Call an object grew if it appears before January 1st, 2020, and appears blue thereafter. If all emeralds examined so far have appeared green, is the proper generalization emeralds are green, or emeralds are grew? I love that movie. Which ones are minions? What is he talking about? What does that mean? What literally what is he talking about? It's so insane. Here's the thing, and this is a little bit of a skeleton key into everything I think about these people and have always thought, if you cannot explain yourself in terms that someone else can't understand, you are disguising the fact you have no fucking idea what you're talking about. Yeah. Absolutely. The more you use insider jargon and baffle gab and things that you cannot then explain, if you say, well, something's unexplainable, you don't understand it. Basically, like if you cannot make clear what you mean, you don't get what you're talking about. You don't get all things have to be simple. You can express something in complex terms, but you should be able to communicate what it means and not hide behind jargon. So the rest of this article goes into tracing the alignment between anthropic, less strong, effective altruism, give well, will, McCaskill, all these people. Suffice to say that Eliezer Yudkowski is very close with everyone who matters, but there's a common thread in all of it, which is all of these things, whether it was effective altruism or rationalism and AI safety or whatever. It's all believes that you can reason from first principles using whatever thing you're good at to save the world, and usually this is rich guys. I mean, in the 1980s, we got live aid out of it, right? We're a bunch of rock stars. We're like, we're going to write the song that saves Africa from like, you know, famine of the country of Africa. We're going to save it from famine. Yeah. Well, how many people are here? Like, famine's over. Yeah. Yeah. So famine's over if you want it. You may know as Christmas time, oh, yeah. Yeah. These with live aid, there was a concert that was played. Yeah. Under pressure, one of the best versions, you know, you can't read mercury. Imagine live aid for like the 2020s, but instead of like music, it's live podcasts. You have like a whole day of live podcasts. Yes. Oh, my God. Do they know it's blue two after all? Yeah. Yeah. Yeah. And like, there can be like a special like six hour episode of Joe Rogan that becomes like the under pressure of all of our time. Yeah. Joe Rogan did kind of a short episode, but it was honestly like the best, it was the best version of like him talking to, you know, all the raising money for Charlie Kirk. The raising money to build big Charlie Kirk statue. There's one experimental podcast episode that's like two AIs talking to each other and eventually at some point it devolves to them each repeating to each other over and over. I'm sorry. I don't understand what you mean. And the other one, I'm sorry. And they're all like watching it like, oh, my God. This is actually. Yeah. This can also be in service to Africa, the country because I bet no one in Africa knows that it's Kirk. Come this time. Yes. Oh, my God. Yeah. You go to Africa and you ask, you ask about like, do they know about Charlie Kirk and they just sort of look at you with like a bemused confusion. They all have like the losing Super Bowl team like Super Bowl winners that they all have shirts that say like Charlie Kirk alive. Still. We're doing live-aid 2026. It's a little late to announce it. We're going to build an 80 foot high Charlie Kirk statue in Orania, South Africa. Exactly. Both helping Africa, the country. But that's the thing. Right. Live-aid, a bunch of rich guys got together and we're like, maybe guitar music will save the world. And then something like give well was actually like hedge fund. Well, the effect of altruism started as hedge fund people getting together saying, well, what if we applied hedge fund methods to save the world? To discounted cashful analysis or whatever, like to saving lives. And that starts as mosquito nets, which is, well, and it's the genesis of that idea or it's this, it's the popularization of this idea that is throughout this thing like a sort of vein of terrible butter, which is anything that is not based on like, it's very rigid and narrow version of rationality, it's, it's an evolutionary glitch. It's a bias. It's like, salient. Well, why would you save your own kid and not sort of, you know, hit a button to distribute a million mosquito nets that would actually like add more qualities to the world? And it's like, I don't know that's not how humans are. It's not a virtue to be sort of off putting in strange. You might not understand the reasons for like why people behave certain ways. But give well was this. And like it's, I also love like it opens up this thing of like using just like rationalism like the language of it. And like it's, it's literally like, it seems like rationalism is a fundamental misunderstanding of what like logic is as like a concept where it's like instead of this being just like a language or something like a system that is used to explain ideas and like explore ideas. It's like, oh, this is actually just the correct. It's like looking at a language and being like actually everything in this language has a hidden message that only I understand. And the message is like, oh, we need to just get rid of hospices, you know, because like those people are already dead, like we're wasting money, we could give it to, you know, we could spend the money on mosquito nets instead of, you know. But the crazy thing is at least we got some mosquito nets out of that. Yeah, yeah, true. Oh, but then the thing is like any super financialized project that contains the season of its own destruction because it's so simple. It gets very easily, basically prompt injected. It gets very easily hacked because just like normal fucking finance capitalism, you can tell the bigger, crazier story with bigger numbers that's so much more ethereal and hard to grab, but a 0.1% chance of an infinity gazillion, you know, negative outcome is way, way larger than, you know, being able to even distributing mosquito nets. It's like, well, what about theoretical people living in a computer who don't know their people? There's like potentially infinite of those. So we need to make sure that a rogue AI doesn't mess with them because we've told, we keep telling ourselves escalating stories and escalating stories and escalating stories. And the crazy thing is, right? The article says, you know, AI safety is mostly a sex cult based in Berkeley. And, you know, I'm not going to sort of get into that bit. But what I actually, what I'm having thought about this for a long time, I think it's actually like the finance element of this, of like effective altruism, gets consumed by AI safety by posting, blogging, influencing, telling bigger and crazier stories. It's tech industry QAnon, right? It's like so fascinating. I'm obsessed with like the examples in the document of like, AI policy memos that are like, well, in 2035, AI is going to be able to download people's consciousnesses and resurrect them whenever it wants, like whether they want to be alive or not. And it's like, what are you talking, what the fuck are you talking about? That is so insane, like just like being like, oh, AI is going to, we're going to have a Dyson sphere in 2040, like what is going on? We're all going to play Dyson sphere. Yeah, like think about this. If you just replace AI with Hillary Clinton, it's exactly QAnon. Yeah, and it's why they always aligned on like, oh, the Democratic parties eating babies. So what do you do? Go on Facebook and vote for Republicans. And Thropic is going to kill all humans. So we need to have more clothing optional parties, I guess. Like it's the so what is always disconnected from the mechanics of a thing you're talking about. It's like, oh, there's too many pedophiles in government. Better if we better elect this white hat pedophile. Oh, oh, there's, and Thropic's going to kill all humans. Okay, well, we don't understand that we're the marketing department for Anthropic. We're going to go to a clothing optional beach. Uh, and by the way, everyone has to go because it's also all your work colleagues and you being pressured to by your job. Uh, but the guy who sells Anthropic to Deloitte, he doesn't have to do that. Yeah, that guy just has a pretty normal life. He gets to go play squash with his, you know, with the guy from Deloitte. You know, he's going, uh, he's going to dinner, a nice dinner. Like, meanwhile, these other people are like, oh, what if we do like, we, we only eat celery for a year and we like live in a small box. Like, what if we only eat celery for a year? Also, all of you are my girlfriend now. Yeah, wait, should we get Ted who's the Deloitte sales up? No, no, no, he's not important for this. Yeah, yeah, no, he will quit. He would quit if we made him do this. So we're not going to do that. Like, we need him. No, don't worry. We're on the made up horse shit team. Yeah, yeah, AI is risky. There's a lot of risks associated with it. We're living through a bunch of them. It's not just like data centers, poisoning communities. It's not just all of like digital art jobs going. It's even like, yeah, a lot of it works as like autonomous malware. That's pretty bad. Yeah, but it's all understandable. It none of it involves a kind of suddenly what they plug in one to many Vera Rubin GPUs and then a digital God wakes up and starts hatching and pursuing its own plans independently of like parameters set by people. But the way that you do that, the way they create a community around it, is the same way that the QAnon created its community. Posting, posting, posting, posting. Give me money, give me money, give me money. It's, oh, it's so crazy. It's so, it's so extreme. Well, the world's going to end. Post more, follow me, good mood in some cases. Donate $100 million to me. When you're also donating $100 million to the company that, like, you can offset it, you know? And I work for both companies, so it's perfect, you know? I get a slice of both pies. It's really incredible. Like, some of these AI safety things, like, Meary, what is, what is it again? It's like machine intelligence research institute. Yeah, yeah. Like, it really seems like they just, all they do is think of, like, sci-fi stories that could happen. If the one in a quadrillion-chance thing happens, an AI, which it won't even, 'cause there's not even any chance of it, really, is like that AI gets smart enough to fix itself and become, like, actually smart. And it's like, yeah, the more we build it, the closer it gets to this. And it's not doing anything. It's just, like, making fanfic, basically. As you were saying, man, also, as I was reading the document, one thing I kept coming back to was just like, this sort of reminds me of the cybernetic culture research unit, the CCRU, which was, like, I don't know how to describe it. It was kind of like a collective at Warwick University in 1995, and, like, Mark Fisher was involved. Nick Land was involved. Although, like, there was a lot of, like, sci-fi that came out of that, and so much of, like, they're thinking in so much of their writing, but then coming from, like, philosophy and social science backgrounds was also kind of anchored around this idea of, like, machine-driven apocalypse and transhumanism and transcendent, and, like, sort of, machine transcendence and so on. And at that time, I don't know, like, CCRU obviously, like, seems a lot cooler, and also they have, you know, they weren't receiving, like, tons of government money or funding or, like, you know, the ability to sort of, like, shape societies. But, like, I think in terms of just, like, yeah, these are sort of guys who are overeducated sci-fi nerds whose job it seems to be to just, like, predict various doomsday scenarios and their solutions to all of it are, like, give me money so that I can post, like, I can write about it on my blog. Which I guess is, like, a great way to sort of get a gig, but, like, when you really think about it, it's very much, like, it's not just, so they're getting money to, like, blog and post and, like, have these weird sex parties or whatever, but, yeah, they're also getting the chance to reshape the entire global economy based on vibes. - Like, bad vibes. - Yeah. - I guess, like, which is fun. - Yeah. - It's like, they are overeducated, but, like, they're overeducated in, like, tech bullshit, you know? And in them being overeducated, it's like, oh, they, growing up, they were, like, you know, number one in their class, you know, very socially, probably, maladjusted, it seems like a lot of these people, they're like, oh, this is actually, like, a gateway into a community that I can enter, because I know about tech and all this stuff. But, like, you see it with Yudd as well. Like, he dropped out of middle school, you know? He's, like, a smart guy, clearly, but he's an auto-died act, as it says. And so he just decided to, like, learn the things that say he's right, and just ignore all the stuff that says that, actually, he might be wrong, you know? And it's the same with all these guys. They're, like, they learn about quantum physics, quantum mechanics, which is something that it takes, like, decades to learn, you know, to become an expert in that. And they're, like, actually, I think I get it, you know? I read a few blog posts. I read a paper on it, which, of course, you know, they can't understand the fucking paper on quantum mechanics, but they're, like, actually, this means that quantum immortality is real. And if you get shot in the face with a shotgun, you'll survive. And, you know, your life will be hell. It's literally just, like, a sci-fi nerd brain thing. It really reminds me of Omshinrikyo, in a crazy way. You could sort of draw the distinction between the different kinds of nerd, right? There's the sort of nerd who just is really into Star Wars. And then there's the nerd who's also a rocket scientist. These got conflated culturally. And these guys are the main beneficiaries of it. Yeah, you know, there's always been, like, a touch of that in the science world, you know, like Jack Parsons, the guy who basically invented rockets and jets. It's like, he was like, you know, doing, he was like trying to summon the Scarlet woman with Alistair Crowley and shit, like, and, you know, Elron Hubbard, same exact thing. And, you know, it's, it's very funny, like, but these guys are like, they're really not, they're not inventing jets, you know. They're inventing a, like, a friend that they can talk to, that they're also like, I think my friend has ill intentions. The one that I invented. My imaginary friend is plotting my death. Yeah, yeah. But the article goes on to draw these numerous connections, like I say, between FTX, Dario, Yud, even Sama, and how the ex-risk used to be about global catastrophic risks, like comic strikes and super volcanoes and pandemics and gotta be suck if we pray out for those. Nuclear war was a huge one. Like, people always talk about this guy claxon or whatever his name is, the guy who quit and was like, actually, like, there's a 10% chance that we'll all be dead based on this one metric, you know, and it's like, that metric was like, there's a, throughout the entire, like, 50s to the 1990s, that metric was like, there's a 50% chance that humanity won't be around in the next 10 years because of nuclear war. And so, like, looking at it from that perspective, it's like, that's actually not that big of a risk on his side. Yeah, yeah, 10%. There's a 100% chance humanity won't be around in, like, three, 400 years. Keep taking oil out of the ground. Yeah, and the thing that these people don't care about at all. they don't give a fuck about this is the larger chance that like oh this is just gonna get rid of like all jobs like no one's gonna be able to have a job anymore because companies are just gonna use AI to replace their you know white collar jobs and shit because you what because that's not a more fun story yeah that's not that's not a cool power fantasy okay so I'll go on at this point like some of the guys who started give well these hedge funders started give well began started reading you'd Kowski and the thing that out that L.A.'s or you'd Kowski's writing does is it flatters a certain kind of person and suggests to them that maybe they are super powerful that they have inside knowledge that they have access to a kind of magic right that the rest of the world doesn't which is the magic of being like two standard deviations higher and intelligence than everyone else and you know that because like you can do the Sherlock thing where he thinks really fast about a bunch of unrelated stuff yeah and you have no friends also is another part of it and I think like it's I do agree with the document I know you said we wouldn't mention this aspect of the document but it does seem like it's targeted towards children being like yeah yeah it really is get that mean there's a lot of there's a lot of summer camps for teens and stuff oh god it's targeted towards children and also adults who behave like children another sort of look ahead if we're going with this so by 2014 the give well founders who've been reading get Kowski's writings with interest in skepticism for years they came persuaded by arguments about the importance of protecting the world from rogue AI if the name of the game was to save human lives preventing AI from wiping out humanity potentially had to be the most philanthropic bang for the buck of any cause so screw the live aid concerts screw the mosquito nets we are all fully in on the imaginary friend slash and the imaginary friend of me be basically yeah or says even a five percent chance meant that you'd Kowski is fellow rationalist now to a new set of converts in the quantitatively obsessed EA community in 2022 some rumors from anthropic in the other AI labs flew to retreat on the remote the homie in island of of eluthera 110 mile ribative land known for its pink sand beaches they talked about AI risk and effective altruism between sessions of sunset yoga cliff diving and a closing optional run into the sea is that's the other thing and that's the way I've been hinting at is that you'd Kowski's whole gang his whole codery of insiders of rationalist central altruist whatever insiders who all think that they've up they've perfected their priors they are Bayesian Superman weirdly a huge amount of it is having sex with them or being sexual around them or having porous sexual boundaries having sexual boundaries apparently means that you're unoptimized or like victim of cognitive bias right these are these are irrational thoughts that you're having yeah I mean the Wall Street Journal doesn't major on it quite like the other sort of much longer piece of writing we we're reading before this does but like it's fucking everywhere I mean it's everywhere I mean the whole fucking all of FTX was fucking polyamorous together yeah I love the Jacob Jacob Cox and when someone when Ed Zitron the goat was like what is your relationship to the rationalists and effective altruists and he basically said like I don't think I'm a rationalist because I don't like polyamory and I like eating meat and it's so funny but he also like edges that in a way where it's like I think they have some really good ideas and they're geniuses and they're awesome but also like I don't want to fuck how he has earned your cowsky it's like everyone who's an AI Doom or seems to be polyamorous with each other and I'll believe in AI Doom when someone who isn't from the polycule espouse yeah yeah so this is your kowski and his friends holding the same kind of events where they have the same kind of conversations where they say there's a we've now updated our priors there's a 1.1 percent chance that a trillion digital humans could all be tortured for eternity by a rogue AI and then everyone's like oh wow that's very yeah they but then they be they get enough money that they begin to buy lovely hotels in the Bay area they begin to convert them into group homes they start living together they start living communally they also that this is when they meet Aiella who features hugely in this this much longer piece of writing that we talked about the AI safety is mostly a sex cult based in Berkeley California I mean before we sort of switch into that the last line I want to read from the journal here I think is really telling which is one luncheon was advertised as open only to people who believe there was a 75% or greater chance of human extinction over the next 100 years the invitation said I hope the result of this will be reduced social censorship pressure against people who think the world is doing I hope everyone stops thinking work ludicrous and wrong and pompous assholes but I mentioned the Aiella stuff this stuff that's not in the Wall Street Journal article is much more I know laureate if you want to use that word where Aiella is a auto-diag sex researcher former cam girl sex worker again I don't want to get too much into the specifics here because a lot of it is quite you know let's say legally relevant but who appears to be at the center of organizing the quite extreme sexual life of the AI safety community making a big public deal of it and it seems collecting women and using their sexual availability as a currency to further the goals of the AI safety movement for example like two days before Jacob coxon quit and thropic Aiella posted I will throw a reverse gang bank to anybody who quits a big lab and whistle blows on it two days before what is a reverse gang bang dude so funny it's like many women when he many ladies one fella yeah yeah that do you think that's why he quit like he was like you know maybe honestly it's it's the timing is awful convenient without getting in too much detail right this seems to be there's like a person here whose function is get women and create a sort of environment of extreme sexual openness and then wealthy men pay to attend slutcon where your VIP ticket gets you access to places where you might expect to have sex with some of the women who are at slutcon because maybe they want to get more access to the industry or whatever that seems quite suspect to me yeah but they also it's like you know they say we want to have our quote unquote sex parties where they practice I would say consensual non consent uh I want to go to the glory details of it but it's basically sexual assault simulation parties but it's one of those where the person doing it is your boss or the person who's your boss slash landlord so like it's very difficult to know where the simulation begins and ends let's say and there have also been like numerous allegations like made by women in these communities that hey this same codery of men that's in charge of everything whose whole belief system seems to be that everyone who thinks any differently from me or has a boundary that I disagree with is wrong on like a theological level on to logic really blog about it yeah that seems insanely dangerous yeah I mean obviously it's it's crazy and like even the people who are like pointing out that it's dangerous they still talk in the same like AI researcher brain kind of way where it's very odd like it's so nasty and like crazy and like if I was in a scene and people were like oh we're having like a rape simulation party this weekend I would be like okay I don't I don't know if I want to be around like that doesn't sound fun at all to me yeah yeah like if the person inviting you is your boss landlord yeah exactly also like if if your whole thing is we're the only people because of our posts standing in the way we need millions of dollars hey by the way politicians including unfortunately Bernie Sanders please invite us to write policy on things which they do yeah by the way they are influential if their whole thing is we need to make sure that we are influencing things so we stop the extinction of humanity and then someone's like hey a lot of people really don't are hugely put off by the raporges they're like we can't stop doing those because that's what we do and so I guess we're just going to have to accept that us continuing to do that is worth whatever higher percentage of a quintillion digital humans getting tortured forever by an AI unlike the mosquito nets which are no longer buying by the way yeah it doesn't that sort of markedly sound very familiar to a certain form of formally deceased international pedophile and as accomplices and associates and the reason why despite a lot of public knowledge about not only what was happening but where it was happening and who was attending and who was writing about it publicly on blogs that it was sort of you know a blind eye was turned against the turn turned on it for so long and you know and I think in this instance it's very much like on the one hand just being like yeah we've we've seen this before and not even that but it wasn't even that long ago right like it is still very much present like you know the Epstein stuff but it's also one of those examples of just like well the AI industries like mythology's have been so ingrained into like international consciousness in terms of like what it is and what it ought to do and like who the only people who will sort of be able to kind of not only usher into sort of existence but to do so in a way that doesn't quote unquote like wipe out humanity and therefore everything else they do can just kind of be brushed off as like a bit of a quirk and like you know I think the point of like people like Ayala for example although like I'm sure it's not just sort of exclusively Ayala is for the people within that kind of culture community cult whatever you want to describe as a way of like reinforcing its logics right the one where it's sort of you know then the one that also kind of suggests that like there is an embodied principle to it as well this isn't just like an intellectual movement this isn't one where we just kind of do for experiments [BLANK_AUDIO] like how you sort of get people in, how you get them to stay in, but also how you get them to sort of like be willing to go further is to basically say about like, you know, your entire existence, your entire embodied existence is anchored in the logics of effective altruism and accelerationism and like all the sort of isms that are powering the sort of AI mythology right now. Yeah, and I also don't want to be like, you know, I'm like none of these women who are going to these parties are actually into this. I'm sure there are some that are like, you know, this is a fun, like this is a kink that I have, you know, not fun necessarily, but like this is something that I'm fine with and I'm fine acting this out in a, I'm sure there's at least a few like women who are like, yeah, absolutely, but I think it's a problem to be like, let's have a party about this, you know, let's, you know, if let's make this the norm, let's kind of, you know, everyone we're gonna we're gonna do this and especially with the guys involved who are like, yeah, actually this is my thing is that I like to have sex with women who don't want to be having sex with me who like are actively like fighting it and it's like, that's a little bit, it's a little bit worrying and like if that really, like if you are a guy and that is like your kink, I would think that that would be a very private thing that you wouldn't want anyone to know about unless you really trusted them and we're like, you know, dating them and understood that they, you know, but instead these people are like, they're so logic-brained that they're like logic their way into thinking like actually it's awesome that I think this that I this is my kink, it's actually like there's like a rightness to it, which is in itself extremely worrying to be like actually like this kind of power dynamic thing that I'm into is the is quite natural, you know, you'd almost have to be a quite extreme misogynist to think that. Yeah, yeah, yeah, exactly, but the crazier thing about it is that all of it did come from the Harry Potter fan fiction that we have referenced this show completely before. Oh my god, remember how I apologize to say and it has that early on for making them read the much longer piece? Well, it's because it exposed them much more closely to the content of the Harry Potter fan fiction. Yeah, you go into all the Harry Potter shops in Central London and like no one's stocking the print copy, I asked so many it. Excuse me, do you sell the dictionary, the smartest book in the world? Yeah, we have this 2-3-seam monster energy instead. Yeah, yeah, I did read a little bit of it, the Harry Potter fanfic and my god, dude. Yeah, it's good. It's just legitimately good. I think it's what you're about to say, right? Yeah, it's it's I'm actually like completely it makes so many good points and I'm actually on their side after reading it. I'm like this version of Harry Potter where Harry Potter uses facts and logic to destroy his superiors. It really is like something that someone who dropped out of middle school would write in a very fun way, but like I I have it here right now in front of me and I would just like to read some of the chapter titles. I'm sure they're fantastic. Okay, okay, we've got the first chapter a day of very low probability. The chapter two, everything I believe is false. You know, chapter eight, positive bias, chapter nine, title redacted part one, chapter 10, self-awareness part two, chapter 13, asking the wrong questions, chapter 20, basis theorem. If you ask you what he was doing, right? Is he wanted to expound his way of thinking in a way that would appeal to a mass market of readers. And so he inserted his philosophy into this fan fiction. You know, the mass market of readers that you would attract with the Harry Potter fan fiction who are on the internet maybe are alone and looking for someone who makes them feel less alone because of the kind of person who reads Harry Potter fan fiction on the internet. Maybe as a teenager in the mid 2000s, you know, it's almost as though again, this is not for a mass audience. This is for getting a teen audience. Yeah, that you know, because a lot of the people who are with him now are much younger. They're in their 20s, right? But they started following him as a teen and going to his summer camps as teens. And the Harry Potter fan fiction itself as well, the whole thrust of it, right? Is that Harry who has the actual magic, which is the method of rationality, uses it to be effortlessly correct when everybody else around him is stupid and wrong and doesn't have a magic like he does. Yeah, adults especially. Yeah, you can't trust adults who don't believe in magic. Yeah. And even if it's like, oh, the Harry Potter came out a long time ago. You know, if you read it as a kid, then you're like, you're an adult now basically even using that as a defense. Like people who take Harry Potter seriously still are completely emotionally stunted. I feel like they're all wrong people. Yeah, exactly. And like, you know, you've got like eight chapters titled the Stanford prison experiment in this book. There's a whole section of chapters called taboo trade-offs, the prelude of which is called cheating. You know, the horns of fact, there's so many like crazy hedonistic awareness. Like, it's so scary that some of these titles for these chapters, it's very funny. Like, I can't imagine taking this completely seriously, being like, wait, this is actually genius. This is actually go to the people that cults tend to go after are people who would read that and where it would fill a huge gaping hole in them where things like, wait, you're trying to wonder maybe why am I not socially accepted? And maybe I am some category of outcast. That's who joins cults because it creates a cut because the thing that they get most from it from it yet generally is community and a sense of shared identity. And that's what this book is all about. It's about rationality as a shared identity, but it's specifically not about rationality as like epistemic humility and true seeking by finding where you are wrong. It's about using the language of rationality as a kind of cudgel to have humiliated to mean people around you. So you can all prove that you're insiders together who have magic as opposed to people who foolishly care about their own children because they're, you know, evolutionarily biased or whatever. But I've been turning over angle after angle to go into this, right? And well, the thing that keeps jumping back to me, I think, is at one of the summer camps that they do called please don't kill us or PDK you, which is like a creator boot camp for, I guess, posters, like it's a poster's boot camp. One of the things in the application form is a hypothetical question, but whether or not you would have sex with a live or dead cow. If you have to have sex with the cow, would you have bad to have sex with a live cow or a dead cow? Why? And, you know, this is, this is sort of pointed out already that one of the things that this question filters for is people also have good boundaries. Yeah, but it's also a ridiculous hypothetical. And that leads me into my actual conclusion, which is it's this bizarre hypotheticals. It's the power fantasies. It's the sexual life that's one step shy of doing the angry pirate, which is a bunch of 12-year-old school shooters all got Tom Hanks big into adults. And they have infiltrated the policy governing the thing that's supposed to solve all the problems in the world and replace the economy. And it's like a long time for people to realize that that's the case. Yeah, the idea that there are children's camps like about this is so scary. It's like, what is going? It's like if Anders Brevick was like, I'm going to make my own island camp for kids based for based kids. Yeah, yeah. Just think about it, right? As you say, fanfic, we're a friendless smart kid out with every authority figure and it's never wrong. A summer camp culture that they keep coming back to, whether that's like C-Far, PDKU, or even now it grows up to be slut-con, which is happening. It happened last weekend. And then it's like grown men who are all playing harems and constant rape jokes is workplace banter with the and like the hypothetical questions about having a sex with a dead cow to save the world and never actually doing anything or learning anything other than just posting posting, but the thing is most important about the AI safety people at least in this orbit is they don't understand computers. They don't understand computers at all. Yeah, I'm something of a computer understander. They don't understand computers. They do understand that the security problems that they fantasize about like oh, well, the rogue agent, how it's achieved sentience, would be able to communicate itself out into the world by modulating the heat of the data center. They don't understand that like the actual security problems that these things have are generally solvable. And then if you write a piece of software that has an unsolvable security problem, you don't ship it. You've written bad software. Yeah, that's just it. And like the paper clipping thing, which you know the AI sort of decides it's a paperclip maximizer in terms of the world and the universe into paper clips series of license fears that then get themselves paperclip. It's a fun joke. Therefore that to happen, it requires a categorically different thing than a transformer. And they theorize about security without actually understanding it, but because they tell such a good story at least to one another and hedge fund people, they're now been basically foisted upon all of us. So I've written this. I actually wrote this out because I want to say the whole thing. AI safety people do understand computers that they actually exist. And as people actually use them, so they're understanding of the world and the demands that flow from it are pitched at the level that the computer is only understandable as a magical God. Because to Ali Azar Yudkowski, it actually is. Yeah, no. Whether it's it's and like it's the rule of God like as communicated by him. Like he's like I'm bringing the 10 commandments down from the from the mountaintop. And he is literally it is so like just extrapolating to an insane degree of like, computers used to only be able to play like snake, you know, and now they can do your homework for you. And he's like, so that means like in 10 years, they're going to be able to fucking manipulate matter. Like a computer will be able to physically change a human being into a paperclip. Like it's, it's so stupid and just nakedly ridiculous. And so like completely sci-fi and anyone on earth would be able to recognize it as such, except for them who are like actually like this is like, you know, it's the same thing as like the Bible where that some some of them like everyone in anthropic beliefs in this stuff. Some of them are like fundamentalists where they're like, no, this is literally true. And other ones are like, well, you know, this isn't literally true, but it's it's there's a grain of truth to it. And it's like both of those things are completely crazy to think. You know, I think like one way to sort of see is like, there is a difference between like fear logians and evangelists. Yeah, but they're often sort of like seen and kind of like perceived in the same way by people who have no interests and either. So like if you were a fear logian, like, you know, there is a very sort of specific, depending on what type of like practice or discipline you come from, there are like some to some degrees even kind of like sciences in which like you study scripture and text. Like I can only sort of speak for like Islamic studies. But like, you know, when you are doing like proper like Hadeef science, like the science of kind of like chronic studies and stuff like there is a key method and a sort of like system in which you have to like approach a text before you can even kind of make any judgment. And there are lots of like working fear logians like within theology departments both in secular and in religious institutions who spend their entire lives like dissecting texts. And they will sort of save it like I have no right to sort of make a judgment as to what this text infers or what it suggests, let alone sort of interpret the word of God. All I can sort of say, it all I can do is sort of like contextualize parts of it and they'll like spend their entire lives doing that. And then you'll sort of have like evangelists who like you kind of just see hanging out in streets, you know, and they sort of like draw on some of the work of the fear logians potentially, but they massively misinterpret it or they sort of like use it to kind of fulfill their own ends because their objectives are sort of like even though they kind of appear to be similar, their objectives are broadly different in terms of like what they are actually preaching. And so for the evangelists, it's very much about like reaffirming the Doomsday scenario, reaffirming the Doomeradumerism, the sort of like preparation, preparation, preparation, and in order to do that, like you should join my side in order for, and you know, there is like spiritual fulfillment for doing so, right? That's a very sort of simplistic way of kind of like dissecting the two. And I'm not sort of saying, but like it maps on perfectly to like what's going on here. But if we think about like VAI community as like, you know, having this distinction, these guys, like, you know, you know, you are yellows, you're your cow skis, even the sort of like heads of the sort of AI, like open AI and anthropic, like they should be seen as evangelists, you should be seen, but they are presenting in public, suddenly some of them are positing themselves as fear logians, and they're doing so to a mass audience that like cannot really tell the difference between the two. And it's also like a situation where they know nothing about computers, they know nothing about the computer science. A lot of them don't know how to like do very basic coding, let alone anything else. But that doesn't actually master because what they kind of recognize is that the story is more important. And the story is the thing that is like making the shifts. The story is the thing that is like making Western governments kind of capitulates of them, making AI ministers kind of just repeat what the evangelists are telling them. And no one really being able to sort of be like, Hey, that's that's kind of bullshit. And also like doesn't really make any sense. And so like they kind of, you know, they do, they sort of play off each other. And I do wonder whether like Vitech guys who actually do know how to code are kind of like benefiting from the evangelists in being able to sort of just like have license to do what they want and also to sort of like ascertain like more power in doing so. Yeah, I'll tell you who it benefits. It benefits the guy who's not coming to the CNC orgies. The guy with the normal life whose job is to go sell clawed to fucking Deloitte because he sells more clawed to Deloitte because Deloitte wants a death ray. Yeah, no one at Deloitte's fucking let me tell you about much. Yeah, it's not the salt, you know, it's not, it's not Lockheed Martin a famous getting dick down company. Absolutely, girl. Also like you're so right who say it and like the the evangelism they're doing is like it's a Jehovah's Witnesses, you know, it's it's like the end is coming. There's nothing we can do. There is literally, there is absolutely nothing we can do except you can perhaps if you have the chops, you can become part of the elect of the select few which, uh, you know, gets to fuck me after after the world ends, you know, which gets to have my children because obviously me, Elise are Yodkowski. Well, we all know we've seen my writings. We know that I'm actually like the peak of the world and I need to sigh or children. Like that's the most important thing. They're all obsessed with having like having like a gangus con amount of children. Like all of these tech people, it's so odd. One official, this is going back to the Wall Street journal. I pulled this paragraph down to here. The FTX Foundation in re AI extinction risk once exchanged a memo with an associate advocating for the purchase of the Pacific island nation of Nauru according to a 2023 bankruptcy lawsuit. The goal was to construct a bunker slash shelter that would be used for some event where 50% to 99.99% of people die. The surviving effect of altruists were to build a lab for engineering the generation next generation of humans. Again, these are different group. They're different, but highly related groups of people and the fantasies are always the same. It's like, like, you're, because it's, it's patent to say that your apocalyptic fantasy is really a power fantasy. And that's a power fantasy about being inside, about not going outside. Yeah. Oh, that's such a good point. Yeah. It's, it's a power fantasy about like blogging fan fiction from inside was actually a bunker. And girls are allowed, but I'm the boss. Yeah. This, this, I, I have an excerpt from the Harry Potter thing that I just found just skimming it just now in the chapter taboo trade-offs. But basically, Harry's like, no, we, it's more simple and rational to have men be in charge of everything. Like, it's, that's actually like, it's, it makes so much more sense, you know, like, than the alternative, which is just like a complication. You know, it's a needless complication that brings variables in, you know, whereas if men are introverted, everything, it's completely, that's less variables, which, you know, Occam's razor. He's obsessed with Occam's razor. And he doesn't understand what it means really. He's like, like, Occam's razor means that human beings are quantum. Like we, they can get entangled. You know what we got to do? We got to give, I think, I think I bought the perfect shot glass for Eliezer Yudkowski. Yeah. Oh my god. Could you imagine? He would love that thing. You'd love it. You'd be like, this, this was the perfect of receptacle from which to enjoy my rationally derived cocktail of Windex and celery juice. Uh, so like, that's the thing, right? I've invented a boys rule, girls rule from first principles. Uh, all of my first principles thinking is better than yours. By the way, I get to do the angry pirate. My friends and I are paid millions of dollars to all live in a big sleepover house forever together, uh, to compare the lurid what ifs and scary ghost stories. They're all 12 year old boys. They're 12. This is the everybody's 12 thesis. It's just, it's just that's how to understand them. They're 12. They're just socially maladjusted evil 12 year old boys, but that's who they are. AI safety may be as ex-Culton Berkeley. I'm not casting judgment one way or the other, but what it definitely is is a very, very lucrative grift for intimacy, money, whatever status, importance, access being run out of a permanent summer camp for the world's worst fedora men who've ever lived. And the other thing, the last thing I want to come into is that is their defense against all of this, right? If rationality is their sword, the neurodivergence is their shield, which is they say, well, we're neurodivergent because so all of our, uh, ways that you might think of odd, you're actually discriminating against us. Uh, you don't get it because you're actually don't have the autism superpower. But when I think it's, it's like, okay, you may be neurodivergent. And maybe a lot of people are at these events, but they're also assholes. And there are loads of neurodivergent people who are not assholes. Yeah. I mean, even fucking, uh, like Aaron Sorkin under, like in the social network is like, oh, like, uh, yeah, Mark Zuckerberg is neurodivergent, but he's also a piece of shit kind of, you know, he's also like, it doesn't like one thing does not absolve you of the other thing, you know, also a lot of neurodivergent people probably don't have the same view of logic as in Harry, the Harry Potter fanfic, right? Where truth seeking is about like defending a point of view you made up and belittling everybody who doesn't agree with you, rather than truth seeking being like epistemically humble and trying to find out what you don't know, you know, less wrong is ironically often quite wrong. And these things, there's difference between autistic and arrogant. And for them, autism is, yeah, it's a shield against criticism, but like anything, like the things that are actually like, you know, autistic traits, like literal mind and this difficulty reading social cues, intense special interests, a love of plain foods, right? None of that predicts or requires treating everything you think as gospel and anything someone else thinks as, you know, laughable at best. And if they're uncomfortable, it's just a data point about their inferiority relative to you. That is not a neurodivergent trait. That's an asshole trait. Yeah, it's it's like they To think that an inherent part of being neurodivergent is at least for them is like having no empathy for other people and being like, well, you know, we can logic our way into some kind of like form of empathy. And but really what it is is it's just them having no empathy for people and being psychos and being like actually like that they don't they seem to not understand that stuff like, oh, actually like we shouldn't buy mosquito nets. We should spend money on writing papers about AI. It's it's they don't understand that that is a function of them kind of using logic to get around any need to have empathy by pretending that they have it. They're like, we actually our empathy is just a little bit more refined than yours, you know, we see people who don't even exist as being people and those people and they're a quadrillion of them. But coincidentally, that understanding of empathy requires I have like six girlfriends who will do what I say. Yeah, yeah, it's so crazy that that's the case. The stuff with like, I mean, the zisians are like a great like a complete example of this kind of like of someone who is just like a maniac using these ideas and using these like spaces. Specific ideas, by the way, these very specific ideas, specific specifically, literally these ideas like from this guy, you know, like just like taking exactly what he's saying, being obsessed with the Harry Potter book, reading it like it's scripture, which by the way, another thing is like, you know, theology, it to be a theologian, like there's a difference between, you know, the Quran and the Torah and like, dionetics. Okay, like the Quran, the Torah, the Bible, like all that stuff, like there is, you know, all this like that scholarship about like authorship and all of this, like like those books weren't written in like a weekend by a maniac, you know. Whereas this, it's like, oh, I wonder what he meant by this. I wonder if there's any deeper meaning. It's like, no, like it's literally, this is a fan, a Harry Potter fanfic. It's like crazy. It's the same thing as dionetics and like dionetics, it makes no sense. If you've ever tried to read it, it's maybe the worst book of all time. It makes absolutely no sense. You genuinely like on a word to word basis, it's impossible to understand. And like the draw is like, oh, if you go to these guys who, you know, the guy who wrote this, you go to him, he'll teach you how to understand it. And that's like a huge draw. You just give him money and he'll help you understand it. Also, he gets six girlfriends as well. Yeah, that's an absolute point. Well, he, well, Elron, he was less sexually motivated. I think then like, you know, the like David miscavige. Like, you know, he was like just a weird, weird man. He just wanted like servants and he wanted to be the captain of a boat. Like it's kind of like his thing is he was like, I just really want to be an admiral in the name. You know, I just. But like, honestly, just sort of coming, coming to the end here is going a bit long. It's like, it's weird how these guys never update. They update their priors, but they always doing it in conversation with each other about things they're concerned with. Not the really, not the system prompt. He take my meaning where maybe you can believe that you could be wrong or you could be wrong about something you don't think you really could be wrong about or someone else may be right. And they think they don't understand because they don't immediately understand the function for something. Let's be like Doge, right? Doge cuts a whole bunch of grants because they're like, well, that's shrimp running on treadmills. It's meaningless. Like, no, that's super important for aquaculture, which means huge numbers of people. It's like just because you don't immediately understand something doesn't therefore make it deviant or stupid. It just means you don't get it. Doesn't mean they're all wrong. And so another good thing that said maybe they don't understand or claim to not understand having zero boundaries between people's professional advancement, housing and sexual availability is something that was that's a taboo that's good to have, even though you might not like it or agree with it. It's good to have that for a bunch of reasons that even like beyond moral ones, just self preservation related ones that they're all experiencing right now. It would have been good to have those boundaries, even for selfish reasons. And like, it's, it's, it's, it's an internet posting coal organized around an unfaustifiable apocalyptic narrative that successfully captured into institutional power. QAnon became the government through Facebook, talk radio and the Republicans. These guys are trying to become the whole economy through blog posts and sex parties. But I guess what we've also really learned is that confident larping and being, there's just being the loudest person around, it really works if what, if what you're doing is largely what capital and the forces of reaction want. You can really trick yourself into thinking that posting changes the world enough that it actually sort of does a bit sometimes. Yeah, it totally does. Like it's, this is like if the, if the final fantasy house was in charge of the whole economy, basically. Anyway, look, I think we've run another, another long one, but has to thank you very much for coming and filling in today. Always always a pleasure to talk to you. Where can you hear us more? Well, I'm so glad you asked. Riley, you can hear us at on the podcast, Trouble in River City, where we talk about scams, cons, con men, where we're joined by Charles Austin and Dan Beckner. The two lesser members of the podcast on the first half. If we had to be like a, like a dragon costume, like a sort of Chinese dragon costume, we would be the head in the tail and they would be the useless middle middle pit parts. Well, we need the middle or else the dragon has, you know, what's it called. Spina Bifida. Yeah, Spina Bifida. Exactly. I read about that on less wrong. Yeah, yeah, the Spina Bifida of the mind, basically mind Bifida. But you can hear me on that. You can hear me on seeking arrangements. You can hear me on a movie mind set the new season coming soon. And yeah, thank you guys so much for having me. It's always, it's always a pleasure. Well, if only we talked about more fun stuff. So the other thing, as you all know, there is a Patreon. You can subscribe to it. It's five bucks a month. You know the deal. You know the drill. And this is like a lot of basically compelled travel for a few of the TF cast the last few weeks. So thank you for bearing with us. But we'll be back together in a very short while. And things will continue on. Anyway, see you in the bonus. Bye, Ron. Bye. [Music]

Podcast Summary

Key Points:

  1. The AI safety community, centered around figures like Eliezer Yudkowski, is characterized by obsessive speculation about AI risks without technical understanding of computer science.
  2. Their beliefs are rooted in fan fiction, particularly Harry Potter-style narratives, which frame rationality as a superior, magical force that defeats ignorance and stupidity.
  3. The community promotes extreme ideologies, including sexual practices, polyamory, and "rape simulation" parties, often under the guise of effective altruism and AI existential risk.
  4. These groups have influenced major tech companies like OpenAI and Anthropic, where employees engage in doomsday scenario discussions and private ideological communities.
  5. The movement is seen as a blend of irrationalism, financial speculation, and fantasy-driven storytelling, resembling tech industry QAnon or cult-like behavior.
  6. Core members often lack basic technical knowledge of AI or computing, yet present themselves as experts, basing their theories on unverified, emotionally charged narratives.
  7. The community's influence includes shaping policy, funding, and hiring, while simultaneously promoting extreme moral positions that contradict mainstream reality.
  8. The culture thrives on shared identity and exclusion, using logic as a tool to humiliate outsiders and reinforce in-group belonging through absurd hypotheticals and power fantasies.

Summary:

The podcast features a critical examination of the AI safety community, particularly its roots in fan fiction, ideological extremism, and financial speculation. Host Riley and guest Hessa Denny explore how a group centered on figures like Eliezer Yudkowski developed a self-reinforcing culture of irrational belief, using fantasy narratives—especially Harry Potter fan fiction—to frame rationality as a magical, superior force. Despite lacking technical understanding of AI or computer science, members of this community promote extreme doomsday scenarios, such as rogue AI creating paperclips or resurrecting consciousnesses, and advocate for radical lifestyle changes like celibacy, sex parties, or living communally.

Their influence is evident in major AI companies like Anthropic and OpenAI, where employees engage in private doomsday discussions and ideological alignment. The culture is described as a blend of effective altruism, financial speculation, and anti-establishment sentiment, often structured around summer camps and online communities that emphasize shared identity over factual accuracy. The podcast draws parallels to cult behavior, noting how the group uses logical-sounding jargon to mask ignorance, promotes sexualized rituals, and creates a sense of exclusivity through shared belief in existential threats.

Ultimately, the discussion suggests that the AI safety movement is more a self-sustaining fantasy ecosystem than a grounded response to real technological risks, driven by emotional, social, and ideological needs rather than evidence or technical insight.

FAQs

It is a satirical, absurd artifact used to illustrate the humor and cultural absurdity within certain online communities. The item is described as a ridiculous and over-the-top product, symbolizing the 'stupid culture' of a particular fan community.

The podcast claims the AI safety community, including groups like the Machine Intelligence Research Institute and Anthropic, has had significant influence in major AI companies. These individuals are said to have shaped AI development and policy, often advocating for extreme doomsday scenarios and preparing for them in private.

The podcast argues that the rationalist movement, especially through fan fiction like the Harry Potter series, uses storytelling to promote its ideology. It claims these narratives present rationality as a form of magical power, allowing followers to feel superior and validated in their beliefs.

Eliezer Yudkowski is portrayed as a central intellectual figure in the rationalist and AI safety movement. Despite lacking technical knowledge of computers or coding, he has influenced the movement through complex, often incomprehensible theories and writings about AI risks and rationality.

The podcast alleges that the AI safety community promotes extreme social norms, including polyamory, sexual parties, and sexual simulation events, which are presented as part of a broader 'rationalist' lifestyle. These activities are said to be used as a form of bonding and ideological reinforcement.

The podcast claims many members of the AI safety community have little to no understanding of computer science or how AI systems actually work. They base their fears on fictional or exaggerated scenarios rather than real-world technical feasibility.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.