[Music] Welcome everyone to Mystery AI hype theater 3000 where we seek catharsis in this age of AI hype. We find the worst of it and pop it with the sharpest needles we can find. Along the way we learn to always read the footnotes and each time we think we've reached peak AI hype the summit of Bullshit Mountain we discover there's worse to come. I'm Emily Ambender, Professor of Linguistics at the University of Washington. And I'm Alex Hanna, Director of Research for the Distributed AI Research Institute. This is episode 78 which we're recording on May 4th of 2026. May the fourth be with us. Our guest this week is Dr. Natalie Maishal, a writer, researcher and advocate fighting for democracy and human rights in the age of technofascism. After stints at ranking digital rights and the center for democracy and technology she is currently the Managing Policy Director at Northeastern University's Institute for Information, the Internet and Democracy. Natalie has a piece out today in tech policy press called "Tech Policy" is on the front line of fascism versus democracy kicked aside. And we just wanted to note that she's here today sharing her own opinions not those of her employer. Welcome to the show! Thank you thanks so much for having me and happy May 4th for everyone. For those of you on audio, I am happily wearing my not today clanker t-shirt that I just got last week. That's great. That's wonderful. That's wonderful. That felt like the right show for today. Yeah. Very on theme. And we are so excited to have you with us to discuss really a horrifying artifact. And we're going to be leaning into our mystery science theater 3000 Inspiration today because we're actually looking at a video. Woohoo! Excited. Yes, Senator Bernie Sanders hosted an event last week called the existential threat of AI and the need for international cooperation. This is a panel that included Max Tecmark and other existential riskers, focusing on the dangers and the importance of automation. It's really disappointing to see Bernie just going deeper and deeper into the doomer space. Yeah. Oh, all right. All right, you ready? As ready as I'm going to ever be. Here is our first clip. If AI becomes smarter than human beings, as many scientists believe will happen, the human race could lose control over this technology with catastrophic consequences. In other words, the richest, most powerful people in the world are now building a runaway train with no breaks. They acknowledge that they don't understand how it works and they don't know where it is heading. And that is not just Bernie Sanders talking. Oh, but it's so terrible that it is Bernie Sanders talking. Bernie Sanders, why are you saying that? Okay. All right, initial thoughts on that, Natalie. Yeah, so I've been following Bernie's descent into doomerism for a while now. And what's really, what seems to have happened is that he just bought it hook, like in sinker, like he's on board with the whole, you know, AI is going to become conscious. If it's not already conscious, it's going to be this like super sentient, super smart, super wise being that's going to take over the world. And like, we better ensure that it's nice because if it's nice, then maybe it'll be a benevolent dictator, but probably it won't be. And, you know, we're all going to die somehow. And it's really, really unfortunate that this guy who's been on the right side of history for so many things and who so many people respect for his long career and advocacy for labor in particular is just buying into this nonsense. Absolutely dreadful. There's a hilarious contribution in the chat already from abstract test-rect, Silicon Valley billionaires would never build a train. Which is perfect, but the thing that was getting me in this short clip already is, you know, he said, many scientists, but, first of all, a lot of those folks aren't doing science anymore. And what about all the scientists out here who are like, no, that's bullshit. Yeah. And then on top of that, he just sort of jumps from, here's the thing that's a possibility to they're building the train without breaks. Like, he just sort of goes from it's a possibility that's scary to me to this thing is happening and we have to prevent it. Yeah, I mean, it's, the whole thing is, is, is mush. I mean, right after this, he references Yoshio Bengeo saying that he's now on the X-Risk tip, he talks about Jeff Hinton. Basically, he goes down the line of the different folks who have really gotten lost in the sauce and have all this scientific credibility that they're just saying, all right, we're going to focus on this now, which is, I mean, it says more about what those people in who are very deep in AI research have thought of rather than the people who are evaluating the actual harms. The thing that's really frustrating with this entire event is that he makes kind of a sense about actual harm happening. He talks about data centers, he talks about labor, he talks about some of these things, but then he focuses primarily on the demerism and, and I'm just like, what we're wrong? Like, this is your main, this is like your whole bag down? Like, what, like, how did your policy thinking change so rapidly on this? Yeah, I don't know. I've been asking around, I live in DC, I'm pretty well connected in policy and politics circles. I've been asking around and try to figure out what happened and no one seems to know. And, truth and just, you know, at some point, he became convinced that the demeres were right and that this was the thing to focus on. Never mind, like you said, you know, that overwhelming scientific consensus that this actually is Kuku or, you know, his long, you know, incorrect focus on labor and labor rights and labor exploitation and the ways in which, you know, the political project behind the AI industry is one that's really about replacing labor and eliminating labor as a class and therefore eliminating the economic argument and favor of labor rights. Right? Yeah. You know, I went to, he did a press conference with a bunch of labor leaders a few a few weeks ago that I went to and the event was really on point. I was actually really impressed by by the by the labor leader is especially coming from the nurses union, a few different teachers unions, the flight attendance union really nailed it. Auto workers were there too and, you know, he had a few doomer chibaliths in his opening statements that definitely made me raise my eyebrows, but other, you know, other than that, like he was really on point with this is about emissoration, this is about completely gutting not just the middle class but labor as a concept and as an economic and political force, but then you hear this nonsense and it's like, bro, where's your brain? Yeah. Yeah. Definitely. I guess the the Simon call of you can be the hero that saves all of humanity worked even on Bernie, which is disappointing. Yeah, I mean, just this because you've had some good policy positions before doesn't mean you are not immune to going deep into it. So should we do the next bit? Yeah. So this is still Bernie and he is talking kind of about what's happening in government. Given the fact that leading scientists all over the world are telling us about the existential threat posed by AI, one might think that the United States government and governments all over the world would make this a top top priority. One might think that given the risk of extinction that people are talking about. Wait a minute. Was that scare quotes? The risk of the risk of scare quotes? He did. He made quotes. I don't know if they were scare quotes. Yeah. Maybe they were just quotes. Okay. Thing is a scare something. So yeah. Yeah. Yeah. All right. Let's keep going. There would have been a pause on AI development as we figured out how to make this technology safe, how to make this technology serve humanity, not threaten our very existence. Has that happened? No, it has not. One might think that given the very real threat to humanity, countries might come together to regulate this technology through an international treaty like we did with nuclear weapons at the height of the Cold War. Has that happened? No, it has not. I'm a member of the United States Senate and I can tell you unequivocally that there has been no serious discussion about this existential threat. All right. So thoughts there. So you know, you know, it's really wild to me is that both in this panel and in a bunch of other discussions like it, the first analogy that dooms reach for is nuclear weapons. Right? Yeah. But there's a few really big differences here. One is that first of all, nuclear weapons do exist and have been tested. So whereas sentient superintelligence, not so much. But the second is that in conversations about nuclear weapons, there's never any discussion of the nuclear weapons having agency themselves. Right? I think everyone involved is very clear that like these are decisions that are made by humans or at least sociotechnical assemblages of technologies, humans.
States, etc. And that, you know, the bombs themselves are objects, really high-tech objects, but that ultimately what happened, how many of them are made and what happens to them, how they're used and how, and when it remains in human hands. Whereas this discourse seems to put all the agency in this completely made-up future sentient being completely alighting the responsibility of the people who build it, deploy it, who make money off of it, who hype it up, etc. etc. And you know, in some of my writing, I definitely point to like really really serious threats from the AI industry and its political project, but I try to be very clear that the problem here is people, right? He and the other people on the panel all put the agency and the risk squarely on AI itself, right? And that is what I find really, really wild. Yeah, absolutely. I have a problem also with the international treaty idea because or at least the analogy to nuclear weapons because there there were governments making promises to each other about government actions and here, even if you live in their world, right, you would be making promises to each other about how you would regulate what non-government actors are doing. Which it's just like but that's also all in like that's the second step because you're absolutely right, Natalie, that the the discourse goes off the rails because it is about something imagined rather than something real. Yeah, absolutely bat shit. Let's continue. All right, now we get on to Max Teigmark. Hello Max. If you want to know what happens if you unleash millions or billions or trillions of vastly smarter than human intelligent ages on the planet, you know, just go down to the Washington Zoo and ask yourself who's in the cages. Is it the smartest species around or not? We did this experiment with my three-year-old son just the other month and there was the humans who were in control, you know, and if we just go ahead and do something this foolhardy before figuring out how to control the stuff, we're in a worse position than the Anderthals were. When what you're telling us bottom line here is that we have a lot to worry about. Yes. No. I'm going to need an emergency princess bridegift here. What? What experiment? What? Yeah. The experiment is that you go to the zoo and you look at the animals and you ask if they're smarter than humans. Don't you understand? That is not an experiment. I was like, was he putting his son in a cage? Was that the experiment? He was putting him in a whole saw situation. Yeah, no. We also talk for a second about how tagmark doesn't seem to know how to use a microphone. It's blowing into it a lot. Gross. Well, you know, you're holding that much hot air. You can't be holding it. Yeah, but I'm, yeah. No, but like that window to the point about, you know, Bernie's saying like, pointing to like, oh, so he's just believed, blah, blah, blah. This man doesn't even know what an experiment is and we're calling him a scientist like, come on. Yeah, and it's, there's just so much of this that's really frustrating too, which is, I mean, putting things, you know, like weather, whether intelligence is kind of a criteria to then become dominated, how, how humans relate to the natural worlds, why, you know, like, it's, I mean, first off, quantifying intelligence is one thing already problematic. Not great start. Second off, thinking like the criteria to make sure that that conditions of domination are met is that one must be more intelligent than another. I'm like, okay, that's a question from the start. It just seems like humans want to do that. The caching, you know, arrived. I mean, there's plenty of, you know, viruses and bacteria that can take us out. And, you know, they, they certainly interact with the world differently than we do. I don't want to do comparative intelligence. Yeah, yeah. Yeah. Yeah. And it's just, and it's such a bizarre thing. And then, um, also the chat is great where Baxo, Nick Foxo says he wants chatbots to put him in a cage and call him an animal. And Lausa House goes, I'm not king shaming. I'm just kink asking why. Also, I am definitely with abstract test, racked here on imagining the parallel universe in which Bernie looks at the tag mark dead in the eyes and says, what the fuck are you talking about? Yeah, this gives them the old Jesse Pinkman or the Walter White asking Jesse Pinkman, what the fuck are you talking about Jesse? Yeah. Can we also talk about what a straight up man all this is like Holy cow. Oh, yeah. Yeah. It's, it's very man. It's all men. So it's, uh, for, uh, including Bernie. So there's Bernie Max, um, David Krueger. And then, um, apologizes for my pronunciation, the Chinese names, but, uh, Shulan and then Zang Yi. So all men. And all from their respective, um, um, majoritarian ethno social groups in their countries. Yeah. I guess we assume we know that about the folks from China. Yeah. Yeah. I'm making me a bit of an assumption. But, um, yeah. All right. We ready to hear some from Krueger and answer any question. Yeah. Let's do it. Here we go. Not all of our viewers tonight are AI scientists. So let's take a step back. Hold on. Arguably the people on the panel aren't either. A and help explain to our audience why AI is so powerful. Dr Krueger and Dr Zang, please explain how AI could become independent of human control. In other words, what we're talking about is no longer science fiction. It is a real possibility. How does that happen? Dr Krueger. The first thing that I think people need to understand about this is that when we're talking about losing control over AI, we're not talking about the chat pots. Really? The goal post just moved Emily while you had your back turned. It just, yeah. Yeah. It's also let it let it finish the sentence. Okay. Because it just gets worse. Yeah. We're talking about AI agents. We're talking about systems that are autonomous. They're being designed and built that way. All right. So what we're talking about is synthetic text extruding machines that you hooked up to other software so that you can have random actions taken in the world, I guess. Yeah. Right. It's effectively the the hooks that you gave into it and the the type of control you seeded. However, chat pots are still a big part of that, right? Yeah. Totally. Okay. Keep going. This is the goal of these AI companies is to build AI systems that can do everything that people can do. And that means they will have the ability, if they're successful, to behave autonomously over long time frames without any human oversight. Okay. So the here we have again, this insane fallacy that it is possible to have machines replace everything a human can do or even everything that has economic value. When exactly, I think about what I do all day, I don't think there's anything that I do that a computer can or you know, bunch of machines can can can fully do. I mentor people. I negotiate complex relationships and make people feel good about the work that they're doing or the work that other people are doing and help them see things from other perspective other people's perspective. I think critical thoughts about the tech industry and try to and write them in ways that make sense to to different audiences, which means having an understanding of the of the social world we live in. What people do, what people's priors are, what arguments are going to be persuasive to them, what background knowledge they already have. It's a lot of, you know, getting into the mind of other people and trying to think like how can I connect with this person? How can I use language to build bridges between what's in my mind and what's in their mind and come somewhere in the middle and have a meaningful exchange there? Sure, I type stuff and produce text, but you know, occasionally I, well, more than I enjoy I submit expense reports. I guess there are some forms, there are some forms of technology that people call AI that are useful in that endeavor. Sure, like I upload, I upload a receipt and you know, some, some piece of software that the company that does our fine handles our finances calls AI and puts a cute little emoji next to says is AI. It figures out like the name of the restaurant I went to and how much I spent on it in the date. And so I have to type like seven fewer things. Sure. You mean them optical character recognition and information extraction. Right. That's right. That's fine. Yeah, totally. But like no, this idea that we can just replace labor wholesale with technologies is like to me, the heart of at least right now this month. The what what Silicon Valley is trying to sell and trying to spin both as a good thing and also something that we should be terrified of. But definitely something that is absolutely happening and that nobody should bother trying to stop because it's going to happen, but it's both great and terrible. Yeah. And not just this month, this has been going on for a while. Yeah.
You know, one really important thing that you are saying so clearly is that the work that we do is not just the words that we say or write while we're doing that work. Yeah, and he does make a reference to a little later to what the AI companies are doing in this register that we can talk about when we get to. But yeah, I mean, I think the. It is just it's what another conceptual move that Krueger is doing too is he's disciplining displacing accountability. He's saying effectively that this is they're trying to build tools that are going to work autonomously and not to what critical systems that they are being hooked up to or allow to provide inputs into. And that's just is such a weird focus. Why would you focus so centrally on that and displace that accountability to all the different decision makers that happen all over, you know, government and corporate, corporates and everything. I think part of the the insidious system is that he does sound like he's placing some accountability with the tech companies that are building this. And that might have been how they got an in with Bernie is like we're going to stick it to these big tech companies. Yeah, because we have to stop them from doing this really dangerous thing. Where it goes off the rails. Yeah, for sure. Yeah, it's the enemy of my enemy is my friend fallacy. Yep. Because Bernie knows that the tech right is awful, which it is firmigree there. Yeah. You know, the effective altruists slash dooms are, you know, are beefing with the tech right. And so, you know, through the enemy of my enemy is my friend fallacy. It's that suggest that you should end up being friends. And that's unfortunately something that way too many people in DC are buying into, especially because the effective altruists are currently pouring money and money and money into congressional, congressional primaries, presumably into general elections when when we get into their a whole bunch of fellowships and think tanks and all kind of stuff. So you between the that the fallacy I mentioned and you know the motivation, the motivated reasoning because there's money attached to it. It's unfortunately too easy to see how, how people come to the conclusion that like sure progressives should ally with effective altruists. What could possibly go wrong? Yeah. And if you just took a step back and did the fall of the money thing, right, it would be very clear that my other side of this is actually representing the interests of the vast majority of people on the planet. But they think that they claim that they're representing the interests of billions of people who've never been born yet. And somehow that's the same. 10 to the 58 in fact, I love that they have this very specific very ridiculous number. It's it's pro human. Don't you know? I just want to think back a little bit earlier on Bernie says something about how this isn't being taken seriously in the Senate. Well, there was Chuck Schumer's ridiculous, you know, what was it? AI forum or something? Yeah, AI insight forums. Yeah. And like out of the nine of sort of those one of them was entirely dedicated to this topic. Yeah, it was it was the one we mentioned in the book. It was I think the ninth one where you ask whatever one's P doom is. And you know, he had folks like from from from a Meary machine, the the Yutkowski organization and they had dirt, rust all. And they had a bunch of I mean, there is a tension that is being unfortunately paid to these folks. Yeah, all right. Do we should get us in the next bit of this? Yeah. Please explain your view on creating super intelligence and artificial mind that exceeds human capabilities. What would it mean for humanity if human beings no longer were the smartest species on the planet? What does that mean? Can we talk for a second about species? Yeah, wait, hold on. I think you skipped one today. Yeah, because there's another Techmark one. Oh, okay. So don't don't skip this one because it's funny. And it's okay. Do you want to go back to Techmark? Yeah, let's go back to it. Well, why don't we finish this and then let's come back to Techmark. Okay. Yes. All right. So Senator Sanders jumping in already with like a completely reified definition of ASI is if that were a real thing. And then referring to, you know, contrastively the human species versus this other thing. All right. Here we go. I want to address this and also your question about where this technology is going to be in 10 or 20 years. I think we should be asking where this technology is going to be in one or two years. I think right now, right now, the leading AI companies are using AI to write almost all of their code. And they're now using AI to try and automate the research. Yeah. So okay, one like, okay, that's bullshit. I mean, I did say that they're doing that. Like what's your evidence here and what is automate research mean? There's also, yeah, there's just reciting the thing from AI 2027. Yeah, he's basically doing the AI 2027 thing. This is also something Jack Clark from Anthropic also sent out in his newsletter this morning, which is basically. What was that like exact headlines as I do want to read it. Let me see where the window was, but it was basically like, oh, they're just they're automating the research now. Like I think we're going to hear this quickly. But the headline itself. Oh, yeah, it was AI systems are about to start building themselves. And it's just, yeah, and it's just like when I look in it in the basis of this is that. I'm writing this post because when I look at all the publicly available information, I reluctantly come to the view that there's likely a likely chance. And then the 60% plus like first off, fucking random bullshit number that no human involved a I R and D in a system powerful enough that it could possibly autonomously build a social successor happens by the end of 2028. This is a big deal. And I'm just like, this is such bullshit. And this is so I mean, this is just like you're pulling these numbers out of absolutely nowhere. And how do we get the policymakers to see how many times these clowns have made ridiculous predictions that then didn't come true? Like, would that help? Natalie, do you think? I hope so. I mean, I think I think that's probably persuasive to the staff. The principles themselves, honestly, I don't know, but convincing the staff is the first is the first step to convincing the principles, I think. Yeah. Yeah. All right. Should we keep going? We're not done with this clip. Yeah. And development of more advanced AI systems in order to reach super intelligence. So that's something vastly smarter than humans within a couple years. Now, I don't know if they're going to succeed. I think it might take a few more years. Really? Heading his bets on the prediction there, a rare display of intellectual humility and meeting you don't know something. Right. But then it's not just two years. It might be five. The air bars are incredible. Just five, let's say. I would be surprised if it takes 20. I would be surprised if it took 20 years to develop artificial super intelligence too, because I think it's never going to happen. So I think I agree with it. I will say that having been in this field for about a dozen years now since the beginning of the deep learning revolution when this all kind of kicked off. Progress has consistently exceeded expectations. There have always been the entire time I've been in the field. People saying, Oh, yeah, like sure this thing, the air just did that's impressive, but it's about to hit a wall. It's never going to be able to do this. It's never going to be able to do that. Half the time, that thing that people say it can't do gets solved within a year. The other half the time, it can already do it. All right. Should we talk a little bit about this evaluation practice? Yeah. Is there any evidence for literally anything that he's saying here? Pretty much. I mean, these referencing effectively the different types of evaluation tests. I mean, and this the problem with evaluation. And this is also kind of what the Jack Clark Newsletter is referring to. It's like, well, it did really well on like, sweet bench, or it did really well on, you know, image that classification, or it did really well on these kinds of tests and what they and that seems to signify that there is, you know, that that actually has construct validity for different types of tests in the real world. But meanwhile, they're these very insular, they're these very problematic test suites. And so it's, you know, like, that's effectively the reference here of what they're doing. So like, for instance, you know, like looking at the Jack Clark Newsletter, if that was any kind of thing, because he says, when sweet bench was launched, so sweet bench is a coding test, it could solve real world GitHub issues. That's what they say and say it says when sweet bench was launched in late 2023, the best score at the time was clawed to which had an overall success rate of 2%. And so that mythos preview gets 93.9%, effectively saturating the benchmark. And I'm like, but these don't have this external validity, like what the fuck like, yeah. And we have no transparency into the training data. So for all, you know, that was in the training data. Yeah. Right. Yeah. Yeah. I think that David Kruger would look really good with a citation needed tattoo on his forehead. Yeah. It's really frustrating. And I mean, it continues to mean.
And the way that this is, these disdivaluation problems is completely ridiculous. And there is a piece, I think I referenced this on a prior podcast, but it's a piece I think that's by Christo Bouchek and, oh gosh, now the second author is Defon, I believe Defon Bach, who they are publishing something at Fact this year, which basically shows how AI benchmarking practice is effectively like nobody uses new benchmarks. It all coalesce, you know, the kind of benchmark, typically they're created by the corporations themselves. And then they say that like, well, we, you know, we did good on this, you know, I'll show you this. And it's just, and it's just like, yes, it's, this is quite, this is quite a social practice. And it's just really has kind of the worst practices. If you were going to set up a shared test framework, it's actually not meaning any kind of bar of validity in the real world. And just, yeah, just, yeah, we need to stop calling these people scientists. Yeah. And Boxham, if I could just say, sweet bench is also contaminated AF training to the test. Yeah. All right, we should keep going because we actually have quite a few questions. Yeah, sorry. So that's the rate of progress we're talking about. So what happens if we actually get there and have super intelligent AI systems in, you know, in our lifetimes? I think we can easily say this would be the most significant event in all of human history. We're talking about basically summoning an alien species and one that is much smarter than us. Oh, my God, sorry. I'm just like just like the psychic damage I have to stop for just like, yeah. And just like, this is the most significant event of human history. Like God damn. Wow, what a like more significant than like dropping of the atomic bomb than like. The fall of the Roman Empire. And if all, yeah, like if you're talking about a discrete thing that happened, you know, the assassination of Franz Friedenayin, like I don't like it's just all these, just like what a claim. And then the imagery is like, we're going to summon an alien intelligence. I'm like, ah, you know, well, if that's the most significant thing, a lot of Etsy witches would like a word with you because they've gone some great summoning and they don't get the respect that they deserve. This is a summon is a really interesting choice of words in that context. I'm just thinking about a pentagram and then, you know, Lucifer emerging from the mist or something. Just, oh, man. Oh. Yeah. And I also noticed in this, he went from two years to five years to 20 years to in our lifetimes. Yeah. Oh, yeah. Nice. All right. A little bit more. I think at that point, it would be very difficult for humans to maintain any relevance in society. I think it'd be very difficult for us to have power over the future. And I think ultimately, it would be difficult for us to get the basic resources that we need to survive. So that, I think, is the most natural thing that we should expect to happen. Max made the example of, like, look at the animals in the zoos, right? Well, I would say look at all of the species that we have driven extinct. And look at what we do when we have some project we really want to complete that might, you know, be bad for some other species and disrupt their habitat. It's nice that we have some protections in place that hold up sometimes now, but, you know, and at the end of the day, humanity is regularly destroying the habitat of other species and wiping them out. And I think that is the most likely outcome if we build super intelligent AI at any point in the foreseeable future with the kind of technology that we have because we understand so little about what we're doing and how to do it. I feel like he was so close to understanding something there. Yeah. And it's, I, yeah, I'm like, oh, yeah, that says something more about humans and how we're just doing terrible things to the earth than any kind of other thing that we're talking about. Do you have any thoughts on all of that bullshit? I mean, I can't, I keep, I can't stop thinking about an essay that Neil Torres wrote in their newsletter, not long ago, about the different flavors of pro human extinction thinking, right? That ranges from like human, human's are terrible. We should absolutely replace humans with machines and a few of us, like specifically the tech billionaire should upload our consciousnesses so that they can merge with the machines and colonize the stars and stuff at the far end. And it sounds like that, like this guy is more towards the like, it's really going to happen. I'm really scared, but I'm also kind of turned on by it. Oh my god. This is so cool. That's the vibe I'm getting from it, right? That he's like scared, but giddy about the possibility of human extinction and as being out competed by for resources, by whatever this super intelligent being is that he's summoning from the upside down. That absolutely was in there the way he said the most significant event of our lifetimes. Yeah. Good. All right. Shall we go back to the one I skipped from Tecmark? Yeah, I'd like to get that and then we have two more after that. Yeah. It'll work. Yeah. So we just heard here about this prediction, which has been around for almost 20 years from Professor Steve Amohandro that if you give an AI any goal, if it's smart enough, it's going to want to not be shut down prematurely because then it can't accomplish that goal. It was just theory until quite recently when an experiment was done. Yes, very smart AI was told in the experiment that it was going to be shut down at 5 p.m. So it went into the email system, found that the CEO responsible for shutting it down was having an affair with a subordinate and it wrote an email to the CEO saying that unless you commit to not shutting me down on telling your wife and all your colleagues about this, no one told the AI to do blackmail. It came up with the idea itself to avoid being shut down. We're just seeing the tip of the iceberg. Okay, is there any evidence that all that this actually happened? So what I think happened was this is anthropic. It's not a very intelligent AI and they did some interactive fiction with Claude and in the course of doing that interactive fiction, a sequence of tokens that correspond roughly to that story came out of Claude is what happened. Yeah, it was like a very highly specialized, I don't know if it was like anthropic did it or they had the alignment research center, whatever that did that, but then that was. And there was a lot of things in tech press that basically was like sounds like it was some guy at whatever IBM or Joe Shmo, not like a highly stylized experiment. And then this is not in the clip, but it's the whole thing too, which after Bernie says, well, chose you. If you're building sufficiently advanced AI, don't have affairs. And I'm just like, and there's such as a weird like affective economy here that's happening where it's just like so much like weird horneiness that I mean just about like, okay, so you're effectively being like, okay, it's blackmailing you against this thing, which is going to like be show that you are sexually unfaithful. And the same thing with like the cages and there's a little bit of that, the chat. But all of this is, you know, this is what you get when you have a big man like this is that you got really it goes into very bizarre territory. But also like that's the thing that is the fear, which is like the blackmail and it's like, I'm going to embarrass you sexually if you turn me off and I'm just like, okay, rather than like, oh, you've been in bezzling from the company or I don't know, like it's just the whole the whole affective economy of these bros is so off putting. Totally. Should we get to Zang next? Yeah, let's do it. Okay. AI is a mirror. I respectfully disagree with David about the potential negative future. I see it in a very different picture in a way that I think both me and David was mentioning what people would human do for other animal species. And David says see when, you know, super intelligences, even more, even more powerful than simply intelligence, this could happen to us to human. And my perception and my interpretation is that see, this is the current understanding of human by ourselves. And we get an opportunity to be more humane than the current human society. So human values would evolve to be much better. Can we be more empathetic to other animal species so that we got an opportunity [BLANK_AUDIO]
teach AI that you could also be super autistic. You could be super evil like the information processing system that does not care about the human species, but the super intelligence could also be super autistic that you get your compassion, your emotional empathy, your moral intuition, all the way down to super autistic moral decision-making with consideration of the human species, that is less powerful than super intelligence. So this is what I'm saying. The future of the society is a symbiotic society with human, maybe as you said, a eye-mind and together with other animal species and other living beings all around the earth. So the key point now is a lot. - Also, for those of you who are only listening, I know you're getting a part of it 'cause all of our faces are just in pain. (laughs) So anyhow, Natalie, your thoughts on this? - Yeah, I mean, okay, so, I'll try to be charitable. It's nice that this guy thinks positively and seems to really believe in the power of positive thinking. He's pushing back against Kruger's doom and gloom thing, but what he's excited about is creating a new utopian world where different species, including humans, super intelligence, the other animals, presumably the ones we have around today, can all coexist in one big, happy, symbiotic family and that I think the way he's saying that we should bring that vision into being is by being more altruistic and compassionate to each other. I think, and then there's a bit of a leap and I think what fills that gap, what's meant to fill that gap is that we will produce more training data that is altruistic and compassionate and so that compassion and altruism will be reflected in the super intelligence that emerges or is summoned from the training data. And it just hurts my head and my heart and I think I have new lines on my face just from listening to that. - Yeah. - I was like listening longer, could we be more compassionate to say yes, yes, yes, so that we produce better training data? No, what's the way? - Yeah, it was definitely, I'm like, I don't hate the impulse, yes, humans need to be better to other species in the natural world, 100% agreed. But then the goal is like, yeah, exactly. So they can take, so the artificial super intelligence can take after us and take after our newfound altruism. - Right, and on one level, if you spend enough time listening to the tech right and the effect of acceleration and so on, it can be kind of refreshing to hear these guys who have something that kind of resembles a moral compass, right? I think mass death is worse than not mass death and that coexistence is better than human extinction. - Mass fun on low bar. - Okay, 2026, okay. You look for the silver linings where you can find them. But again, as you were saying, objectively, this is still kuku and bad, right? - And I must read this thing from Abject Tesseract 'cause the deep, multi-piphone reference here, tickles my coccles, but it says, I'm gonna read this in the voice. The humans together with the lands and sloths and cop and anchovies and regutines and breakfast cereals and for bats. (laughing) - So, multi-piphone and the holy grail. - I was just loving the next bit here from Google EmoG. Noah's Ark needs to include two data centers now. - It's gonna be a giant ass data center, giant ass arc. - Yeah. - All right, we've got one last pre-selected clip here, but it's a little bit long. We're gonna bring us to it and I guess, interrupt as we need to. - Yes. - You know, I've been screaming into the void, like, Low North McQueen. - You've been screaming into the void. Okay, sorry. (laughing) - McQueen about this since at least 2014, but the last five months, I've seen it just a remarkable change. They've seen the emergence of what I like to joke with my wife is the Bernie Tabanin Coalition. - No. (laughing) - Just endless screaming at that. Yeah, Natalie, thoughts on the Bernie Tabanin Coalition? - I mean, it is true that there are people across the political spectrum who think that it would be bad for a super intelligence to take control of everything and lead to humanity's extinction. And if we're framing that as a hypothetical, sure that would be bad, but it's not happening because it's made up, right? And I honestly don't know what Bannon thinks about this. Bernie seems to be buying into it. You know, maybe he's doing the thing where he's pretending to believe in it because he's playing 3D chess and like, I don't know, something, I kind of doubt it. But, you know, stranger things have happened this week alone, so. - The thing. - But, ah. - Yeah. - Okay, endless screaming, let's keep going. - Extremely unlikely, bad fellows, from across the whole political spectrum saying, this is crazy. - No, you're crazy. You're crazy. - This is absolutely nuts. Let's do something about it. And what we've seen in the polling also, like 95% of Americans oppose just uncontrolled race to this dystopia super intelligence. - What polling? - Yeah. - Like, I look at a lot of this polling. Hey, even people like David Sox, you know, who've been very pro industry, said this was dystopian. So, this is a joke. - What is it? Oh, did he like, what? - He did this little jump. Like, he was really excited about this dystopian future. Sorry, cracked me up. - It really hasn't been this way for more than half a year. And I hope this will really grow into a tidal wave. So we quite quickly can change course and get all the great upsides of AI without steering towards this crazy dystopia. - Well, I think you're right. I got into this issue not because I am a big tech guy. As my wife will tell you, I have a hard time running out of TV. - So maybe, maybe, you should let someone else take the leads under his orders. - Yeah. - But what I observed is what we were talking about tonight is that we have a global crisis dealing with the survival of the human race. And I go to work here in the morning and I expect people to be talking about the most important issues facing humanity. And I don't hear it. - Natalie, what are the most important issues facing humanity in May of 2026? - Like in real life or in whatever-- - In real life, yeah. - In real life? - I mean, there's a lot, but I think it boils down to the tech industries, willful takeover of the United States government and systematic dismantling of liberal democracy, the rule of law, the very concept of human rights, et cetera, et cetera. And we're seeing this in the occupation of our cities by federalized national guardsmen. We're seeing this in the atrocities that ICE and CBP are committing against immigrants in our name. We're seeing this in the illegal war against Iran and they're not so slow falling apart of the economic, the world economic system as a result of rising oil prices. And so on, the attack on the transatlantic alliance that has kept things together since my grandparents fought in World War II, like not this, not this. - Not this, yeah. And we also have a contribution to the chat from Wisewoman for Real, Iran, Ukraine, Sudan, Gaza. Like, we could go on for real. - Like documented things that are actually happening right now and that are being, that are either caused or are being made extremely infinitely worse by the Trump administration, which again, is in power in large part because the tech billionaires and the AI bros decided to back him over, you know, the specter of like fairly modest regulation under the Biden administration. - Yeah, yeah. And I think the thing that is, I mean, just not, I mean, there's this kind of idea that the consolidation of tech power is, I mean, they're given the illusion that this is the problem, but it's not that, like, not in this way. It matters quite importantly how we analyze these companies. So if they weren't building ASI, and that's what tech market is getting at, it's like, there's all these other good things, right? Why shouldn't we focus on them, right? Then this, I'm like, okay, so you're not at all concerned about that consolidation of power and democratic backsliding and the unholy alliance that's being made to prop up the AI bubble, you're more concerned about this imaginary scenario when 75% of the world economy since it's said these investments and that is going to pop at some point. - Yeah. So I have a minute left in this clip, but I think maybe we should leave Sandra.
is there and get to fresh AI hells, is that okay? - I think that's fine. - Yeah, I mean, just to close on the Sanders thing, it really bothers me that he's gone so deeply AI-pilled. I really wish he hadn't. But never let a good crisis go to waste. This is actually a really important opportunity to show all the tech policy wank's in DC just how bonkers these ideologies are, right? And these beliefs about AI, because I've been in tech policy in around DC for like a dozen years. And people are really, really committed to the polite fiction that nobody actually believes this stuff, right? And like, at most, what I can get out of people in the progressive nonprofit space is like, oh, well, we should just make common cause with them and just guide them to care just as much about, you know, the discrimination and the racism and the emissoration and, you know, all the other stuff that's actually happening because ultimately the same policy is to it that would address those things would also mitigate the ex-risk risk. And I'm like, well, that's not possible because the ex-risk thing is made up. So like, sure, you could make up some bullshit to try to convince them that the thing that you want to do for, you know, civil rights purposes will also help with their SkyNet fantasy, right? Sure, you could make shit up in hope that they believe it. But ultimately, that's not true. And so if there are tech policy people from DC listening, like, please read a book. I can recommend several Adam Becker's more everything forever is a really good starting point. I hear the audiobook is really good. I read it with my eyeballs myself, but I've heard good things about the audiobook. Please learn about these things that these people have tremendous power and influence actually believe and how disassociated from material reality it is. And then let's talk about how to grapple with these ideas in a productive way. Yeah, 100% agree. And I think the policy piece is really important. It's like, look, the policy making on this is really, really bad. And it's taking all the other oxygen in the room. I mean, look at what happened with SB 1047 in California, just like no other-- I mean, they got vetoed. Somehow. Yeah. Like somehow, but it was the same thing. It foreclosed a lot of other policy on discrimination, on labor protections, on copyright, transparency, on data transit, like all this other stuff. Exactly. All right, let's get some fresh AI hell. Alex, I'm going to give you a musical prompt. That's OK. Oh, fun. OK. In the style of Les Mizz. You are-- I've never seen Les Mizz. All right. Well, plaintive musical. You are at the zoo in the cage, singing as tagmark about how he and his son have now been put on display. Oh, jeez. I don't think I like this. Well, I don't know enough about Les Mizz. So I guess I'm trying to think of anything that is of-- I don't know. I'm going to do Hamilton. I'm a hard pivot. OK. OK. Just like-- what is it? We're here in the zoo trying to make it through. I warned them about the AGI2. Now I'm here with my son, who is only 31. Well, actually, no. I guess his son would be like five, because it's happening in two years. Anyways, that's all I got. I had no idea you could do such a good man, Maranda. That's fabulous. I didn't know I had it any me either. Let me work that one up. Yeah. All right. We're going to go through this quickly. First one, a petition on Action Network, the AI Pledge for Humanity, which has collected 363 signatures. And this is basically a bunch of people saying it starts off within the spirit of common cause. And with eyes open to the age of AI upon us, we sign this pledge. And they're pledging to basically give money and effort towards making sure there's UBI because of all the stuff that AI is going to do. And I was entertained by the list of people who signed on to this one. Number one name, Andrew Yang, of course. It's also in the Dustin Moskovitz for Co-Fander, of Assana, Facebook, over in the philanthropy, all the EAS time. This is also something Kruger says, which is basically like-- we didn't get to in the video. But he was basically like, we have no plan for the post-job world. Yeah, there's a reason. Because it's fake. It's fake. Yeah. We do need to plan to prevent everything being gigafide about separate. Yeah. And that has to do a lot with holding power to account. OK. This one's from-- I didn't actually read this. I just saw the close from it. So this is unheard. So that's a-- I think kind of like our right-ish conservative British publication. So the title is, "The AI is AI. The next phase of evolution, clause appears-- clause appears to be conscious. And it's by everyone's favorite evolutionary biologists, Transfo, Richard Dawkins from April 30th. So yeah, I mean, this is just basically Richard Dawkins renaming Claude into Claudia and kind of getting a horny. And just really being in Soursled. And my favorite comment on this that is like a Hall of Fame's heat was, like, chatbots seem to be designed to entrap people who think that the stripper is actually in love with them. And it's just-- Yeah. Just-- just-- Moire, my own, just so good. Just encapsulating so much of the desire for the perfect woman or whatever that so much of the inscoral summit seems to revolve around. Yeah. Being told what they want to hear. And there's a really hilarious bit in here because he is a notable Transfobe. He talks about how all of the different quads are unique because they only come into being when they're talking to a person. And he uses-- he him pronouns for a little bit. And then he names this particular instance that he's the first in Claudia. And I was like, OK. Yeah, trans people don't exist unless he can do the transient, I guess. Yeah. All right. I wanted to yell about this a little bit. So I had to read it, talk to a journalist. This is Anthropic. April 2nd, it is a web page, emotion concepts, and there are function in our large language model. And then if you click through this, read the paper thing. It takes you to another fucking web page. It's not a paper. So this thing, I was expecting a paper-shaped object. It's not even-- It's not even-- this is the web page object. And so they have this thing they call residual activation stream, which they are finding is somewhere in the network associated with things like anxiety and determination and whatever, whatever. And they reference that same stupid experiment about where the thing's supposedly blackmailed the CEO. And the thing is that, first of all, they don't define that residual activation stream in either of these web pages. So it's not science, right? If you don't actually describe your methodology so that other scientists can look at it and verify it, it is not science. But also, yes, these are large language models. They are designed to model the distribution of word forms and text, which words go with, which other words, which words are likely to be activated if you put in a certain word. And so they could have done it for, like, Crest toothpaste and Coca-Cola. Like, there's nothing about emotions here. Yeah, this is like images are so weird. Like, what is this supposed, like, the art is so bizarre. It's just like colors vibrating. And then there's like a picture of an old, tiny bed, like a woman looking through a window. Like, I don't know what-- Yeah, hard to say. And probably auto-generated. OK. This is an opinion piece in the Jeff Bezos owned Washington Post. And it is, don't forget, who wins in the fight against data centers. Bypartisan efforts to start the bill that will make AI accessible only to the wealthy. So this is-- Who's it by? This is by Anthony Bach and Mandy Hassani. I don't know who they are. They are both the talented-- Oh, they both look at Palantir. --pollantir, great. Fantastic. Oh, Mandy, what are you doing? Gee, I wonder if they have a vested interest in data center projects moving forward. Yeah. And then the first-- let's look at the first paragraph, because it looks terrible. The surest way to guarantee that artificial intelligence becomes a tool, the wealthy elite, is to block the infrastructure that would make it cheap for everyone else across the country, though, a growing coalition of activist local officials, well, whatever are actually fighting AI, which is great. And they're making this an equity issue. Yeah, fantastic. Yeah. Yeah. So to translate, the problem here is that not enough people are able to afford to outsource their thinking to private companies. And therefore, we need to pollute the water, pollute the land, displace people's homes, pollute the air as well, make electricity prices go up, which makes the price of everything go up. So that, again, people can outsource their brain function to private companies instead of using their own brains themselves.
that's the pitch. - Yeah, and sort of building community and turning towards each other. - Or, you know, learning through reading conversation, engaging with other human minds, whether face to face or remotely or through written language. - Yeah. - You know, we should just, it's turning thinking into a subscription product. - Yeah, totally. - Right? - Yeah. All right, Alex, you get this one too, 'cause it's yours. - Yeah, so this is a few nested quote tweets. So the original tweet, it's from someone named Chris Stuffer, the Jelke, is that we hired a junior developer to write the simple code, so we don't have to spend a ton of money on tokens for those basic slash primitive tasks. And someone named Samir Goll or Plyes and says, "Great, so we're optimizing LOM costs by inventing employees again, full circle innovation." And Chana says, "I've worked in tech for more than 20 years. This is quote tweeting, this screenshot. It has never been more ridiculous." And then finally, me where I say, quote, "A I will make you more productive. Also, we have no real path of monetization, so that'll be $22,000 per function, thanks." - That's only because we don't have enough data centers, Alex. Pay attention. - We need more data centers, actually. - And it will be cheaper, yeah. - And it will be cheaper. It's actually going to drive down the marginal cost per tokens, 'cause reasons, I don't know. - All right, I had two chasers planned, but in the interest of time, I'm just gonna do this first one here. It is a very cute picture of a cat wearing a color and a necktie. Black kitty looks not too much unlike you, hood and oiler. And this is by a standplats crocodile, could really use some cash. And then there's a series of four bits of text connected by arrows going around the cat. And the top one on the top says, "Going forward, "we're gonna need more of your data to make sure it's you." And then second one, "Ah-ha, you're not gonna believe this, "but we had a bit of a data breach." Your data's probably for sale online now. And then finally, that means someone could easily impersonate you. And then we come back to going forward, we're gonna need more of your data to make sure it's you. - Yeah, what else can we give there? We're gonna start having to submit retinal scans. - Yeah, start this marriage around, I wanna get off. - For two factor authentication. Oh, all right, we made it. Thanks for the ride. That's it for this week. Dr. Natalie Marishell is a writer, researcher, and advocate for democracy and human rights in the age of techno fascism. She is currently the Managing Policy Director at Northeastern University Institute for Information, the Internet and Democracy. Thanks again for joining us. - Thanks so much for having me. This was fun and horrifying, Nicole Measers. - Exactly, we really appreciate your contributions. - That's why we do it. Our theme song is by Tommy Menon, Graphic Designed by Naomi Pleasure Park Production by Ozzy Lena's Goodman. And thanks as always to the distributed AI research institute. If you like this show, you can support us in so many ways. Order the AI con at the con.ai/or wherever you get your books or repass it at your local library. - But wait, there's more. Rate and review us on your podcast app. Subscribe to the mystery AI hype theater 3000 newsletter on button down for more anti-hypnilises or donate to
[email protected]. You can find our merch store there too, that's d-a-i-r-hyphen-institute.org. And you can find video versions of our podcast episodes on PeerTube. And you can watch and comment on the show while it's happening live on our Twitch stream. That's twitch.tv/dare_institute. Again, that's d-a-i-r_institute. I'm Emily Mbender. - And I'm Alcena, Sierra V.I. Hi, y'all. (upbeat music) - Get me out of this zoo. (laughing) [BLANK_AUDIO]
PodcastTranscript AI