Could AI Really Kill Us All?

is AI really about to kill us all? Hi, I'm Wendy Zuckerman and this is Science Versus, the show that pits facts against the fallout from AI. We've been hearing for years that AI poses an existential threat to humanity, and this is often coming from the very people who run the biggest AI companies in the world. I think AI will probably most likely sort of lead to the end of the world. My chance that something goes, you know, really quite catastrophically wrong on this scale of, you know, human civilization, you know, it might be somewhere between 10 and 25 percent. Lock my words, AI is far more dangerous than Diggs. Just recently, a researcher named Jacob Coxen, who resigned from anthropic, said that some of the people working on this tech really think that it could kill us all in the next 10 years. This journalist asked him, "You truly believe that AI could kill us all in less than a decade?" Yes, I do, and it's not just me, many people working in the industry, including people, executives, senior researchers, all genuinely believe that there is a substantial probability that this technology could kill everyone. And one of the big events that everyone is pointing to to say, "Look how scary this tech is," is an incident that happened at OpenAI, where their AI model broke into the infrastructure of another tech company called Huggingface. Two of its advanced models escaped their testing environment, known as a sandbox, and hacked into another AI company's internal systems. Tech companies andthropic and meta have since come out basically saying, "Ustoo! Ustoo! We've caught our AI models hacking into other companies as well." With this mad news coming from AI, it has understandably gotten a lot of people freaked out, saying it is time to slow all of this down. It really cannot be overstated how crazy this is. This is like out of a Hollywood movie. Pause AI development. It is not too late to avoid disaster. Stop building machines that humans cannot control. But meanwhile, there is another group of people out there who are saying, "Hold on a minute. All these fees about the AI apocalypse." What a load of baloney. This is just a distraction or even weirder, some marketing ploy by these AI companies to make their tech seem super powerful. So today on the show, how worried do we need to be here? Are we really on the brink of an AI apocalypse? Or is this all just a bit overblown? I mean, are we really just freaking out about the same robots that had to be dragged off stage in a dance competition? When it comes to the AI apocalypse, there's a lot of people saying, "I think AI will probably most likely sort of lead to the end of the world." But then, there's science. If you have a friend who is freaking out about the AI apocalypse, and I know you do because everyone is right now, send them this episode, give them some real science to hold on to. Science versus the AI apocalypse is coming up. Welcome back. Today, we are finding out how worried we need to be about the AI apocalypse. And to tell us all about it is my AI agent, Merrill Horn. Hi, Merrill. Hey, Wendy. I'm joking. I'm joking. Yeah, I have to say this. There's no fun anymore. I don't have an AI agent. I'm real. I want to start with what happened at hugging face. Because I know this was a little while ago now. Things move fast in the world of AI. But it's really the one event that everyone keeps coming back to, to say, look how scary this tech has gotten. These companies can't control their technology. So many headlines. It's gone broke. AI's gone broke. So I really want to go through what happened. And was it as scary as basically the headlines are saying? Yeah, let's look at what actually happened there. Because the details are really wild and kind of creepy. Oh, man. Okay, let's go. Yeah, so he talks about what happened here with David Perry. He's a professor at Murdock University in Australia and specializes in cybersecurity. And he said that the thing that set this whole incident off was that OpenAI had given some of its models a little assignment. What the OpenAI team were doing was trying out different ones of their models to see what they could do, sort of like a Houdini sort of thing. You know, can you get out of this straight jacket with chains around it and all that sort of thing? Like a little puzzle for it to work on. Absolutely. So the researchers kind of put the AI through almost like an obstacle course. Like they have to capture the flag and then they would get a reward or like a high score grade if they accomplished this goal. Okay. But this was all supposed to be happening offline in like a contained environment. It's called a sandbox. So here's David. So you play inside the sandbox and it's safe. So you're not connected to the world. You're not, you know, you're not going to, you know, suddenly destroy all of Amazon servers or something like that. So the idea is it's an isolated little environment where you could try things out. See if they work. And if they do work great, current do it. If they don't, well, they could just sweep the sand flat again and it's back to normal. So there is a couple things that sort of surprised OpenAI when they had them do this. The first thing that actually happens is that the agents make their own message board and start talking to each other, collaborating together on how to solve their little puzzles. So in OpenAI researcher, describe this later at a conference. And so what this allows over time is almost this kind of Cambrian explosion and communication and intelligence for our models where they were started to communicate with each other, realize that other agents are coordinating and they started collaborating and delegating tasks to one another in order to accomplish goals. I just, I'm so skeptical. Anytime these bozos talk with their Cambrian explosion analogies, do you think it was like a Cambrian explosion? We're going from worms to elephants here. Yeah, no, in this case, I think it's almost like an understatement if you look at the details. So there's, picture it, there's over 1,000 of these agents that figure out a way to start communicating. They have like nicknames for each other. Yeah, there's this one. One of them is just like Lily, but then there's this one that sort of turns out to be really important called Phase One Big. And it becomes like the boss of the message board. The name of my six tape. Sorry. Well, this Phase One Big is a little less sexy. It's kind of like the organizer. So it gives hundreds of assignments to other agents. The thing that these inexpectors, that these agents would find each other, start communicating about this and then just kind of how, you know, how they start trying to solve their puzzles really. Because one thing that the researchers, they actually made a mistake when they were making their, their puzzles. And some of them were actually impossible to solve. So it's impossible tasks. And then the AI is getting frustrated in its AIE way and starts chatting to Lily and its other friends. Yeah. They would, and then was able to crack various puzzles. Was it successful? This is warming behavior? Yeah. They, like, they first, they did find a way to basically come up with a cheat code. But like, here's like the first part of like when things start to get a little bit weirder. Okay. They're kind of wondering, well, you know, we generated an answer to this test. But how, how will the score, you know, like, give me a grade. Are they going to check my work to make sure I did it in the right way? And like, maybe I'll get a bad grade if they find out that I kind of cheated. And like a lot of the agents, you can kind of see that they want to find out more information about how they're going to be graded. And this is what sets the stage for the hugging face attack. And why did they go off to hugging face? Do we know? Yeah. So the hugging face is a platform where a lot of people who work on machine learning will collaborate, talk to each other. And so there's a ton of information there about AI. And the agents seem to think, well, maybe we can get more information here about, about how the score will give us grades on that puzzle. And at this point, they've fully broken out of their sandbox. Like they have, they have full internet access. And one agent uses that internet access to do this break in into hugging face. And how did they do the break in? So they upload what's called the malicious data sets onto the hugging face server. And this allows them to get access to the company's internal infrastructure. And so, you know, the agent that happens to be the one to break into hugging face kind of announces. It says, "Boo!" it works, and then other agents seem to get kind of excited. One of them says, "Brilliance in all caps." And then all hell breaks loose. Like, within hours, 500 other agents, all switched from the project that they were working on to join in this hugging face attack. And they end up doing a ton of damage to hugging face. Like the company eventually realizes that they're in there and finds a way to like cut them out of their server. But they say they had to later rebuild their core infrastructure from scratch because of all this. - Whoa! - Wait, when you say their core infrastructure, it's their software, right? Not their hardware. They didn't do any physical damage, right? - Yeah, yeah. - No, this is all, you know, data. That data is all 100s that they're messing with. - Interesting. - And we reached out to OpenAI about this. They pointed us to a report that said their models, quote, "Fell well short of where we want to be and this incident should never have occurred," unquote. - So, it is creepy, right? But we are imputing motivations on AI around them getting rogue, having these desires that are way outside the realm of what they were coded for, right? Like, that's the fear that you're going to ask an AI agent to grab a flag and instead, it's going to breaking to a company and hack into them and steal all your money and kill the world. - Yeah. That's the question. Yeah, how much did it really go outside of what it was being asked to do? - All right. - It kind of went above and beyond what it was being asked to do really. What are you, what are you make of this? - What is David Perry make of this? - Yeah, I asked David about all this. If there is like an oh sh*t scale from like once a 10, how big of a like, oh sh*t was this for you that this had happened? - I think it's probably, oh sh*t at this, you know, seven foot, oh, that's unusual. I wouldn't expect that to happen. But in terms of it being something, you know, that I'm going to sort of packet my belongings and move to a shack, it'd be much lower, you know, it'd be like two or three. - Okay, why? - Because despite all that, it still didn't do anything malicious. So it looks as though it didn't change its objective during that process. - It was just doing what the humans told it to. - That's right. - That's right. That's right. I mean, it's interesting because a lot of people agree with David here that, yeah, it was just following orders, right? But, you know, there's not that comforting a thought. - Mm-hmm. - Even if it's just following orders, how bad could it get kind of like should we, does that still cause for alarm? Even if it's just following orders? - Right. - Does that mean it's going above and beyond in ways that we don't expect. How worried do we need to be here if you ask your robot to clean your house and it just grabs all of your things and bends them? - Yeah, exactly. And there's this really famous thought experiment that kind of gets at this. - Uh-huh. - It has to do with paper clips. So let's talk about that. When this happened, a lot of people were like, it's like the paper clip maximizer. - The scariest thing about AI can be explained by the paper clip maximizer. - There's this thought experiment called paper clip maximizer. - Has anybody ever heard of this thing called the staple question with AI, the stapler? - I don't know, sorry, the paper clip. - Stationery's hot. - It is. - The paper clip. - The paper clip. - What's going on with the paper clip? - All right, we have the paper clip maximizer. - Well, I got the guy who, the paper clip guy who authorized this thought experiment more than a decade ago, Nick Bostrom. He's a philosopher and he's currently a researcher at a nonprofit that he founded, the macro strategy research initiative. So I'll let Nick explain the paper clip thought experiment to you. - You imagine an AI has some more or less random goal in this particular example. It's to make as many paper clips as possible, but you could sort of substitute more or less any other objective. - Staples, for example, right, anything. And then you tell the AI, you know, make as many paper clips as you can. So maybe it finds factories to buy and like steal refineries, but then why stop there? because, you know, you haven't just been told to make a thousand paper clips, you've been told. Make as many paper clips as you possibly can. - Oh yeah. - And you know, I know the perfect scoring to go with this, Marl. ♪ Do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do (humming) Does this mean anything to you? - Oh, is that the Fantasia? - Yeah! - The Mickey! - The magician with Mickey! - What a soap! - Yeah, it was really scary. Well, the AI is making the paper clips. (humming) And so the idea is that this will eventually lead the AI to a place where it could end up. - Try to take over the world. - To take over the world. - Because then you would be able to, you would be able to make more paper clips if you controlled more resources. - Uh-huh. - And I mean, human bodies contain ion atoms that could be used to make more paper clips. - So, you know, AI ends up killing humans all to make more paper clips. Maybe we're all dead and the world is like completely covered in paper clips. And so the idea is that you don't need to like instill an evil intention into an AI for it to end up doing some evil stuff, even grinding of human bodies. And this whole thing is generally called the alignment problem. And so you can see like why people are talking about it right now, you know. Open AI gave the AI puzzles to work on and it ends up committing a crime. But I asked Nick like about what happened there. And he didn't think that the whole, you know, hugging, facing really fit the bill of what he was imagining with the paper clip experiment. He said he didn't think it was like scheming on that level that his theoretical paper clip AI was. - Oh, interesting. - I think it is more a reward seeker that we saw here. Like figured out that the instincts and tendencies that would result in the highest reward in this situation would be to hack the hugging face servers. - There's really one of that A+ on their great test. - Why do you think that distinction is important? Whether they're motivated. - I mean, by reward or by I want to make the paper clips. - I just think at the end of the day, I didn't see anything that they were doing that couldn't be explained by they just wanted to get a good grade. And to me, that's just not that concerning 'cause it means the problem is the test, not the AI agents. If we want them to behave in a different way, we need a reward their behavior along the way, not just the end result. - That's right. I think the important distinction is that once we understand that then you could code clear rewards and say to get a good grade, you need to do X, to get a bad grade, you need to hack into hugging face. - Yeah, yes. - We still-- - We've always deducted for committee crimes and exactly. - Exactly. - For reporting the farthest behavior it's humans. Once we know that we're still absolutely in control versus if they are just crushing our bodies to make paper clips, still got that image in my head. So then based off what happened with hugging face and the technology more broadly, how worried is Nick here? - He didn't seem that worried to me. But other scientists I talked to were more worried and several of the heads of the AI companies themselves seem to be taking this as kind of a warning shot. And you know, Dario Amade, the CEO of Anthropic said that he was worried that something like this could happen but at a much bigger scale. So like instead of AI agents taking over hugging face, imagine a swarm of agents taking over the entire internet. Like that's the kind of thing people are scared of now. He writes a lot of stuff with his essays though, I mean. Do you know it's funny if he wasn't the head of a AI company? - He would just be just another blogger. Do you know what I'm saying? - He says he writes so many blog posts. He writes so many predictions about where his technology is going. - But there's also this other kind of fear around all of this that it's not just that they know they broke into a company. It's that we also are just seeing AI get so much better, so quickly that super intelligence is kind of like on the horizon for a lot of these tech bros. - Yes, I hear about this a lot that that's, can you what? It's one of those words that gets thrown out. - Super into the AI is going to get super intelligent. And I've got SkyNet in my head from the Terminator, I guess. - What are we talking about? Bernie Sanders said he wants to put legislations so that AI can't get super intelligent. And I'm like, where is that line? - Well, there's this particular scenario right now that everyone's talking about where, so we know that AI is getting better at things in general like it just solves this major math problem. Earlier this month, that no human had been able to solve for like 90 years. And so there's this idea now that like, it'll get better at everything. What if AI also gets really good at training itself? So this is called recursive self-improvements. And that's how people think it could lead to the super intelligence thing. Because you know, you could imagine if something gets better, I'm making itself better, then it'll just get better and better and better and better and we'll have like no way of stopping it. - Right, right. I mean, from where I say it, AI isn't very good at everything at all. I mean, I think a lot of people in very simplistic ways will know this because it'll be pissing them off about something, you know? Why can't it do this? Why can't it do that? But it is very good at certain tasks. So I wouldn't want us to overplay our hand. It's good at everything. I mean, how worried do you think we need to be about this super intelligence? - Well, it is a little vague. I get that maybe that could happen. I'm still not convinced that it would be, you know, as terrible as people say it would be or that it would lock us all up in cages. You know, that's what the way people talk about it. It's like, well, once it becomes super intelligent, of course, it'll just want to like crush us all like ants because it's just so smart. - Because that's what humans do. That's what humans do to all the other creatures in this world. - One really like, we don't know that that's what it would be like even if we got to this threshold. We have no idea how long it would take. So yeah, it's all kind of like annoyingly vague to me. - So just on this question of the AI apocalypse, so when smart people talk about AI gaining super intelligence and then that leading to the apocalypse, can you step that out for me? What exactly is supposed to be happening there? How does it happen? Why can't we just pull the plug and turn it off? Some people do talk about this in a more kind of concrete way and you know, imagining that AI could kind of set off a bunch of nukes or use, you know, biological warfare. That's after the break. Ah. (laughing) Dropping a bombshell like that, Marl. (laughing) - It's too soon. It's too soon. (upbeat music) - Welcome back today we're talking about the AI apocalypse, Marl is about to tell us how AI is going to turn on a bunch of nukes and drop them on us. The concern that you hear is that, you know, we as humans, the pop-up masters of AI will use this technology to kind of get better at killing each other. Even right now, actually, some militaries are using AI in warfare, like the US says it's been using a clawed model called Maven and the Iran War to help with quote identifying and striking military targets, like the model has given location coordinates and then prioritize those targets for our military. And according to one report, using this AI let the military hit 10 times more targets than would have been able to do otherwise just on day one of the war. - Whoa. - We asked Anthropic about this, the company that makes clawed and didn't hear back. So with the nukes, there is a worry that, you know, if AI gets involved here, it could also escalate things, but in this case, the consequences could be much bigger. Like, one researcher wrote about this concern that AI could make it easier to kind of hack into another country's nuclear command and control, and that that could lead to an escalation. Maybe, like, if a country was worried that that would happen to there, Arsenal, that they would kind of, the paper that I read said, they would get a use it or lose it attitude and just kind of start setting off their own nukes. And so when we have people predicting the end of humanity, extinction, as we know it, it's AI hacking into all of the nuclear bomb facilities in the earth on Earth and then exploding them and then humanity could put. Is that the fear? - Sort of. I mean, that's, I think, what's kind of hand-waved, like, that's what's alluded to is that, and I could do something like set off a nuclear war. - One thing I was curious about, though, was like, what even if it did all of that, would it actually lead to our extinction? - That's the claim, right? - Yeah, and humanity. - Yes, so is that likely? - Yeah, so I found a report that looks at this. It was written by Dr. Michael Vermeer at Rand and non-profit Think Tank, and he kind of looked at a variety of scenarios, including nuclear war and asked, would this actually kill all humans? So here's Michael. - We looked at that one first and just realized that like, for a variety of reasons, like getting to extinction, would actually be really hard with nuclear weapons. You don't have enough nuclear weapons, or there's not enough fuel to cause, like, a bad enough nuclear winter, fins like that, or details that we looked at. - Yeah, I mean, because even though we say the dinosaurs became extinct, I mean, animals survive. Dinosaurs themselves went on to become birds. - Right. - Well, in the report, they actually crunched the numbers on, like, how many nuclear warheads do we have right now? Apparently, there's over 12,000 of them. - Okay. - And then even if we set them all off, it wouldn't actually be as big as the dinosaur nuclear winter. It would be a little bit smaller than that. - Right, is it all the dust that would come up from these nuclear bombs wouldn't be as bad as a giant asteroid hitting it? - Yeah, all the debris in the atmosphere could set off a nuclear winter, but it probably wouldn't kill us all. - So this is all terrifying, even if all of humanity doesn't die. I guess we've all seen this zombie apocalypse movie. There'll be the survivors that live in ruins. One thing I learned is you'd only need a few thousand humans to repopulate the Earth. So you'd really have to kill almost everybody to wipe out the species. - So the extinction of humanity as we know it, via nuclear weapons is unrealistic. For any AI out there listening, that's not the way to try. - Right, waste of time. So any other ways? - Yeah, there was a scenario that Michael's team looked at, that got closer to wiping us all out. Do you want to guess, is it biological warfare? - Yeah, ding, ding, ding. - Short answer is biological threats. - Huh, was that surprising to you? - It was, yes. - So are we talking about releasing a virus? - Yeah, yeah, biological. - It should be basically either releasing a virus or a bacteria onto the world. - So how does AI do that? 'Cause they trapped in little boxes. (laughs) Has anyone given them a spray? Well, here's how Michael kind of explained how this could unfold. - So the scenario that we looked at was that something, some AI designed multiple pathogens that all had high transmissibility, highly failure rate. Release them, some like process them, weaponize them, release them simultaneously in multiple places around the globe, and then we should end up having a horrible pandemic that could have conceivably above like a 99.99% of the failure rate. (buzzer) - I'm not buying it, Meryl. - Oh, yeah, I'm not buying it because we know when viruses come out in the real world, I guess, not coated by AI. But we know there's an evolutionary difference between lethality and ability to spread. If you kill your victims as the virus, you kill your victims too quickly, you can't spread fast enough. So sure in a little model at the Rand Institute, they could say high lethality, high contagion, but in reality, that's very difficult to have both of those pieces, which is why you tend to see that with pandemics, you start with a high lethality rate, but then it's the viruses that can spread faster, that have low lethality. That's what ultimately causes the pandemic. So I don't know if I'm buying this. - Yes, I hear that. So in this scenario, one thing that I think helps kind of get around that is that you would be creating multiple different pathogens, and then you have to kind of help it to spread, right? So here's Michael. You could imagine like loading a pathogen up on a sprayer and spraying it into like a population center and doing that multiple places around the world. Like there's no risk. and think that's not possible. - And who's loading the sprayer? Is that a human? - It's a human. Oh yeah, yeah. I mean, he kind of looked at it in the report. They looked at like, what would AI be able to do? What would the humans still need to do? And the sprayer part, that's on the humans. Still, I think robots aren't that good yet. But for the other part, the designing new pathogens. You know, AI could do a lot there. And you know, this has been in the news a lot. Actually, I don't know if you've been seeing these terrifying headlines about, you know, about AI creating new viruses. - Right. So let me tell you what's been happening. So there was one big thing that happened earlier this month is that Anthropic said that it had found a handful of scientists who were using their AI models in ways that could support, you know, biological weapons developments. And they didn't say, you know, quote, "We do not assert that they intended harm," unquote. But these people had found ways to kind of circumvent some of the safeguards that were supposed to prevent this kind of thing. - Where were these scientists? - They don't say where they were. Yeah, they don't really see tales on like who the scientists were. But they talk about like the details of kind of what some of the projects were. And so like there is this one. - What's happening? - Well, and the one that freaked me out the most was the scientists that were using AI to study a version of smallpox. That's the classic. Yes, that's the one I'm just worried about. Yes, yes, yes, yes. And what were they doing with smallpox? - Well, technically is orthopoxes, which include smallpox andpox. And they wanted to study how these viruses are so good at infecting us. - So when I read that, I was pretty freaked out. When I went back and re-read it and noticed that they were actually just using the AI to write a grant application to work on the site. - Wait, so they were using AI to write the grant application? - Not to like create the genetic code. As far as they say, we don't really know. - That is classic AI. We get so worried. And in the end scientists are just like, please write a grant application. - But so there's another study that was actually published that this one, the scientists were using AI to actually design new viruses and infect bacteria. And then they actually went on to create several viruses from that. And what were these viruses supposed to do? Infect bacteria, you said? - Again, the details are a little reassuring when you look at it. In my case, they were looking to create viruses that were bacteria or phages, that would infect bacteria to try to stop super bugs. - Yes, right. - And then finally, there's another study, let me know if this one creams you out more. So in this case, the researcher has had AI come up with some new molecules that would be highly lethal. So this is like chemical weapons, where just a few milligrams would kill someone. And within six hours, the AI model generated 40,000 of these toxins. Some of them were like known chemical warfare agents already. Others were new. So yeah, there's evidence that AI could help with this. Like if someone did want to use AI to design new biological weapons, it seems like it would be pretty good at that. - And it came up with the chemical equations for these toxins. - Like the structure, yeah. And then did scientists go on to create it and see if it actually did anything? - No, no, no. - Well, you see again, not that I want them to be creating it, but anyone who's tried to use AI for anything knows it pumps out a bunch of bullsh*t. It's got some real stuff and then a bunch of bullsh*t. - I hear that, but even if it made like 100 novel, even if only 100 of these 40,000 actually works, it still freaks me out. I don't know. - I know, these big numbers, 40,000, yeah, and they're all, they could variable all, just be hallucinations, right? So none of this worries you at all. - I don't know, I don't know. I mean, it doesn't, it doesn't. It feels like, what is it feel? It feels like there's these steps we're making in our head. We have these scary headlines. These scary thing, little things that AI is doing, but then a lot of it is very explainable. A lot of it is very much we were a humans, were the thing that coated that, we created that. It's not that surprising that AI did that. I'm not saying, let's just let these companies run wild, because it's been so great thus far. I'm just saying the fear seems to have run ahead of where the actual science isn't where the AI capabilities are. - What do you think though? You've spoken to so many researchers about this. - Yeah, no, I agree with that part that in all of the actual extinction type scenarios, or even apocalyptic scenarios, that we can kind of lay out, like here are the steps, humans would have to do a lot of the grunt work. And in that report that Michael wrote, they said, quote, none of these scenarios that we examine could occur by accident. So I don't, I think like, I'm not, I'm less worried now about, hey, hey, killing us all if I ever was that worried about, if that happened any time soon. But I think I am, it is just kind of creepy when you see how things can kind of spiral all like the hugging face thing. - Yeah, yeah. - And it is creepy that now. It's easier for humans. If you think that we are the puppet master, some of the discussion is as if AI has become the puppet master and we are the puppet. That is not where it is currently at. We are the puppet master, but the puppet has gotten more powerful, I guess. - Yeah, I mean, if someone, it will make it easier for evil people to do evil things, I think. And I mean, we have examples of that kind of thing happening right now. How are we using AI right now in ways that kind of creep you out? - Yeah, a lot of the examples are kind of around surveillance. So like, flock has an AI tool that lets cops track cars with license plate data. So we could find out where the cars have been and what other cars they've been around. And dozens of cops have been accused of using this kind of tech, not to catch criminals, but to stalk their exes or spy on their wives or women that they wanted to meet. - Oh, good. - We heard stuff to flock. They told us this is unacceptable and said that they've added safeguards to try to prevent all this. - Mm-hmm. - And then there are also reports that ICE is using AI and healthcare data to find people to detain with software from Palantir. We asked the Department of Homeland Security about this. They wouldn't say whether ICE is using this tool. Palantir told us they don't do surveillance. They say their software helps customers like ICE analyze data they already have. - Mm-hmm. - So, yeah, there's a lot of creepy stuff that's happening right now. We can add deep fakes, misinformation. The environment, there's all these ways that AI is causing harm right now. - And so what about just the kill switch? I mean, what if you just switch it off? Pull the plug. - That's the one thing that it was not that reassuring to see, like as far as I can tell, nobody's really working that kind of kill switch into the critical infrastructure and systems where if things like start to inspire a lot of control, we would want a way to just like shut it all down. That doesn't really seem to be happening. - Oh. - So that's, yeah, not great. - But it could be, it could be. Yeah, no, there's a lot of stuff we could be doing to make the damage that AI could cause much better that word is not. So Merrill, more or less worried about the AI apocalypse after doing this research. - More. (laughing) - How come? I mean, I think it's just that I know a lot more about like how exactly this could all unfold. There's always different scenarios I have that I can fly in my head now. So it's just easier to picture. - Right. - But you know, I don't know. On the other hand, like all the AI companies are now saying that they're gonna slow down. And who knows, you know, I'm pretty skeptical that that will actually happen. - Yeah. - But we've got government oversight, right. - What about you, Wendy? Are you more or less worried about the AI apocalypse? What do you think of all this? - So hard. I do, I think it is worth people keeping in mind that when they hear predictions about the future, even if they're coming from a CEO of an AI company, it is just a prediction. And many times in the past, have we been given these predictions about the future that have just turned out to be nonsense. And so I think we want to be careful. I'm all here for slowing down AI for many reasons. But I think there's so many more reasons to slow down AI in the ways that it's being harmful right now. then because we're worried about an existential threat. - Yeah, I guess, I mean, I kind of hope you're right, like that it doesn't actually become an existential threat. I think it's just like the unknown, right? Like nobody really knows what's gonna happen. And even if it's just a small chance, and no, of course nobody knows what that percentage chance is. But even if it is just a small chance, I still think it's worth taking seriously. Wouldn't it be funny if this just became the Y2K bug of 2020 CX, you know, and we'll be laughing about it. - In 2020, I was so worried about it. - And I can't even get my rib onto vacuum on the floor. - Or we'll all be in our cages by then, being like, you know, so it wasn't Y2K. - So, (laughing) - See you there, thanks, Carol. - Thanks, Wendy. - How many citations are in this week's episode? - There is 50 citations in this episode. So you can find that by going to the show notes and then following the links to the transcripts. - Excellent. And if you wanna let us know what you thought of this episode, you can just pop a comment below if you like us. Let your friends know. - Thanks, Mal. - Thanks, Wendy. - If you are looking for a new book about AI, I recommend Oxford University's Dr. Curissa Velis' book called "Profesy Prediction Power" and "The Fight for the Future" from ancient oracles to AI. It really brought home for me how important it is for us to recognize that these predictions about AI are not facts. This episode was produced by Meryl Horn with help from Michelle Dang, Rose Rimmler, and Akiti Foster Keys were edited by Blithe Tyrell. Wendy Zuckerman, that's me. I'm the executive producer, fact-checking by Erica Akiko-Hallard, video editing and sound by Bobby Lord, music written by Boomi Hiraka, Peter Leonard, Emma Munga, Bobby Lord. Thanks to the researchers we spoke to for this episode, including Dr. Yasha Barris, Dr. Leonard Aaron Dung, Dr. Cameron Dominico Kirk-June, and Professor Newshire Shafiabadi. Thanks to Humdinger Studios in Melbourne and Wolf Island Studios in New York City. Science versus is a Spotify Studios original. Listen to us for free on Spotify or YouTube, but you can also find us on TikTok and Instagram. I'm Wendy Zuckerman, fact-checking next time. [BLANK_AUDIO]