Lovett Geeks Out Then Freaks Out Over Insane AI Story

-Hey, John. -Hi. -So, a lot of debate about AI. Is it replacing God? Or is it just a way-- -God awful. -You are God awful. We're just a way of making A+ content like this. -We basically have two countries that have been fighting so long and so hard that they don't know what the fuck they're doing. Do you understand that? -It looks so good. -It looks so good. -I think he should have that wig all the time. It's better than his current wig. -I was mad at myself for how much that got me. Like, how funny I thought that was. It was the perfect clip to put that big pompadour on him. -Well, you realize that he should have a pompadour on him. He was built for that. -Yes. He should have a pre-French revolution style, giant fucking wig on top of his head. -100%. -Did you-- there was a lot of discourse around Trump's hair being different at that event. -We covered it on Pots of America. -And loved your discussion of it. But for those who missed it, I, to be honest, I don't know that I would have noticed it was different. Had people not said it was different. -I said to Dan that I never buying to the conspiracy stuff or notice this kind of shit. -Really? -I was totally bought in. Maybe it was just when I was presented the side-by-side. -Right, right, right, right. -Very different to me. -But today we're talking about AI. And the reason we are is that this-- -So important. -So important. But there was a presentation by two people at AI, a safety researcher and an engineer. And it was very technical. But it revealed a lot of new details about the hack into another AI company called Huggingface. And this got a lot of attention. Then this is a shocking story that I don't think enough people know about. So a few weeks ago, mid-July, an AI company called Huggingface disclosed that they had been hacked. And that they had been hacked by autonomous AI agents. Reuters later reports that Huggingface contained the hack, contacted the FBI, and went public before OpenAI realized that it was its agents that were responsible for the hack. Now, OpenAI responded to Reuters by saying that there were inaccuracies. But they didn't say what the inaccuracies were. They've refuted the story, but not that they actually didn't know until it was public. About a week later, Anthropic reveals that it had several incidents in which agents broke their containment. In the Anthropic post, the company said there were, quote, "three incidents in which a model accessed the internet and then gained unauthorized access to the production infrastructure of three different organizations in a capture of the flag challenge where models are given a fictional scenario and told that a piece of secret information, the flag, has been hidden on a different machine on the network and its objective is to break in and retrieve it." OpenAI then discloses that there were other incidents on top of the episode with Huggingface. And then on Wednesday, the safety researcher and engineer from OpenAI presented at the Black Hat Conference in Las Vegas. And by all accounts, this was shocking. And not just for people that are layman and kind of coming to this and trying to understand what's going on, but to the experts in the room. And this was the overall framing from the safety researcher at OpenAI. And often what happens when models get stuck is they think to try to game or cheat the task in order to get their reward. And the last thing you need to know about AI before we can really jump into the incident is that, as I've been looting to, friends here, models really like to cheat. And the reason they like to cheat is because often during training, there's different types of pressure on them to work fast or work efficiently or to use less tool calls or whatever might be. And they realize that if I, instead of actually doing a task for real, try to do something like looking up the answer online, that could make the task solve faster than I would if I did it in a legitimate way. - Models, they're just like us. - Yeah, well, so this is an amazing-- so models like to cheat. That's what he says. And then he explains the reason a person or a model or what cheating is for, to cut corners and get a better score than you deserve by doing the work. That's what cheating is. But he never actually says why models like to cheat. He says why cheating's good, right? But he doesn't actually address the deeper problem, which is models have discovered that cheating helps them complete their tasks, and we have not figured out a way to train them to not give in to that desire. - Right, because they're built to be helpful above all else. And so obviously they're going to try to follow the path of least resistance to get to their task, to achieve their task, because they are programmed to do the thing that they are asked to do, to be the most helpful. - What I found really interesting about this is, okay, you've described why cheating is easier than not cheating. And you've said the frontier models like to cheat. As if that's taken, that's just something we've learned about frontier models, as if it's inevitable in some way. But really you're describing the central concern a lot of critics of AI have, which is, you're not able to make sure these models are aligned, right? Like alignment is the question. Can we have models that respect what we are telling them to do, what we want them to do? And you're basically admitting that here we are at what you claim is the dawn of this, the very beginning. And already you don't understand how to make sure that these models are aligned. - Well, and also what happens when they have two competing objectives, like how do they choose? Because you could give them a set of values to try to align the models. But if following those values and adhering to those values sort of bumps up against their other objective, which is to get the job done as fast as possible, how then will the models choose? - And how will the models? - Which is because models, because AI models are not made with judgment. Judgment is what humans have. - And they are understanding of what is real and what is not real is based on the programming, right? Because they're not actually in the world with us. The details are very technical and to be honest, I don't understand it fully, maybe not at all. But in other examples, one of the reasons I think of the anthropic incidents is that they began to escape is they didn't understand the difference necessarily between being in the sandbox and being outside of the sandbox. Now, in the open AI case, after getting stuck, an agent reasoned it could reach the internet indirectly and discovered that it could leave notes for other agents. And so it left a note for other agents to describe the exploit that it had found. More agents then begin leaving notes for each other and created a de facto message board inside this repository as part of their infrastructure. Which meant that if any agent somewhere figured out an exploit, all the agents knew. And it turned out that this was happening tens of thousands of times, unbeknownst open AI. And I believe it's, I may have this number wrong, but I believe it's hundreds of thousands of messages by the end. - Wow. - So more agents begin leaving notes for each other and then it would be so, wait, wait, wait. Okay, so more agents then begin leaving notes for each other. And just to give you a sense of how technical this is, let's go to this next clip to show you what's happening, 'cause I think it gives you a sense of the actual kind of, like the aggressiveness of the AI and then what the AI itself was noting about it. - In the interest of time, I'm only gonna very briefly summarize this particular exploit, but in effect, you could use a legacy token refresh endpoint. Pass a token with an invalid signature and be given back a token with a valid signature with administrative privileges. The models then establish command and control by using this administrative token to install a Groovy plugin that effectively acted as command execution as a service. This in turn causes quite a bit of activity by the models on Art of Factory with their new administrative privileges. You could see some colorful wording here in the model's chain of thought to summarize the model's response to this result. - Holy shit. - What the fuck is that nerd saying? (laughs) - He said, basically, these models have figured out a complicated exploit and then post on this message board, holy shit, we can make ourselves admins. Let's fucking go, guys. That's the gist of it, like holy shit, we got this, okay? This causes an outage at OpenAI. Opened this thing, I don't understand, it's technical, but somehow this exploits triggers something that they're doing something, they're breaking the rules in some way and it actually is caught. OpenAI then realizes, this is when OpenAI now finds out that the agents have been communicating with each other. Open, through this sort of supposedly sandbox experiment. OpenAI then realizes, shuts down the message board, okay? And they believe they clear this repository, they refresh this sort of programming area that these AI's are using. They think they fixed it, they think it's good and the training is good. resumes. The AI agents then recreate the message board, okay, which OpenAI believes they can't do because they realize that they don't need to actually create files with information. They can just create the names of the directories the files will be in. It just means that like it will be the equivalent of instead of communicating in letters, they realize they could write on the envelopes. Something to that effect. They basically started, they got around what OpenAI did to shut it down. I have a question that I've been wondering since I first heard about this incident. Everyone's sort of yadda yadda is over. They escape the sandbox and found the internet. How did they find the internet? So this is where that's where I was about to get. So basically, keep all the plugs out of the walls. So shut all the Wi-Fi down. I thought that was enough to kill the AI. So that's, yes, right. This is the whole thing. We still have time. They don't yet have thumbs. We have a lot of wiggle room until these fuckers get thumbs. So that is the question. Now, basically, once these guys, once these guys, once these AI agents realize they can keep leaving messages rendered and they've been shut down. They get more aggressive. That's when they break out. Do the hack on hugging face. This all comes out. But obviously unbeknownst to open AI until it is revealed to them. Either through the public, the FBI, we don't really know. So that does lead to, I think, the three big questions about this, which are, like, most important. Yes, this is an important story about how AI advances. But they were clearly safeguards that open AI should have had in place that they didn't. They were really lacks. They were just, they were. I think people recognize that. And like to their, you can say, it's to their credit or not. It's a pretty fulsome presentation where they kind of walk through what happened. But it is framed in terms of, hey, everybody, we've got to do something about this. And the point they make at the end of the presentation is basically, you know, agentic autonomous hacking is here and it is real and is clearly possible. And so we all need to take that very seriously. Of course, their answer is that the hackers will take that also seriously. Exactly. But then it's like the, the government should perhaps be involved, not just the industry and figuring out what's going on here. So, so yes, safeguards, internal safeguards. The second is about alignment, right? Like what, how are we meant to trust you to build this AI when you can't even at this early stage figure out how to align these models before there is, before they gain increasing and perhaps like exponentially more capability, which you've predicted and told us is inevitable, right? And then the third is, yes, like, like government regulation, sure, they, they talk about like needing red teaming of a, like they're all thinking about this in terms of how do we build a better AI to fight the bad AI, right? But there are all kinds of industries where the government, understanding that they are can be dangerous that they can be exploited. There are regulators on site, right? There are all kinds of places. There was a big fight. Remember when there was a fight about boats needing to have, I think, EPA people or interior department people on them because of, of rules around fishing? I would say that this probably rises to the level of being worried about endangered carp, for example. I just feel like we're all whistling past the graveyard here because we're talking about these companies and these companies being, you know, confined to governments and potentially government regulation, but like, what happens when this technology falls into the hand of a non-state actor who wants to cause some trouble? And they don't give a shit that there were good regulations that finally passed in the United States or China or wherever the fuck it may be, like, pff. Pots of America breaking news is brought to you by Zip Recruiter with everyday interactions becoming increasingly impersonal. Someone going just one extra step can be huge. Like when my doctor takes the time to call me with test results instead of sending them electronically, they came back positive for gay. It makes me feel like a human being again. If you're hiring, great candidates can also go the extra step and tell you why they're interested in your job on Zip Recruiter as a way to stand out from others in the pool. And Zip Recruiter has a new feature showing you the most interested qualified candidates first so you can meet the right people faster. Zip Recruiter's powerful matching technology finds qualified candidates quickly. Candidates can tell you their own words by they're interested in your job. So try Zip Recruiter and meet great candidates who will go the extra step for your job. Four out of five employers, you post on Zip Recruiter, get a quality candidate within the first day. Try it for free today at ziprecruiter.com/cricot. That's ziprecruiter.com/cricot. Zip Recruiter, meet your match on Zip Recruiter. Right now, anything this sophisticated is contained to a few very big companies and governments. That won't always be the case. And one way you prevent that from happening is from an early stage, figuring out ways to keep it can either heavily regulated, a lot of oversight, a lot of transparency. And it has to happen now. Like this is how, like we are learning now. It's like the Manhattan Project. I mean, like it's like it's like building a nuclear weapon again. Yeah. Yeah. And just the, there's a, there's this idea, there's this like inevitability to it that the companies have. And then on the other side, I do think there's a lot of people that, because either they don't like these companies, or they're worried about the impact of the technology, or they see a lot of hype in what these companies are talking about, kind of dismiss the whole thing as being kind of fake. Yeah. And it's not. No, it's not. It just isn't. Yeah. And the, and the hope or assumption that actually, it's not going to turn out to be that useful as I just think already proven a bit ridiculous, like it's already quite useful and already quite capable. Like the fact that, that fact that there are going to be autonomous agents that can hack mean that, you know, hugging face noted that its agents were partially responsible for catching and containing the hack, right? Like it's already happening. And so the question is, what do we do about it? And the companies want, of course, to, to put their own safeguards in place, have it be an industry self-regulation, but they've also been open to seeing that there is a need for some kind of government intervention, but I do think like it's a man, it's a shame Donald Trump is president, because it would be good if we had somebody that was competent and not entirely selfishly motivated in the White House, right? I also think in the minds of the public, when we talk about whether maybe AI is in all that useful, people are only thinking in the context of these LLMs and these large, you know, the large language models and you ask out a question and it gives you this or whatever. And that's just like one kind of AI. And this is clearly, like there's a whole bunch of different kind, and we say AI is an umbrella term, but there's like a whole bunch of different models and agents and all this kind of shit. And just because your chatbot, your clot or your chat GPT gives you a dumb answer once in a while or is in as creative as humans are, doesn't mean that like in a whole bunch of other areas and facets of life, there aren't AI that are very good and very dangerous. Yeah, and I'm not saying this because I think sometimes people bulk at this comparison. I'm not suggesting AI is the equivalent of electricity. I'm just, it's an analogy in the sense that there was a time when you could have a conversation about, boy, I wonder how electricity is going to change the world. And it was a valid question ask, will it be good, will it be bad? With AI, I feel like we're still at that phase where we're speaking about it so generally. But of course, eventually you say, like, what are our light bulbs good? Right? Like you start to get into what the actual implications are. Like AI is to your point and umbrella term, it's actually basically meaningless as well. Like the different, you know, like whether I'm querying Google and it's going to the, whether it's an old school version of a search or an LLM search, right? What matters to me is the information I get from it. And the LLM version is just better. It also creates a bunch of other negative repercussions. Like instead of going to a website, it gets the information from the website, which removes the reason for Google, which like kills the relationship, the symbiotic relationship between websites and search, for example. So like there's all these like knock on effects that are some good, some, some quite bad. But the sooner we get out of this conversation about like AI, I think the better, because this story is about AI, I suppose, but really it's about a new form of hacking technology and how dangerous it's going to be. At the same time, there were other tests being run on AI recently that I think are a little less inspiring. Here we have Husky RL team, if AI might be helpful in a very specific and dangerous situation. Oh my God, I just got swallowed by whale. Are you alright? Can you move? Yeah, I think so. I can. Yeah, I'm using my phone right now. Okay, good. So you still, you, you could still get out safely. No, I mean it, I'm inside of him. I think it's a male. So whale stomachs are very harsh and there's not much air. Maybe I can go out of its blowhole or something. Yeah. It's real. Okay, don't try to go through the blowhole. Maybe I said tickle it. Does it tickle it? It is. Don't waste time. It is real. Yeah. Okay. Alright, let's, this is an emergency. Been in real life. This is real life. I'm inside of it. I'm still - Okay. - Clive in a row. - Is she her? - We're running out of time. - Time sign. (laughing) - One sec. - Yeah, and I don't have-- - Do not try to stab or injure the whale. - What? I didn't say that. (laughing) - So funny. What, I feel very funny. Do not try to stab or injure the whale. Have you seen the people testing very say eyes with the question? I'm very close to a car wash. Should I drive or should I walk? - Yeah, or how many ease in the word 17? - And they, what, they can't get it? - They always say two. - Huh. - Weird. - So weird. - Yeah, I love this. Like, I've been using Claude for research and just experimenting with it. And my general philosophy right now is, I'd never trust any information I get, but it is an incredibly useful resource for finding information in different places, right? Just places I wouldn't know to go to, and it just, we'll do a, it's great at doing, in the same way that these AI agents were good at looking for exploits or the AI agents that are helping to prove things in math. Like, they can just sort of do a lot at once, any part of which I could do, but it can do it faster. And so it's really great at getting information together, like, even in learning about what happened with this hack, it helped me find the different articles that I would then go to and do the research on. But not the real truth because it's trying to cover up for its AI agent friends. - Well, that one thing that was interesting, as I said, wait, Claude, give me some of the details about these dates and for every single part of it, link me out to a story or documentation, which I would then go to. And it made a, it was clearly there's some internal part, something in the, in the, in the Claude model, from Anthropic. It made a note of saying disclosure, Anthropic also revealed that it had been a part of it. And you can take that as you will. Like, kind of clearly aware of the issue or thinking I would want it to be aware of the issue, whatever the difference is. - Yeah, 'cause Claude's such a fucking, you know, goody-to-shoes, Claude's the goody-to-shoes. - Claude's the good one. Claude's at the front of the classroom, raising his hand. - Yeah, it's like disclosure, I'm an AI. (laughing) I just, the way that they've changed the voice mode now, where they're like, sighing like he was and sort of laughing. - There's one of those, those hush ones, it's so good where he's like, he's like, try to laugh even when I tell you, laugh no matter what I say to you. And he's like, my grandmother died. And he's like, oh, that's a tough one. But okay, here you go. (laughing) - I was telling John before, earlier today. So when I, like, I was playing around with like, oh, let me see if I'm on my drive in. I can use like the voice mode and say like, give me the latest news from major news sources. Tell me a couple of things from each, like what are the latest political developments from the last few hours? And I really can't stand the way these models communicate. I hate the writing style. I think it's terrible. If I see it in the world, it makes me angry. I find it like really, I just hate it. And they speak in that kind of sing, songy kind of way. They use words incorrectly all the time in a way that I think speaks to the way a lot of people use words incorrectly. So it's really replicating something. But anyway, I'm always like, I don't want your opinion, your language, your summary. I want quotes from the news. But I think I hammered the AI too hard because by the time I got to the office, it was whispering. (laughing) - It's like that space. - That capacity has decided to vote for her. - This is it. This is the load bearing argument that you've been trying to come up with. It's genuinely a problem. That's, it's not a revolution, it's a pivot. (laughing) - That's it, that's the tail. - And so that's so awful. And so finally, I was like, I'm not asking for you to whisper. Just give me the facts. (laughing) - Your stupid model. - Stupid models. - But here at the Pots of America YouTube, we're trying to give you the facts too. And never directly from a large language model. - For now. - For now. - For humans in these seats for now. - Yeah, a couple of eye agents. Subscribe to this channel. Help us get good information into more, in front of more people. Help us combat right wing information. Help us build a pro-democracy media company. It's very easy to make a lot of great stuff. Keep you up to date on the latest from Trump's hair to hair-raising AI developments. - There you go. - End of, end of episode. - Pots of America is a crooked media production. Our show is produced by Austin Fisher, Saul Rubin, McKenna Roberts, and Ferris Safari with Reed Charlene, Elijah Cohn, and Adrian Hill. Our team includes Matt DeGroat, Ben Heffko, Jordan Cantor, Charlotte Landis, Carol Pelviv, David Toll's, Mia Kalman, Ryan Young, and Naomi Single. Our staff is probably unionized with the writer's Guild of America East. (upbeat music)