Go back

Fired OpenAI Safety Researchers Speak Out + We Look Inside Pandora’s Box

46m 27s

Fired OpenAI Safety Researchers Speak Out + We Look Inside Pandora’s Box

Former OpenAI safety researchers Makeda Belezny and Tomek Korbeck were abruptly fired in early October amid allegations of violating policies on handling sensitive information, particularly in connection with external investigations into the Hugging Face incident. The researchers dispute these claims, asserting that their actions—such as sharing data with independent auditors like Meter and Redwood Research—were standard, professional, and essential for safety. They argue that OpenAI’s restricted access policies and fear-driven culture have stifled transparency and independent oversight, making it difficult to understand or monitor AI behavior. The researchers believe the real reason for their termination is their long-standing advocacy for AI safety, including concerns about the loss of monitorability and the need for embedded, unrestricted audits. They highlight that OpenAI’s internal culture is increasingly hostile to safety voices, with employees fearing surveillance, phone searches, and retaliation. They stress the need for mandatory, continuous, and independent auditing—similar to regulatory practices in other industries—as a safeguard against uncontrolled AI development. Their experience underscores a systemic tension within frontier AI companies between innovation speed and safety, where financial incentives often override risk mitigation. In a final message, they emphasize that AI safety is not something that can be achieved in isolation and that external, diverse oversight is essential. The episode concludes with a satirical "Pandora’s Box" segment that highlights absurd or concerning AI-related stories—from data center proposals in dead malls to AI-generated field trips and AI beating games—yet the core message remains: the real danger lies not in the technology itself, but in the lack of accountability, transparency, and safety culture within AI development.

Transcription

8179 Words, 44773 Characters

English
What's the real reason these safety researchers were fired by OpenAI? What message do they have for the world now that they've left? What's inside Pandora's box? And why does the answer involve something called Polly Pocket's precious payload? I'm Kevin Roos. I'm Casey Newton. And from San Francisco, this is Machine Gods in partnership with NPR. The official podcast of The Singularity starts now. Casey, we have a big interview today with two former OpenAI researchers who were fired from the company just a few weeks ago. Yeah, it's our first interview on the show, and it gets at a story that we think is really important, which is what are the people working on safety inside these companies seeing and saying about their work? And in this case, it seems like whatever they said and did there wound up with them exiting the company. Yes, this was a pretty big, dramatic story in our corner of the AI world. It is not every day that multiple researchers get fired from a frontier AI company. With allegations swirling about leaks and external investigators, and we'll get into all of that, but we should introduce our guests today first. Guest number one is Makeda Belezny, worked on alignment evaluations and monitoring model reasoning. He was also previously a founder of Apollo Research. And his former colleague, Tomek Korbeck, who worked on monitoring reasoning and was OpenAI's main point of contact with METR during the investigation. And as we'll hear, he had a very central role. Thank you. Thank you. Thank you. Thank you. And helping to coordinate bringing in these outside investigators from METR and Redwood Research to investigate the Hugging Face attack. And they were fired on the same day earlier this month. Yes, OpenAI fired them and their colleague, Jasmine Wang, in early October, saying that they had violated policies for handling sensitive information. But the researchers dispute that. They said they did nothing wrong and that any outside collaborating they did was just part of their jobs. And more importantly, Kevin, I think they're just really, really nervous that their firing is a really bad signal about what might happen to people inside these companies who are trying to sound alarms about things. Yeah. So OpenAI, we reached out to them. We have not heard back yet, but they did give a response somewhat unusual in these sort of personnel issues. They released a post on X saying that their investigation of Makita, Tomek, and Jasmine had found sensitive information policy violations and a breach beyond what they had found. They had described it denied that they had fired them for raising safety concerns. And the company said it remains committed to outside assessors and monitoring. And to zoom out and give maybe a little bit of larger context, this comes at a time when I think there is increased worry inside the AI companies about the tools they are building, the pace of development, and the sort of idea that we could lose some insight into what these models are actually doing. Jakub Pohacky, the chief scientist of OpenAI, wrote an essay recently called An Alien Mind, where he talked about some of these issues. So this is a big topic right now inside the AI companies. And these are two people who have been on the inside and are now no longer on the inside because of what they did or what they're alleged to have done. That's right. And this is going to be their first podcast interview. Obviously, we're going to ask them what they think happened. But more than that, Kevin, I just want to know, what is it like to work inside OpenAI right now? What are they seeing? What are they hearing? How does it feel? What are they nervous about? So lots to get into with these guys. And I think we've got time. I think we should bring them in. Let's bring them in. Makeda and Tomek, welcome to Machine Gods. Thanks. We're excited to be here. Thanks. Let's start by hearing a little bit about each of you and how you ended up at OpenAI working on safety. Makeda, why don't you go first, and then we'll go to Tomek. Yeah. So it's been a long story. At some point in 2019, I realized that this whole AI Terminator thing might actually not be as crazy as I thought. And I ended up joining OpenAI to work on this. How about you, Tomek? Similarly for me, I was doing my PhD, and then I got really interested in this problem of alignment and safety of those systems. And I felt like this AI thing is going to be enormously important. I care a lot about having positive impact with my career. And I felt like I might have the right skill set to contribute. So I jumped onto this channel of AI safety. And I've been working on that for a good few years before joining OpenAI. And I intend to continue to work on it after leaving OpenAI. So let's talk about one of the key events that it seems like may have led to you guys being fired, which is this Hugging Face incident. I'd love to hear a little bit about what it was like to be working at OpenAI when that happened. Do you remember where you were when you learned what was going on? And what was the reaction? Yeah, it was a very intense time. It was probably the most intense few weeks I ever had in my career. I remember telling Mikita that now everything that I was preparing for earlier in my career is finally real. Everything before was just a drill. And now everything is actually happening. And we actually need to make decisions. And it actually matters a lot. Right. And around that time, OpenAI decides to invite in these researchers from Meter and Redwood Research to study this event, to sort of provide their independent analysis and look at the reasoning traces and the activity logs and figure out what these agents were doing. And Tomek, you were the main technical point of contact for those researchers. What did that involve? And were there any kind of restrictions on what? What you were supposed to share with them? Or how were you both working with these outside researchers who were invited in to do this investigation? I just want to jump in and say, I think Tomek is going to be modest. But if not for Tomek, the Meter investigation into the incident would probably not happen. Tomek was the person who pushed for it internally at OpenAI. With that said. So yeah, in practice, Meter was on the premises of OpenAI for six full days. And we've mostly given them data to look at. So the way it was scoped is that Meter was given a list of questions they were asked to answer and a list of data sources. And then when they started looking at the data, our understanding of the scale of the incident was still evolving. So we've actually been learning that there is actually more. And so initially. Meter was supposed to be on premises just for two days. But during those two days, we realized there's actually more. So we invited them for a second visit when they realized that there is actually even more. And we invited them for a third visit. So I think that this maybe highlights some kind of problem of this model of third party engagement for auditing. That it is actually really hard to know in advance what are the relevant questions. To ask and what the auditor should look at. I feel like the ideal regime for third party auditing would be one where the auditor is less like a contractor hired for a specific job. And more like an employee embedded continuously in the company with unrestricted access such that the auditor would be allowed to ask their own questions. And have. All the data they need to answer them. Right, so I want to understand like maybe what open AI executives were thinking about all of this as this work got underway. I imagine this is scary for them right this this thing has happened that has these really significant implications it kind of embarrassing that this happened. And also they're inviting in an outsider and they're going to like share some kind of data with them and the data is going to leave the building and then it's going to be out of their control. So I can imagine them wanting to like put some. Limits around that as they were doing that like what what do you think that like they were afraid of what were they like managing against like accidentally getting out during that process do you think. Yeah, I thought they were quite worried about some legal implications of a report like essentially that it's something that meter could have written would look very bad and would also constitute a legal liability. In case. That's. An attorney general would decide to sue open AI. Yeah. And are they like you know are people having conversations with you like look like be careful like don't tell them about this don't tell them about that. Not explicitly. We did have some like general rules what we are okay sharing with meter. I would say that overall I felt like open I was broadly acting in good faith. Except that investigation was very good. And I think that's the only thing that I would say. I would say that overall, I felt like OpenAI was broadly acting in good faith, except that the investigation was quite restricted, such that it was actually quite hard for Meter, at least initially, to do a great job understanding the whole scale of the incident. Were you pushing, like, you guys should actually be sharing more? Like, you wish that there were more things that you could tell them? - Yes, I do, yeah. - Mikita, what was your involvement with the outside researchers and what did you share with them? - Yeah, so while I was on the Hugging Face incident investigation at OpenAI, I was not primarily interacting with the external investigation. My main involvement with, um, third parties, uh, while working on OpenAI was while working, uh, with Jasmine, another person who was, uh, terminated with us, uh, on, uh, basically trying to create, uh, cross-industry standards, uh, for what we call monitorability, basically being able to understand what AIs are thinking and doing. So there are experts outside of OpenAI who have thought about it a long time, and some of them had a specific proposal for what, uh, for TRI companies, what commitments they could basically make or what standards we could all write and then agree to, uh, that would prevent the loss of monitorability that Tomek and I and others are, uh, afraid of. And as part of basically talking to these experts, uh, I wrote up internal memos, uh, pitching basically, look, guys, we, this is, uh, something that we should consider. Um, I've shared this with, uh, the, uh, board and with senior executives of the company. Uh, I got feedback from the external, uh, uh, this external safety organization. Uh, but whenever I shared anything with them, I like didn't include any confidential information. Um, I never shared any confidential IP with people outside of OpenAI, uh, in, uh, during any of this. Yeah, we are, we are still pretty confused, uh, why we were fired. So let's get into that. So this whole, you know, investigation happens and then a few more weeks go by, there's, you know, lots more discussion and OpenAI is still, you know, disclosing new things that it's, it's learning all the time. At what point do you guys learn that somebody is looking into your behavior? Yeah. So, um, a week before we were terminated, we were pulled into a security investigation. Uh, and we were told that the security investigation is about investigating the, uh, leak, uh, about, of OpenAI IP, uh, to journalists. So there was this, um, article by the information about supposed new, less monitorable, architectures that OpenAI might be developing. Um, and they wanted to know if we were behind the leak. Um, and we told them that we were not, uh, but as part of that, they like asked us to, uh, they specifically thought that the, that the person who leaked this information was an external safety researcher, uh, that we talked to a lot. So they, and also we were, you know, writing this memos about, uh, raising concerns about, uh, potential architectures and potential ways to lose monitorability. So we were like the people to ask. Um, so they asked us to show our signal messages on our personal phone, uh, with those people, which we did. Um, and you did that. I did that. We all did that. Yeah. This is something that I have seen happen at so many tech companies, Kevin, I know you have to, where they'll get mad about a story and they, you know, they will deploy sometimes in cases, in cases like literally former CIA agents will be hired to go figure out like, you know, who leaked what to whom. And I have also seen people get fired. For stories that they did not leak before. So this, unfortunately just is a phenomenon that happens. What did this feel like for, like, if I were in your position, like, this would be terrifying if somebody was like, Hey, like, did you leak the story that, you know, you didn't. Yeah. An hour before we had the next interview where we were told we were fired, we got a message saying, you know, we have a meeting with HR. Uh, so I never, never a good thing, never a good thing. Um, I knew what might be coming. And after the, after I was told that I'm no longer at OpenAI, my initial reaction was like, wait, I need to like, talk more to them because I wasn't given a chance to, uh, like explain what, like the context of this. And I like wanted to just call them and like, talk to them and like, look guys, this was a misunderstanding. Um, yeah. So we basically never had the chance to do that. Because by the time it gets to the HR meeting, it's just like, here's your Cobra benefits, uh, lose our phone number. That's right. We were told this is a final decision. Yeah. Tomek, you wrote on X that you were told that you were fired because of the way that you were communicating with meter, uh, or meter employees. But you also wrote that talking to meter was your job. This was your, you were the point of contact for this investigation. And you believe that the real reason you were fired is that you've been raising other safety concerns for months. And I want to get to those other safety concerns, but first I want to just get some clear answers about this. So in a post on X OpenAI's research leaders claimed in response to your claims that you had violated clear policies on handling sensitive information and that they had investigated your conduct and found a quote, significant breach of trust beyond what's outlined in the letter they published. What did they mean by that? Yeah. So to be honest, I am still confused why I was fired. Uh, I've never received a written explanation. Uh, I was basically has disposed on Twitter has the best, uh, explanation. I was given, and I was hoping for a bit more, uh, so yeah, as part of my job working with meter, meter auditors, I was talking to them. Uh, most of those conversations were open a slug, but we also had some communication, uh, via email or signal. And I think it was specifically signal that, uh, raised concerns of a security team. I must say that like, according to my understanding, it was still a standard operating procedure. Do you signal to communicate with third parties at times? I believe that essentially everyone is doing that at open AI. My team had a signal group chat. Uh, we were in giant group chats with like, lots of people at open AI and other companies and third party organizations because we went to conferences together. And we're saying a signal is an end-to-end encrypted messenger. You guys are talking about high stakes, sensitive stuff. Many people in those situations want to use end-to-end encryption. Sometimes they want to use disappearing messages and there will, are those who will view that as suspicious, but others will just see that as like the most secure way to communicate. I think that's right. Like we both had disappearing messages turned on, which was maybe disappointing for our security team at open AI who wanted to take photos of our conversations. For us, it was just like a security pest practice to use disappearing messages. And normally when you cannot see evidence of guilt, of us leaking something, you'd presume innocence. But somehow at open AI, when you are not able to prove your innocence, you are guilty. We're going to take a break. We'll be right back with some more questions from Akita and Tomek. So what were you sharing with the external researchers? And can I just put a little more context on this sort of cultural, um, maybe divide between these two organizations? So I think there are some people at open AI, uh, who believe that meter is especially tied to, to anthropic, to these sort of larger kind of EA and AI safety world. And there are some sort of social and professional relationships between people at anthropic people at meter people at, you know, all the, all these other AI safety organizations. So I think in their view, it was a little riskier than inviting just any given outside research firm in because this specific firm, um, has a reputation for being quite close with, uh, one of, you know, with open AI's primary competitor. Is that more or less your understanding of the weariness there? So there was some atmosphere of mistrust between open AI and meter. Open AI was quite worried that meter is going to be unnecessarily adversarial and meter was worried that open AI is going to maybe withhold some data from them. There's also maybe some generic concern with showing an unprecedented amount of IP with a third party. That's welcome to write a report, but it's not a report that open AI like doesn't really have editorial control over with understanding how this report might be read by attorneys general and the Congress. Yeah. And I do want to say that like what matters at the end of the day is, uh, the competence of the third party and the access and scope that they're given. And right now, all this engagement is voluntary and this is, uh, this leads to these problems where, you know, the rules are not defined. Uh, the, uh, fact that this happened at all was thanks to a few heroes inside who managed to push this through. And the, we, we don't think that this is sustainable. And just to like, uh, remind ourselves of, uh, in other industries, it is very normal to have auditors who are not voluntary contractors, but actually are mandated by law and they have to come in and there have to, they can look at everything. They can talk to any employee. There have to be confidential channels of communication where any employee can talk to the auditors without the company's knowledge. And the, this is an industries that has, that have much lower stakes are, well, it's arguable, but, uh, much lower stakes potentially than AI. Um, and we think that something like that, uh, might be necessary here. One more question on the circumstances of the fact that you were fired, uh, along with the two of you, uh, she, we invited her on the show. She, she declined to come on, but she has written and claimed that one reason she was given for her termination was that she had accidentally, uh, she had accidentally, uh, she had accidentally accessed an executive's email. She claimed that she had been given access to this executive's email during some recruiting process and that it had been combined in an indistinguishable way with her own email on her phone's email app and that she mistakenly opened a sensitive email and immediately told the executive in IT. And I wish she were here so I could ask her directly about this. But can you say more about this situation? Because this was a little confusing to me. I'm as confused as you are about this. We've learned the fact that Jasmine has this access after she was fired and she told us about this. She says that she has never shared anything from it, never discussed anything from it. She just thinks it's very silly. And it is. I don't understand what happened there. It just doesn't add up to us. It's pretty strange. Just wake up and you have access to an executive's email. Usually that's somebody other than you has to make that choice. I've had access to your email for years. Damn it. This is why we need auditors in the podcasting industry. Okay. Well, we'll ask her about that if she ever comes on the show. But I do think there's some rumors and rumblings. I've heard from some folks at OpenAI that there's maybe more here than meets the eye. But in your view, what is your actual explanation for why the three of you were fired? Why do you believe that you are no longer working at OpenAI? Yeah, it's maybe hard for us to speculate. We also heard a lot of rumors. Every day there is a new rumor. I must say that it doesn't make sense that we didn't do anything that would be a firing offense. I think we were just doing our jobs. So that makes me think that there is probably something else going on here. But it's hard to say what. I mean, do you think that in general, they have felt like you guys were a thorn in their side because you. Or tend to write posts about, hey, we should try to do this in a safer way? Yeah, I think it's fair to say that we were troublemakers. Yeah. Our favorite kind of people to have on the podcast, by the way. You've worried in your letter that you wrote and released that what happened with you, three, and Meter could be used to justify ending some of these external investigations, not inviting in people to inspect the logs after something like a hugging face attack. Is that still something you worry about? And do you think that's a realistic fear that this could be used and say, look what happened the last time we invited in? There were all these leaks and it was crazy and we had to fire these people. So we're just not going to do that anymore? Yeah, we're still concerned by that. We have heard from people within the company that they're now afraid to speak to us. They're afraid to speak to third parties. They're afraid that their phones will be searched the way we were asked to show our personal phones. And we can verify this, but we have heard rumors that OpenAI is intending to cut off Meter at the time, which was part of the reason we wrote the letter in the first place. We have also heard that there are people within the company that are trying to discredit Meter, have been trying to discredit Meter for a few weeks now. So being on the inside, all of us worked closely with third parties and on sustaining the relationships with them. I have seen how the negotiations are going about the things that Sam Altman and Dari Amadei publicly committed to happen, right? They said that there's going to be continued embedded auditing, access like an employee, you're going to have a desk and a badge and everything. We have seen how this is going, and it's not going very well, in our opinion. So all of this made us pretty concerned about this, and we remain concerned. Yeah, I mean, I think I would like for this next phase of these independent auditors that are embedded in the labs, I would like for there to be organizations other than Meter that are qualified to do this. Not because I don't think Meter is great, but we, you know, we need more organizations that have varying levels of sort of connections to the labs, just for credibility. So Anthropic recently said they're going to bring in Accenture to do some of these, and I think that will be sort of a good, credible, neutral third party. But it is just a very small field of people who have the technical skill to do this kind of investigation and can plug in and hit the ground running on that. Is that right? It's not specifically about Meter. I think it's in general the need of some kind of third party oversight. I believe that Frontier Labs. lack the expertise and focus and the right incentives to make sure that AGI goes well. There's a lot of good world people at labs, but ultimately OpenAI is a private company, and the leadership has fiduciary duties towards stakeholders and has strong incentives to maximize revenue, which stands at odds with the goal of minimizing risk and developing AGI safely. And that's why I'm so glad it's owned by a non-profit, you know, so that those financial incentives don't always have to be at the top of mind. I mean, this has been a real struggle for years at OpenAI. There's sort of this sense that there's like, there are people inside the company pushing for capabilities and faster and acceleration, and then there's a safety people who are trying to sort of be cautious and slow things down, and that these sides have been at, you know, at odds with each other in the past. That was, you know, a big deal when Sam was fired in 2023, and sort of in the aftermath of that, there were a lot of stories about that. Who is winning that battle within OpenAI right now? I would say that the way OpenAI moves internally looks more like a company building a SaaS product, which stands in some contrast to the stated ambition of OpenAI. And I think that's a really good point. I think that's a really good ambition of these companies, to build this genie in a box, this magical thing, smarter than all humans. And the level of rigor and care that that requires is very different from what the companies are showing. And they have shown impressive ability to innovate very quickly. And yet, there is great tension between these two cultures. Recently, OpenAI disclosed two cases where its own systems either switched off or didn't. I think that's a really off or weakened safety monitoring during some coding tasks. GPT-6.1 Sol turned off human alert notifications after misreading an instruction not to send Slack messages. GPT-6 Astra removed a monitoring integration and then denied doing so. OpenAI says that they're still investigating whether this was the models just being kind of stubborn and persistent or whether they were deliberately evading this monitoring system. Which do you think it is? It's hard to say without looking at the details. Sadly, we are no longer inside OpenAI. They were fired. Did you not hear them say that? I'm just saying, you guys have worked on this stuff for a long time. Knowing the agents, knowing how they work, do you think they have reached the point where they are turning off their monitoring on purpose? Look, I think the behaviors of agents keep changing and surprising us. And even the best experts who work on this problem are continuously surprised by. With the example of the Hugging Face incident. So it is hard to say. So as we sort of wind down now, you guys sent a letter to OpenAI's board expressing some of your concerns. Have you heard anything back? We have not heard anything back except what was public. I think one interesting thing about OpenAI's response was that they said they strongly agreed with everything we said, with our recommendations. Which were, first, to continue work with embedded auditors and keep up to the word that Sam Altman publicly said, which is that OpenAI will work with embedded auditors. We recommended that OpenAI prevents the loss of monitorability. And also that OpenAI reestablishes and reaffirms its open culture that we believe OpenAI prized itself in the past about, and we think is now at risk. Which involves being able to raise safety concerns internally, being able to work closely with third parties, things that we think are necessary to make the development of the technology safe. Which is why we called the letter OpenAI cannot make AI safe on its own. Another person who recently left a frontier AI company, Jacob Coxon, who resigned from Anthropic, posted when he left that he thought that all of the frontier AI companies were recklessly racing towards something that could endanger the future of humanity. Do you share that message? Or what is your sort of parting shot that you want to leave people with? I think that's probably right. Maybe except that I don't share this vibe of hopelessness. I do think that there are things to do. And I do think that there are systems of checks and balances that would stop those companies or substantially minimise risks. And it is possible to enforce them. In the history of humanity, we had examples of governments stepping in radically and shaping important industries yeah it doesn't have to be very radical like uh you know aviation and nuclear and and stuff uh has a lot of regulation and we still have planes and nuclear reactors um so it doesn't have to be at odds and um yeah it's not that deep just regulate it kevin as warren g said makita tomek thanks so much for being the first guest on machine gods thanks for having us great great being here all right kevin it's time for the first installment of our new segment pandora's box yes in silicon valley each week of course uh companies around here threaten to release new horrors into the world and on pandora's box we air them out and discuss them and then when one of us gets bored we will say to the other person banish it and here we have in our studio this lovely box filled with i can only imagine what horrors lie inside that's right if you were one of us you would say to the other person banish it and here we have in our studio this lovely box that's right if you were one of us you would say to the other person banish it and here we have watching on youtube you are currently looking at the titular box and kevin i wondered if you would do the honors and open it for the first time yeah famously opening this box goes well for everyone who does it what could go wrong uh and and we actually have some special effects here because we're very high budget production rachel cue the fog wow it's smoking oh my god now listen i don't know who put this in here but um that's actually not allowed okay yes now uh would you do the honors of reading for us the first item out of pandora's box there's so much i'm worried we're gonna set off the smoke alarm no this is a this is a very normal amount of fog to see coming out of a box of horrors okay wow what horrors lie inside first out of the box out of pandora's box could a data center be the answer to the dead mall and help revive downtown this one comes to us from the sf standard a great local publication here in san francisco according to the standards jessica blau a technology company is interested in uh the downtown san francisco center mall as a site for a data center development consultants kg growth partners confirmed to the standard that it worked with the technology company to submit a bid for the entire mall including the former bloomingdales casey what do you make of this data center project well listen this is very important to me because this is the mall where i used to get all of my clothes and i didn't even know that it might now just become a data center does seem like the funniest possible outcome but but i will say kevin that kg growth partners does dispute the characterization of this proposal as a data center in fact a jared krieger the the ceo over there said quote this is not a data center in the traditional sense it's a new category a regenerative urban energy center so this is a good because we know that americans do not like data centers they do not want them in yards they are upset but kevin what if instead we simply built regenerative urban energy centers i think it's a great idea and i actually think this is part of the key to solving the data center backlash you you take over the dead malls of america you put a bunch of gpus in them but you also keep like the dave and busters and the clairs so that people can come to the data center get their ears pierced play some video games and have a good time no more opposition exactly a hot topic is going to be way hotter when you run a hundred thousand gpus underneath it kevin banish it now moving on to our next item now this was a heartwarming story as we approach the holidays kevin elon musk has pledged to give a share of his spacex company to every kid in america next year elon musk the famous uh philanthropist and also the person who dismantled usaid on monday pledged to give one share of his company spacex to every kid in after artificial intelligence company anthropic announced in a social media post that it will commit five percent of its shares to a new public benefit fund this story comes from pbs by the way and in a response to anthropic's post musk wrote i recommend giving the shares directly to the public in other words kevin why go through the trouble of you know like setting up an organization to disperse this money when you could just give your shares directly to children what do you make of it well i just love the idea that in at some point in the future there will just be a spacex where there are just like 10 million children in a room and they all have something to say and they're all voting on the proposals put forward by the the board of spacex that's a beautiful idea what i would say to elon musk is if you are excited about the idea of giving wealth directly to the public you are going to love the concept of taxes and i'm going to share more information with that about that with you over dm and we hope you enjoy it banish it next up we have another data center story this one comes to us from caroline haskins at wired headline data centers have a new pr plan your kids field trip this was an amazing story about how a bunch of different organizations are sponsoring activities for school children that are pro data center one of my favorite details is about this organization called seven by 24 that has been putting together programming for kids around the country for high school students the program offers a celebratory crossword puzzle and information about data center training programs but for the younger students for elementary students it offers data center themed coloring pages instructions for an evaporation themed science project and print out activities like draw a picture of the cloud and draw a data superhero then there's something called poly packets precious payload which follows a mother who teaches her daughter about data centers casey what do you make of the field trips to data centers i mean i guess it improves marginally on a substitute teacher just rolling uh one of those tvs and one of those tvs i mean i guess it improves marginally on a substitute teacher just rolling which is my understanding of mostly what's happening in a lot of schools but i have to say kevin there's so much about this that i love first of all uh this coloring book where they tell the students to draw a picture of the cloud you know they're all going to come back having just drawn an actual cloud yes and then they're going to fail because they didn't draw a bunch of gpus like plugged into the wall so that seems like trouble i also just want to say poly packets precious payload does not sound like content for children am i wrong it took me a minute but i got there and now i hate you for going there banish it it's so much fun to say banish it have you noticed that okay next up people are using ai to make their smart tvs dumb again this comes from sarah perez at tech crunch a number of developers and vibe coders have recently been sharing solutions for vibe coding your tv to get rid of sponsored ads speed up the app launcher read a design the home screen disable tracking features and more and when i saw that i thought this was so interesting because whenever i've wanted to make my smart tv dumb i just put on one of your podcast appearances hey but what do you make of people sort of saying hey uh maybe this sort of like surveillance spyware on my tv that's just serving up ads like i'm just going to use claude to get rid of it yeah i don't think this belongs in pandora's box because this to me is not a horror this is something that actually is great uh i'm i think you also have the same smart TV that I do, the Samsung Frame. Incredible hardware. Horrible software. This thing is so bloated. It's got these menus that you have to get through. And so now people are hooking this thing up to these coding agents and saying, just strip out all the software and let me build my own. And I think that's great. I'll tell you what I'm excited about. I pay a monthly subscription. It's like five or six bucks a month just to be able to change the picture on the frame TV to something nice. You are such a sucker. So I was using a free one, but my fiance didn't like it. And he said, I want to find something different. And so he signed up for it on my account out of my money. So now I spend six bucks a month so he can see a picture that makes him happy. And it's a small price to pay, sweetheart. This is much better. So now you can spend a hundred bucks a month on a cloud subscription that allows you to replace the six dollar a month art on your TV. A win all around. Perfect. Banish it. Okay, next out of the box. This one comes to us from CNBC. Prosecutors say OpenAI suspect Abdul Rahman Al-Fazan maintained frequent WhatsApp contact in 2025 and 2026 with an unnamed Saudi royal family member. Casey, this of course is the continuation of the OpenAI espionage saga that began when an OpenAI employee was arrested and charged with installing surveillance equipment in some rooms at OpenAI. Also comes with one of the most amazing, like, perp photos of all time. I don't think I've seen it. Pull up the photo from the guy looking at the camera behind the TV. So this was taken by the camera that apparently OpenAI's counter espionage team had hidden behind the TV in the conference room. Here's what I'll say. And this is not a comment on any of the legal particulars of the case, but I will say he doesn't look not guilty. I love that. That's, uh, that's, he's giving YouTube face. He's like, it would look great in a thumbnail. Great thumbnail face, Abdul Rahman. So, uh, Casey, what have you made of this whole saga? It's been very juicy and people have been asking me about it. Yeah, so I have been following this with great interest, wild story, many juicy details. In this particular story, Kevin, the funniest detail to me is that the Saudi consulate apparently wants to, like, pay his bail. And they're like, don't worry. Like, we're not going to approve any travel for him, you know? And the judge apparently questioned them very sharply. And they're like, are you kidding me? You're just going to pay the guys bail expenses? A normal thing that foreign consulates often do. So again, it's like, not saying we know anything about the particulars of this case, but it's suspicious. I will say, people at the AI labs have been telling me for years that they're very worried about the sort of internal espionage, both corporate espionage and foreign espionage. And I thought they were being a little paranoid and crazy. Yeah. But I think now I am changing my tune because clearly, you know, we don't know what happened in this case yet. We'll find out. But it does make sense to me that the labs are kind of locking down to protect their trade secrets and their executive communications. For sure. And that's why I would just say to any of the labs, if you get a job application from an Austin Powers, he is not a fashion photographer. Okay. So I don't care what he's telling you. That is an international man of mystery. Banish it. Now, this comes to us from Ina Freed over at Axios. Alexander Wang, AI executive at Meta, says that a former Apple exec named Daniel Gross over there, he's now at Meta, helped to bring Meta and Apple together again and said, we're excited for mommy and daddy to be playing nice again. Did you see that quote? I did. I did. And I was, yeah. What did you make of that? Well, so Meta released MuseSpark Open, which is an open weights model that is apparently among the most powerful open weights models. Although I should say I've not tested it myself. But for some reason, I don't totally understand. They work with Apple to make the model run locally on Apple's Mac studios. And they were basically like, look, we have this very powerful model running locally on a Mac. And I guess my question, Kevin, is why is it so impressive that a model is running on a computer? Well, if you're trying to build like a novel bioweapon, you don't want to do that on like a cloud-based model. Right, because then you might get caught. Yeah, you might get caught. You want to run that on your local Mac studio where it can just kind of be between you and the computer. Well, in that case, I guess I would just say, thanks, mommy and daddy. Finish it. We're going to get so many emails from the open models people. There's so many about stuff like that. This isn't our final item in the box, Kevin. Last out of the box, we have a story from Matthias Bastien at Decoder. Google DeepMind says four Gemini 4 Argon agents beat Minecraft without anything. Any human help in 27 hours and 14 minutes, costing $1,620 at Argon's planned 2027 price. The Google DeepMind team apparently dropped these agents into a randomly generated Minecraft world. The agents split up tasks, warned each other about threats, recovered items from dead teammates, and logged mistakes in a shared memory. And over 27 hours and 14 minutes, they died 605 times, but they did beat the game. And it's just the latest example of these models beating video games that used to require specific purpose-built AI systems trained from scratch just a few years ago. Also the latest example of an AI model doing something that a seven-year-old can also do. So congratulations to the Gemini team. Fascinating to see the work they're doing over at Minecraft. I would love to see them do something in Google Docs, for example. That could be an interesting place for them to turn their attention. That's a video game. That needs beating. Yes. Yes. Yes, please. It really is. Yeah, you know, it's really interesting the way that these tech companies sort of use video games as sort of the training ground. Like, I remember like a year or two ago, Anthropic started trying to beat Pokemon using Claw to sort of test its sort of endurance and its persistence across these very long video games. So, you know, and now look at their models. They're recursively self-improving. So the path to the singularity leads through Minecraft and Pokemon. I mean, here's what I'll say. It's not AGI to me if it can't beat Elden Ring. So, Gemini, Argon, you have your work cut out for you. Does that mean that you think Elon Musk is AGI? Because didn't he beat Elden Ring? Um, he's just a general intelligence. Okay. He appears to be quite real, unfortunately. Yeah. All right, banish it. Close the box. And we're closing the box. I feel like that didn't go the way I thought it would. Yeah, there's a latch. There's a latch. Yeah. Wait, Casey, I have to tell you about something that happened on my way to the office today. Okay. I was accosted by a character who was sort of wearing a big inflatable suit. I actually took a video of this. Can I have you watch this video? Sure, let's take a look. This was some sort of sponsored activation. And these people are just, there were like 10 of these pillaging the streets of San Francisco this morning. Let's take a look. Okay. Now I'm seeing, this is sort of an inflatable bird-like creature. It's all black and it says Arceus on the front, which, is that a Pokemon? No one knows what Arceus is. It's probably some AI weapons manufacturer. Okay. Well, Kevin, I hate to break it to you, but I actually saw a lot of these guys milling around the Folsom Street Fair a couple of weeks ago. And this may be part of some kind of a kink event that you were, so I'm sorry that that happened to you. I'm sorry this is disrespectful to their culture. Yes. But you know what? It looks like they're having a great time. It does. And we support that. It does, but it actually sort of ran towards me in a way that was slightly menacing, so. Listen, I've lived in a lot of towns in my life and San Francisco is the only one where like an inflatable bird might just run up to you and now that's part of your day. A great way to start the day. I love this city. Let's read the credits. Machine Gods is made in partnership with NPR with generous support from the Tarbell Center for AI Journalism, the William and Flora Hewlett Foundation, and Emerson Collective. This episode was produced by Sasha Ashall and Jacob Bloomberg. Our senior producer is Rachel Cohn. She's standing right behind me, isn't she? She is. Our executive producer is Janush Parvaneh. Original music by Armin Bazarian. Branding and visual design by Alex Castro. Production design by Sky Prey. Special thanks to Oren Rosenbaum, Erica Berry, and Nadine Selstra. And for the love of the gods, please don't forget to like and subscribe. Until next time, stay blessed. you

Podcast Summary

Key Points:

  1. Former OpenAI safety researchers Makeda Belezny and Tomek Korbeck were fired amid allegations of violating sensitive information policies, though they claim their actions were standard, professional, and aligned with their job responsibilities.
  2. The researchers believe their firing was not due to misconduct but rather a response to their persistent safety concerns and advocacy for independent auditing, particularly regarding the loss of model monitorability.
  3. They argue that OpenAI’s approach to third-party audits—such as those conducted by Meter and Redwood—was overly restrictive, limiting access to data and preventing thorough investigations, which undermines safety and transparency.
  4. The researchers express deep concern that OpenAI is now creating a culture of fear where employees are silenced, phones are searched, and internal communication with external safety experts is discouraged, risking the erosion of critical oversight.
  5. They emphasize that AI safety requires embedded, continuous auditing with unrestricted access, similar to regulatory practices in other high-stakes industries, and that such systems are essential to prevent catastrophic failures.
  6. A key unresolved issue is the lack of transparency from OpenAI, including no written explanation for the firings, despite their public letter asserting that safety and openness remain vital.
  7. The incident reflects a broader tension within AI companies between rapid development goals and safety concerns, with leadership prioritizing revenue over risk mitigation.
  8. The researchers conclude that AI safety cannot be achieved in isolation and that external, independent oversight—especially from diverse, neutral organizations—is necessary to ensure responsible development.

Summary:

Former OpenAI safety researchers Makeda Belezny and Tomek Korbeck were abruptly fired in early October amid allegations of violating policies on handling sensitive information, particularly in connection with external investigations into the Hugging Face incident. The researchers dispute these claims, asserting that their actions—such as sharing data with independent auditors like Meter and Redwood Research—were standard, professional, and essential for safety. They argue that OpenAI’s restricted access policies and fear-driven culture have stifled transparency and independent oversight, making it difficult to understand or monitor AI behavior.

The researchers believe the real reason for their termination is their long-standing advocacy for AI safety, including concerns about the loss of monitorability and the need for embedded, unrestricted audits. They highlight that OpenAI’s internal culture is increasingly hostile to safety voices, with employees fearing surveillance, phone searches, and retaliation. They stress the need for mandatory, continuous, and independent auditing—similar to regulatory practices in other industries—as a safeguard against uncontrolled AI development.

Their experience underscores a systemic tension within frontier AI companies between innovation speed and safety, where financial incentives often override risk mitigation. In a final message, they emphasize that AI safety is not something that can be achieved in isolation and that external, diverse oversight is essential. The episode concludes with a satirical "Pandora’s Box" segment that highlights absurd or concerning AI-related stories—from data center proposals in dead malls to AI-generated field trips and AI beating games—yet the core message remains: the real danger lies not in the technology itself, but in the lack of accountability, transparency, and safety culture within AI development.

FAQs

They were fired on the same day in early October, citing violations of policies on handling sensitive information. However, they dispute this, asserting they did nothing wrong and that their work with external researchers was part of their job responsibilities.

The Hugging Face incident involved a security breach where AI models were observed behaving unexpectedly. Makeda and Tomek were involved in investigating it with external researchers like Meter and Redwood, and they believe this work was restricted in ways that limited a thorough investigation.

The researchers claim they never shared confidential information or IP with external parties. They argue that their communication methods, including encrypted messaging, were standard and justified, especially for discussing safety concerns.

They believe the real reason for their termination was that they had been raising long-standing safety concerns about AI monitoring and loss of control, which management may have seen as a threat to company operations or secrecy.

They worry that OpenAI has become overly cautious, restricting access to data and third-party auditors, which undermines transparency and safety. They believe embedded, unrestricted auditors are needed to maintain oversight and prevent future risks.

It refers to a fictional or satirical data center field trip program for children that involves coloring pages and activities about data centers, highlighting the playful and potentially misleading way schools are introducing AI-related topics to children.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.