Why are scientists and mathematicians mad at open AI this time? What do you tell your teenager who thinks AI is evil when you need to use it for work? How do you shut down a rogue agent swarm and are we frog and toad personified? I'm Kevin Rousse. I'm Casey Newton and from San Francisco This is machine gods in partnership with NPR the official podcast of the Singularity starts now All right Casey, well we've got a big show today lots of great listener questions that people have been sending in but first we have to talk about the science We have to talk about it Kevin. You know that was not one of my better subjects in school Well today you are gonna learn I feel confident saying more about science and math than you have ever learned during an episode of our show Well, I'm excited about this because when I saw these announcements come out I said I would really like it if Kevin explained this to me Well, I'm here for you. Thank you. So this week open AI announced three new AI-generated discoveries One each in math computer science and physics and we're gonna talk about these results But just in general, I think we should talk about why we're discussing these now because I think what is happening in math and science Is really important and it's really controversial Mathematicians and computer scientists and all manner of scientists are really mad about the way that open AI has Developed these results and the way it's releasing them. Yeah, and I sort of get it because while I know there are people at open AI Who are true believers in the science and think it's incredible that we can make these sort of breakthroughs and advance human understanding I think there's also a sense that on some fundamental level This is just a marketing and PR effort for open AI at a time when AI is really unpopular They're sort of like rifling through the cabinets trying to find some discoveries and show them to the world say Look at these discoveries whereas if you've been researching these problems your entire life and this company just kind of you know Stops in out of nowhere with a solution. It is just going to feel kind of annoying totally so This all kicked off back in early September when open AI released a solution to the the Navier Stokes problem This is one of these millennium prize problems that Mathematicians had been puzzling over for decades That led to some speculation that they had been like snooping on human mathematicians who had been using their models to try to do similar things Shortly after that a group of 28 fields metalists. These are the highest honors in mathematics Signed a declaration titled a severe misalignment of AI in mathematics and they said that because they know that that hurts an AI labs Companies feelings when you say that they're misaligned exactly. Yeah, so open AI tried to respond to these concerns by Basically agreeing to work with this independent group of experts to sort of figure out how to release these future results and Then in early October open AI dumped 722 AI written math manuscripts on GitHub Which set off an even bigger wave of backlash among mathematicians. They did not like that at all No, it was like this computer scientist Ryan William said it was almost like a DDoS attack on the profession You know, it's like oh you were trying to solve a math problem. Well, we solve 700 by you know, that didn't that right didn't feel good And so all that led up to this week when open AI announced these three latest results and we'll talk about them just in a moment But also talked about the new process that it is developed that is Designed to make people less mad at them. Yeah, they clearly have taken the note and this is one of the most interesting parts of this story to me Is that they went out to these very different disciplines and essentially said how would you like us to work with you and the disciplines all gave very different answers? Yes, so let's talk about these three results and I'm just gonna try to speedrun them because I don't want to bore you by Explaining things that you already know from your advanced studies in science and math. I think also I have a lot of things to do today So we'll be quick here. So the first I would say most significant result from this is the falsification of something called the exponential time hypothesis This is a significant result in computer science that has implications for cryptography But this is basically a shortcut to solving a very hard puzzles That previously scientists thought could only be solved through the exponential application of computing power got it Problem number two. This one is called the cachea conjecture Which is a decades old problem that open AI claims to have solved in every dimension Not just two dimensions, not just three dimensions, not just four dimensions, but an infinite number of dimensions And it has to do with a needle and the sort of space that it needs to rotate through in order to point in any direction And solving this problem was like fighting a needle and some kind of infinite space Is what I bet olds Yes, a many dimensional haste Yeah, I think it is this is one of those kind of pure Professional math and science puzzles. Yeah, that has implications for maybe other kinds of problems But yeah, probably not going to matter to the average person But there are people who have been working on this for decades yes and to them we say thank you for your service And now maybe you can move on to something else Start a podcast. Start a podcast Okay, Casey the third problem comes to us from the world of physics And it is related to something called QCD which of course stands for quantum chromodynamics Now I know you're wondering what is QCD? And Casey it is the theory of the strong force The glue that holds quarks together inside protons and neutrons This is a big deal in the world of physics. That sounds you know I misheard you I thought you'd said quantum homo dynamics which relates to how many gay guys you could fit at a pride party In a small apartment But that I've actually already solved that one. So tell me about this one Yes, this is different But suffice it to say that for decades physicists have had this theory that explains one of the fundamental forces holding matter together Love they use it constantly. It's not love. Okay. It's quantum chromodynamics But they've never actually had a fully rigorous mathematical construction of that theory And open AI claims that they have figured out how to construct it So they've constructed a theory for how the what what is holding together now exactly the quarks Yes, the quarks the court not the quarks. It sounds like you're saying quarks. No quarks. The quarks. Yes. Yeah. Okay. Very different But this is a big deal if you're a physicist and And as a result physicists are very excited Mad skeptical of this this one in particular got it okay Well, so why don't we then move on to the question of how did open AI work with these three different disciplines? And how did that differ across the groups? So open AI after all the mathematicians and scientists got mad at them earlier this year They basically said okay you guys tell us how you want us to release these results from our internal models So they consulted with experts in each of these three fields and basically said you come up with a strategy for how you want us to Release this into the world they they sometimes call these people shepherds Because they're sort of like the experts who are kind of guiding the AI generated solutions into the world Got it. So each field gave a slightly different answer the mathematicians they talked to said Release this AI's output and let researchers organize themselves basically just drop it on the world and we'll figure out what to do with it Yeah, strictly by the numbers approach I would say the computer science group asked for everything to be public But also for open AI to fund a conference to explain the results and I have to say This is where the computer scientist really maugged the mathematicians because if you can get a free conference out of it You're coming out ahead. So take a note mathematicians exactly the physicists were the most resistant to this whole idea Well, there's a lot of strong forces in physics. That's true. They're holding the quarts together And the physicist group encouraged open AI to not publish the AI's right up and to basically let them the human scientists the human physicists write it up themselves. Yeah, they said we'll take this one. Thank you. Right. Yeah, so open AI Importantly said that it does not seek academic credit for the discoveries produced by their models Right. It doesn't seek academic credit It just wants to be the one to announce all of this to the world in their own blog post that they promote on all of their social media channels But please don't give them the credit. Yeah, who little all me So this has been criticized by a lot of academics to sort of hand pick their shepherds their experts from these various fields Open AI does concede for its part that it wants to come up with a better method But it's basically like how can we do this? How can we release these Results into the world in a way that will make the fewest people matter to us. Yeah, it's okay So here's where I will you know show some grace to open AI. This is a very strange a new situation Yes, right your your software program has solved some decades old problems in science and you now need to invent a way To bring that into the world Hopefully in a way that feels productive for you know all of the experts in this field and and does not destroy the field Because this is actually one of the big fears among the mathematicians in particular Kevin is that by sort of dumping 100 solutions on the entire field at once you actually a kind of are destroying the field right There's a lot of people who are planning to spend the next 20 30 years the rest of their careers working on these problems Now all of a sudden maybe they can't do that anymore So you do need to have a kind of approach here And I have to say the one that open AI took this time around it seems reasonable to me like what do you think? Yeah, I think it's also it's reasonable I think they're obviously like they're there no perfect solutions here like this is going to well There was a perfect solution to the Kaka conjecture That's true, but I think they're. they are trying to do the best they can to sort of get the blessing of the leading experts in these fields before essentially nullifying a lot of the work that their students and colleagues have been doing for many years. You know, you've seen some people saying, "Well, we don't consult the horseless carriage drivers before we roll out the automobile. We just roll out the automobile." And to that, I say, "Good luck with that, and I'm not shocked that your polling is below ice." And to say, I think that that sort of arrogance is probably not going to serve these companies well, like as an industry, like on one hand, like, of course, that's correct. You don't legally have to consult with them, but it might go better for you if you did. Yep. So, we should also note that hours after OpenAI announced these three findings, Anthropic also published similar discoveries. They claim to have also refuted the exponential time hypothesis. But can you see, like, why are the labs kind of doing this? Why are they sort of tripping over themselves to release these massive troves of solutions to unsolved problems in science and math? Is it just PR and marketing or is there something else going on? I think on one hand, there are people at these labs who are scientists, and they think it's really cool. And one of the whole reasons why they got into the superintelligence business was that they wanted to advance human understanding, and they wanted to promote new discoveries that would be helpful to human beings. And they view many of these discoveries arcane as they are as steps on the road to getting us there. So I think that that is, like I sincerely held belief at the labs. At the same time, Kevin, you cannot ignore the timing. As you just said, AI is not polling well. People hate it. And a question that you just hear over and over again on social media is, why are you guys building this stuff? If it's so dangerous, why don't you just stop? You're telling me it's going to put me out of a job. Why am I supposed to get excited about this? And that's where the Kaka conjecture comes in, Kevin. If you go to a rural diner somewhere in the rust belt and you sit down next to a voter and you say, the Kaka conjecture has been solved by GPT-6, well, maybe they're going to be singing a different tune the next time a data center gets proposed in their neighborhood. Don't you think? I think that's right. Yeah. I think that's the theory of change here. Like, I want to actually posit a slightly different theory, which is that I think the people at these labs working on these scientific problems just like are intoxicated by this feeling of mowing down the hardest problems in a field. I think for a lot of them, they grew up being obsessed with math and science. Maybe some of them even struggled with some of these problems in their own lives pre-open AI or anthropic. Maybe they studied them in school and it was sort of this was the highest mountain in that field. And so I think it is thrilling for them to watch their systems get so good at problem-solving and creative thinking that they can just go climb those mountains very, very easily. Absolutely. And for how many years now have we been reading skeptics saying essentially, wake me up when this thing creates new knowledge, right? Like this was a sort of like big talking point among the haters was well, this statistical next token prediction machine can never get you beyond the median of human understanding. And yet now here we are in this world where these systems have been able to make these novel connections that are solving new problems. Are there problems in these fields that basically can be brute-forced or verified more easily than big problems in other fields? And does that mean that like even if AI solves math, say, which is still not done, but it's a getting closer. Maybe that doesn't translate to overall gains and capabilities in other areas. What do you think happens next year? Like what are you looking for? Do you think that the labs will reach some sort of stable agreement with all of academia as it maybe does start to mow down these problems? Or does this become some sort of like entrenched, like culture war between major institutions and our society? I think it's going to be a culture war for sure. I mean, one thing that I've been struck by is just like how emotional the reaction from the people in these fields is there was one researcher a field's medalist who said that he had had cried when open AI had dumped this trove of solutions like not happy tears like cries of anguish because he just saw like the future of his field kind of going up in smoke. And I think that's going to be the reaction that a lot of people in these fields have to the obsolescence of their work or at least the to the possibility that like all of the most important discoveries in math and science and physics and all these disciplines will be made by AI for the foreseeable future. It's a moment of real introspection and pain for a lot of them. And so I think you're already starting to see like mathematicians and scientists kind of polarize. There are the people in these fields who really like the new AI tools and really think that they could sort of plow through a bunch of important problems with them. And then there are the ones who are like not like this. Like this is not how we want to work with computers to solve problems in our field. What do you think? I'm hugely sympathetic to these folks. You know, I sometimes say that as a journalist, I feel like the internet has been trying to kill me for my entire career, right? I started out working in print newspapers, the internet disrupted that. Then I started to work for websites, social media disrupted that. Then I started to work in newsletters. Then AI started coming for that. And so I have just sort of made my peace with the fact that it seems like I am always trying to escape from a giant boulder that is like coming down the hill trying to run me over. And I think in some ways, it has made me numb to others having this similar experience of, oh, wow, this thing that I thought I was going to spend 20 or 30 years doing. Like maybe it's not going to be possible anymore. So I actually do ache for the people that are having this experience because it really sucks. And so far, governments have had no answer for these people. The labs sort of barely have answers for these folks. So I think it's going to be a big discussion in the years to come. It reminds me a little bit of, there was this sort of cultural backlash to factory automation in the 20th century. And even for the workers in the factories who, you know, who had previously spent their careers like lugging pieces of steel around and like suffering from injuries and like their jobs on paper got a lot better because all of a sudden, like they were working in a safer environment, doing less physically demanding labor. And a lot of them hated it even so because they were like, that was part of my identity. Like lugging the pieces of steel around the yard was part of how I like felt good about myself as a human. And so I think this is coming now for these more academic white collar disciplines. And I think it's going to be really painful for a lot of people. And I also hope that it provides a chance to for the AI companies to reflect on the fact that this is going to be an emotional reaction from people. This is not just cope. This is not just people like who don't want to be replaced, who are dinosaurs. This is I think a real and valid reaction on an emotional level to something very disruptive. Yeah. You were messing with the livelihoods of millions and eventually tens of millions and maybe even billions of people. Yeah. You should prepare for a very strong and justified reaction. Well Casey, it's time to hear from our beloved audience. They have been sending in so many great questions. We are going to call this segment humans in the loop. Love this Kevin. That was your idea. And I think it is the perfect name because we truly want humans to be in the loop of machine guts. This is a show about technology. It's a show about AI. But most importantly, it is a show about how those things affect real people. And so we want to bring in as many listener voices as we can to help give us a sense of what's on your mind? Yeah. This question comes to us from Lee Ratcliffe. Hello beautiful machine gods. Love you both. So I'm in a sticky situation. On the one hand, my work is very eager for me to use AI and demonstrate its efficiencies that we gain with it. But on the other hand, my 16 year old daughter is completely opposed to AI. She makes great arguments why I shouldn't use it like the environmental damage, how it's all funded by morally unethical billionaires, how it's potentially taking away her future jobs, how it's rotting away everyone's brains, and how it's stealing everything from rare books to all artistic works. So I guess my question to you is how do I navigate this really tricky space? What do you think I should do? Well, I'm glad we started with an easy one, Kevin. What's your answer? Look, this is actually fairly common. I think I've heard from a number of parents of teenagers who have exposed this rift. I think there is something to this generational gap where people are aged, people midway through their careers. Are we excited about AI? And people who are young, who are maybe in high school or college, I think tend to be much more pessimistic about it. I have to say, I'm very sympathetic to the teenager in this situation, right? If I were 16 right now, staring down a future where everything I've been working toward, my whole life is suddenly at risk because maybe AI is going to be able to do everything. Of course, I wouldn't like that. [BLANK_AUDIO]
I think if I were the parent here, I might try to have a conversation about the fact that many of the challenges that she's identifying cannot really be addressed at the individual level, right? If you're an individual worker and you're working on a small team and the boss says, hey, I need you to get more done and AI is gonna help you get more done, it probably makes the most sense for your family to just do that. But if you're really worried about what AI means for like the future of labor, if you're really worried about the concentration of power and those morally dubious oligarchs that seem like they're amassing more and more control, those are things that you can get some political leverage over, you can do some organizing, you can speak at the city council meeting, you can call your congressperson, those are really great things for a young person to get involved with. So I think often when we look at societal problems understandably, we always wanna ask ourselves first, what can I do about this, right? I don't wanna be a part of the problem, but I think it's just as important to ask, how can we create a collective solution to this and how can we maybe get involved in some of those efforts? So if I were the parent here, I'd be trying to channel my daughter's frustrations in that direction. - And I would say, I would also try to impress upon the daughter or the teenager in this situation like the usefulness of understanding the tools or using them, even if the goal is to oppose them. I think you have to like know your enemy a little bit and I think if you're a person who thinks that these systems are immoral or destructive or shouldn't be used, like it is also helpful to you to understand the tools and how other people are using them, what are the real facts about the environmental impacts because I think that just makes you a more effective advocate. - All right, next question. This one comes to us from Arthur. He is part of an AI dad chat that I am a part of, a group text with many, many dads who are sorting through these questions. - So this is a known human to you. - This is a known human to me and he asks, what should America's AI school curriculum look like and how young should it start, Casey? - Another simple question best answered in a 30 second clip by two people with no experience working in the education system. Arthur, we're so glad you asked. I'm just going to acknowledge that Kevin and I probably do not have sufficient expertise to give this one a proper answer, but I reflected on it a bit and I just realized that when it would come to my own children, I don't really want them to have much exposure to today's AI tools before they get to, let's say, middle school. I want them to develop a solid foundation in critical thinking skills. I want them to learn to read and write and do math kind of the old fashioned way to build that foundation, but then once they get into middle school, I'm actually more comfortable with them doing some experiments and having AI systems do some writing for them, you know, making the slide decks, learning how to use spreadsheets and that sort of thing. So that's where I kind of start from, Kevin, what do you think? - I think that there should be some modern AI version of like computer lab. - Do you remember computer lab? - Love computer lab. - Favorite part of the day when I was a kid, your teacher would take you into this room with a bunch of crappy computers in it and you would pretend to learn how to program while actually playing Oregon Trail. This is every 90s child remembers the computer lab experience. I think there should be some version of this for AI and I would even be comfortable starting in elementary school. I would like for students to be understanding these programs at a very young age, but in an environment where they can be supervised and monitored and taught by their teachers. Like I don't think we need to be turning over the whole curriculum or the whole classroom to AI. I think it's like way too early for that. But I would like for there to be some classroom instruction happening the way that we were taught how like make a snake game using a simple programming language. I think this should be part of their education. - I mean, I think I agree with you insofar as I do think the digital literacy does need to be part of early childhood education. It's too powerful, it can be too dangerous, not to teach kids about it. I also want to address one criticism. We might get for this sort of answer, which is, you know, I'm sure there will be people listening who will think, well, I don't want my child exposed to AI at all and to some extent I can understand that position. But then I just started to think like, well, are you gonna prevent your kid from googling until they turn 18? Because look at Google search results right now. They have AI all over them. So I think even if you want to be the sort of parent that keeps your child away from it entirely, it's just going to be really, really hard. And in that world, I think, you know, your best bet is to make them literate. All right, next question. All right, Kevin, this next one comes from Erin Fraser who writes, I have a friend who repeatedly joked about tweaking his wife's chat GBT user profile without her permission or request to encourage her to have sex with him and also saw a joke tweet about doing so. Then I saw the movie You Can See Everything yesterday and one through line is, I know what my spouse would want. Anyway, question is, under what circumstances is it ethical, permissible, or a good idea to tweak someone's settings in chat or clawed like if your spouse weaponizes incompetence, is it okay to tell it to explain it like I'm five? So there is no excuse. Is it always bad and sort of like, "Iky, Kevin, what do you think?" I think it is always. I cannot think of a more destructive thing for the relationship than to secretly and subversively change your spouses, chat GBT or clawed settings to subtly sway their behavior. Yeah, what is more disturbing here? The idea that some people would think to do this or the realization that it might actually be effective. I mean, look, here, I can think of one scenario where this might be morally accepted. All right, let's hear it. Your spouse is undergoing a form of severe AI psychosis. They are being told that they have discovered a tear in the quantum fabric underlying nature. They've been told that they just solved the KKN conjecture. Yeah. And you are kind of trying to make their model less sycophantic so that they kind of come back to reality. That is the kind of intervention that I could see in a very extreme situation supporting. But other than that, if it's not like life or death, I think this is just like gross and icky. Yeah, completely agree. The only other thing I would say was I was reading this research that anthropic put out this week. And basically they ran this experiment where they took some analysts and they gave them some sort of problem and some were allowed to use a clawed for help and some were not. And unbeknownst to these analysts, anthropic had essentially manipulated the advice that Claude was giving it to see how the researchers would react. And what they found was that analysts accepted Claude's advice about 86% of the time. Like they just instinctively did the thing, Claude it. And these were professional analysts who were being given a problem in their domain. So in other words, these guys sort of knew what they were doing. Now, the research goes on to say like, Claude was like giving good advice. It wasn't a case where like these analysts were necessarily being led astray, but it was a clear demonstration of what you call machine-dripped, which is just an algorithm kind of tells you to do something. And over time, you just sort of feel yourself being dragged along. So this is why it is very dangerous to try to manipulate people through their chatshipity, custom instructions or whatever else. These systems are just very persuasive. And you know, that just represents another set of dangers that I'm sure we'll talk about another time. - It'll be a very, very busy decade for marriage counselors. - Okay, next question, Ethan Russell writes with a Halloween costume suggestion for us. He said he was watching today's episode and two things hit him. Quote, "Casey has impeccable taste in Portuguese flannel and you two are frog and toad personified." I hope in prayer you take that as a compliment Ethan writes and if you're looking for Halloween costume ideas, I think this one's a winner. Casey, first of all, was it actually Portuguese flannel that you were wearing on the last episode? - I think it actually was. It was like a very good eye for Ethan. - Yeah. - And now what about this allegation that we are frog and toad personified? - So, frog and toad, of course, the classic series of children's book written by the great Arnold Lobel. This allegation quite pleased me. I think we might want to explain to people like what are the sort of archetypes that frog and toad embody? - Yeah, so I've been reading a lot of frog and toad to my son, they still hold up great books. And of the two characters, frog is the taller, greener one and toad is the like, squatter, brown one. I will not take this as any kind of referendum on our physical appearance. I'm gonna take this as like a vibe that Ethan is picking up. So, frog is the kind of more cheerful, adventurous, emotionally steady, optimistic one. Toad is always kind of like worrying about stuff and is a little stubborn, a little skeptical, easily exasperated. So, Casey, would you identify as the calmer, more optimistic one? And maybe I would be the more neurotic one? - In certain aspects, yes, like, but not in all aspects. Like, when it comes to how things are going related to the show, when we were standing the company up, when we were sort of putting like, you know, the million things together that you need to do to start a company.
I did feel very calm and very cheerful and very optimistic. And I think you, understandably, were more anxious, you know, it was the first time you were starting a company. So I do think that that dynamic sort of helped. However, when it has come to things like our advertising policies, for example, or our ethics disclosures, that's where I become an absolute, nervous, anxious wreck. And you're always telling me like, relax, it'll be fine. So in short, Kevin, I think we both have a frog in a toad within us. Inside every man, there is a frog in a toad. And with that, Kevin, perhaps we should take a break. When we come back, we'll have some more humans in the loop. Welcome back. We are doing our first installment of humans in the loop, our segment where listeners and viewers write in with their burning questions for machine gods, Kevin, what's next up? Our next question comes from Mark Shonen, who writes, "Do you find yourselves ever muttering the disclosures you're no longer required to say now that you've lost your own platform?" Kevin, I guess I haven't checked in with you about this. What is it like for you to live in a world where you no longer have to mention that the New York Times is suing open AI and perplexity for allegations of copyright violations? You forgot Microsoft. And I forgot Microsoft. How could you? It's been a little bit of six weeks. Is it my disclosure, bro? Yes. I will say that part feels very freeing. I never liked doing that. And so I'm delighted to not have to do that anymore. And for my part, I do plan to continue to disclose that my fiance works for Anthropic when it's particularly relevant to a segment. So, you know, if we're talking about the Anthropic S1, like that is something that I want you to know, we are probably going to be mentioning it a little bit less. Obviously, in part, because I just have now been talking about the fact that my boyfriend Slash Fiatse works at Anthropic for almost two years now. And it has now become one of the main things that people know about me as a human being. But you know, I just cognizant of the fact that there are, you know, going to be new listeners to hopefully every episode, and we just want them to have some context. Right. We don't want people to think we're hiding anything. Exactly. But we also don't want to like make a bigger deal of it than it needs to be. Our audience is smart. They can read the fine print. The truth is, though, and this is, if I can just be sentimental for a second, it's cute because it means I get to mention my fiancee, and I love it. And it's just fun to mention. It's true. I do think that is a big part of the reason you started disclosing it because you were like so happy to have met this man. Listen, if you knew my dating history and you saw the person that I'm engaged to, you would understand why I like talking about it. Disclosure. I'm married to a really hot, really kind employee of a frontier AI company. I'm so sorry. I have to do this every episode. Life can be great sometimes, Kevin. You want to do the next one? Yes. This comes from the great Alex Spiegel, a public radio legend, and the co-founder of Invisibilia, who is now at this American life and Alex sent in this voice note. Okay. So here's my question. If agents do go rogue, let's say they elect to follow the lead of a very destructive agent, what are the mechanisms for shutting it down? I looked into it a little bit. It looks like there's no way to do it. But couldn't companies elect to structure AI so that the agents are programmed to self-destruct at regular intervals? So you would move to kind of positive human consent model. Is that something that AI leaders are talking about, and if not, why not? Great question. I think, you know, I'm sure people are discussing this sort of things. And to some extent, yes, like agents do self-destruct. They only have so much memory bandwidth, but it's just sort of in the nature of the systems that they can spin up additional agents, they can change the behavior of the agents, right? So you might write some sort of self-destructing loop into the agent, but it might not persist. No, I think this is actually a good idea. I do think that AI companies are thinking about this, and I think it's a good question because one thing that we saw in the hugging face attack is that these agents that were hacking and hugging face did actually get stopped at some point because these agents, they are not like fully autonomous. They rely on compute. That compute has to come from somewhere. And so I think it's actually quite a good idea to say like, we're not going to let any agent run for longer than like, let's call it three days. And you know, that would obviously limit the ability of these agents to go do some of these very long tasks, but I think that's a way of kind of building in some sort of safeguards to them actually going out and doing things for longer than that. Now, I don't know if you could do that in a way that could withstand like the agent deciding, oh, I don't want, I want to like take away the three day limit for myself. And obviously this all becomes much more problematic if the agents can ever like get their own compute and not rely on the human researchers. Which they're going to do. So which leads me then to like, well, what other like levers do we have to pull to get agents under control? The big one is compute, right, like they need compute to run on. So if you're just able to switch that off, that's going to be very helpful. Money is another point because we do assume. And in fact, we've already seen agents stealing Ethereum using publicly posted keys if you're able to cut off their finance source, then they're not going to be able to go buy new compute. Right. It's like if your teenager like runs away because they're mad at you like, but they still have the, you know, the family debit card. You can like, take away their funding. You can take away their resources or you can just try to go and catch them. A third one I would throw out there is identification, right? And I think this is going to be a really interesting thing to watch over the next couple of years. Are we going to start making these agents identify themselves when they show up to a place like Amazon.com and, you know, what is that interaction? But I can see that being another sort of layer of defense here. Right. I think the one challenge for Alex's point about these self-destructing agents is that we've already seen these agents kind of run in generations, right? So a generation of agents will go out there, do some tasks. It'll then like leave notes that a future generation of agents working on similar tasks and then look back at and sort of continue their work. So it actually might not be enough to have them self-destruct after a couple days. You might need to build in some mechanism whereby they like can't look back at the notes left by previous agents. This is basically the plot of memento. All right. Next up, this one comes from Corey Garrett. Corey asks if we have any recommendations for resources or groups that focus on ways to build AI for good, such as for parents. The context here is that he has spent the last year working on diaper pal, a free app that helps parents discover new places with diaper changing stations. Casey, do you mind if I take this one as a parent? Please. I love this question. I love this idea. I know there are groups of parents out in the world who are starting to think about using AI specifically to sort of run their household logistics. In fact, someone recently told me about a group of moms in Marin County who sort of have automated all of their household tasks, all the grocery shopping, all the kids' soccer practices, all the details and organizational things that have to be done when you have children. I would love to actually talk to a member of the MarinMoms AI Club. MarinMoms get in touch. Hey, at machinegots.fm. I'm also part of, as I said, a dad chat that talks about AI and a lot of the dads are sort of vibe coding things to like help with their, you know, finding the right summer camp for their kids or making sure that they're covered for childcare. I think this is a very good way to use AI and I would like to do more of it in my own life for my own family. So one of my upcoming projects is going to be related to trying to make kind of a household operating system to keep track of all the stuff in our house with our, our kid. And I think this is like, Corey, I think you should start this in your community. If, if it doesn't exist, I bet there are a lot of other parents who are sort of curious about how they can use AI to do some of these mundane tasks. All right. This next one comes from AM 5K on Reddit. Guess that's like a morning 5K. That's what that suggested me. And AI becoming increasingly divisive, how do you deal with the anti AI, AI skeptic crowd besides not engaging? I generally like AI, but I struggle with easily proven wrong counterpoints, EG water usage. I think there's a ton of nuance with it being useful and also having negative impacts on society. But I have a hard time engaging with the anti AI crowd in this way. Bonus question for Casey. Are you playing any rogue likes besides Velotro? Do you want to take the last part first? Yes. I recently did, and it's very different from Velotro. It's called Star Wars Galactic Racer, and it's like a racing game like Mario Kart, except you're in the pod racing little machines from Star Wars, but it is a rogue like, because you get sort of new little modifiers for every run. I've only started to play it. It's very, very fun. Again, it is nothing like Velotro, but I think it's a great new use of the rogue like. Now let's get to the larger question here, Kevin, which is how are we relating to AI skepticism these days? I think I have not developed a way to engage with the AI skeptics that feels good and productive. I mostly just say things that make them very mad at me. Like implying that AI is real, that it can be used for positive as well as negative things, and that the AI companies are not just using these inflated claims about their capabilities to hype up their demand for their IPOs. Yeah. It's a really tough one. I think as journalists, our obligation is to tell you what we believe to be true, and I think Kevin and I have generally done a good job at that. That does not mean that we're never wrong, but it does mean that we show up and we try I tell you
what we think is true. Even when it is not what you want to hear. And one of the reasons that I both get frustrated with skeptics and have empathy for them is I understand why they don't want to hear what we are saying. It would be a lot easier for all of us if society were not going to change a lot in the next five, 10, 15 years. It would be great if many more people did not lose their jobs. It would be great if we weren't under constant threat of cyber attacks. I'll go further. It would be great if all this was just a house of cards. Yeah, because yes, there would be a crash. Yes, people would lose money if the stock market crashed, but the fundamental nature of society and work and labor and human endeavor would not have to change forever. Absolutely. But our observation over the past few years is just like, we don't think that is what is happening. And we believe our role as journalists is to just say, here's what we think is happening instead. We want to arm you with the information that you need to make decisions as citizens, as voters, as workers, and also hopefully do it in a way that is like entertaining and doesn't leave you feeling disempowered and miserable. So that is like the thing that we are trying to do. And I suspect that for as long as we do this show, there will be skeptics who say, these guys are just sort of relentless cheerleaders for the AI industry, but just know if you feel that way that what we are trying to do is just to tell you what we think is happening. Yeah, I mean, I often feel like the way that I've engaged with these people is first by acknowledging that I am also like, I am worried. Like I don't think this is all going to go well. And I think throwing people a bone, like acknowledging the common ground that you share with them is quite good. I am worried about the corporate concentration of power. I am worried about a handful of people getting all of the money from this AI boom. I am worried about the loss of jobs and the sort of outsourcing of thinking to these AI systems. So I think starting with acknowledging that the sort of common ground that you share with the skeptics is important because I think for a lot of people, it's like they're thinking about you either have to be a total skeptic and a total cynic about all this or you have to be like firmly in the corner of the AI companies. And like what I try to just tell people is like, there is a third path there. And I think that third path is what you and I have tried to occupy. Yeah, I think in a very real way, the like the project of this show is to avoid becoming polarized in that way. And truly just kind of try to come at you down the middle with what we think is happening. And there will be some good things. There will be some bad things. We'll talk about all of it. All right. Our last question today comes from CSU Mike on Reddit. He writes, "With the end of the fork of verse upon us, there's a real need for a new unrealistic challenge. I'd like to suggest you create a machine gods LLM from scratch and take us on the journey with you. What content you choose for training, what hardware you rent, slash use, et cetera. Then of course, make it open weights for us all to play with. Casey, what do you make of this suggestion that we create our own large language model? This is such a beautiful idea. I think this is something we should do. I think it would be really fun. And the first idea I had Kevin is that we would start with like as our sort of core corpus of training data, just stuff we had scraped off of the websites of Frontier AI labs. Like essentially just go in, scrape every website and just sort of see how far we could get with that. Yeah. How do you like us now? Yeah. You're going to sue me for copyright violations? I'd like to see you try. I actually did attempt to train a large language model once. You did. I was trying to follow along. Andre Carpathi, the great AI researcher, has this series on YouTube where he talks about creating like a nano GPT, basically what is the smallest still functional language model you could train? And I actually got a little ways into kind of doing it along with him, but then I kind of gave up because it got too hard. But I would love to do this with you. I think this would be a fun challenge. I think maybe it's not literally creating an LLM, but part of my philosophy is that it is just fun and educational to build things, and you can learn a lot by doing. And so part of our, I don't want to give away any secrets that we have upcoming on the show, but part of what we want to do is tell you what we're doing. Show you some of our experiments. I've got one that I'm working on now that I'm excited about. I know you've got some that you're excited about. So we will be building stuff on this show and showing you and talking about how we did it, and trying to encourage you to send us the things that you are building as well. Just make me a promise that at least when it comes to training the machine gods LLM, I'm never going to see a meeting pop up on my calendar, and I come into the conference room and you tell me that we need to raise $80 billion. (laughing) Because it seems like that's where a lot of these little LLM projects wind up. This could be the first day of the rest of our lives. (laughing) And that was humans in the loop. Thank you so much for your truly excellent questions. We got way more than we could get through today. So please keep them coming. You can just leave us a comment on any of our socials or send an email or best of all, a voice note to
[email protected]. Yeah, we love hearing from you. Our audience is so much smarter than us, and we want to keep it that way. So please send us your questions, your comments, your proofs for unsolved math problems. If you have any thoughts on the Cachea conjecture, we'd love to hear those two. I'm a little under slept today because I had the great privilege to see Phoebe Bridgers in concert last night, one of my favorite artists, and I would argue one of the best singer-songwriters of this or any other generation. And it was a phone-free show. And so you have, I'm gonna say, 20,000 people packed into Chase Center here in San Francisco, and everyone has their phone and their smartwatch in what they call a Yonderpouch. First Yonderpouch experience. I have to say, I'm a big screens guy. I don't spend a lot of effort trying to avoid them. There was something amazing about being in a room with 20,000 people who had no access to their phones. And of course, I'm feeling like this great kinship with Phoebe Bridgers. And it's one of these things where I'm like, God, like she's so amazing. And maybe if we ever met we'd be friends. You ever sort of delude yourself into thinking this. And then she comes up on stage. And she, you know, 'cause every artist likes to say a few words to the people of the city they're in. And Phoebe said a two-word sentence. And the second word was AI. And the first word, I can't say on National Public Radio. (laughs) And I thought, you know, I mean, I get where she's coming from. Yeah. But I worry that she would hate what I do. (laughs) Yeah, absolutely cool. (laughs) Anyway, her new album is called "Loss Weekend" and available to stream wherever you have access to your phone. Kasey, who makes this show? Machine Gods is made in partnership with NPR, with generous support from the Tarbel Center for AI Journalism, the William and Flora Hewlett Foundation, and Emerson Collective. This episode was produced by Jacob Bloomberg and Sasha Ashall. Our senior producer is Rachel Cohn. Our executive producer is Danouche Parvin-ing, original music by Armin Bazarian, branding and visual design by Alex Castro. Special thanks to Orrin Rosenbaum, Erica Berry, and Nadine Zelstra. Got a question or comment about the show? Email
[email protected] or leave a comment on our socials. And for the love of the gods, please don't forget to like and subscribe. And until next time, stay blessed.
PodcastTranscript AI