Silicon Valley Goes to War

When you get older, you got to have yourself a little bit of the comfort, and that's where the avocado green mattress comes in, man. And unlike most mattress brands, they'll only sell you online. They do have stores, too. You can go into a store, you can try them out. Unbelievably comfortable. Avocado mattress is designed for comfort, support, helping your body stay aligned. You know the last time my body was aligned, it's been, I think it might have been the last time the Mets won the World Series. That's how long it's been, it's been aligned. But these mattresses, avocado green mattress, responsibly sourced organic materials that are built to last. So get an avocado green mattress. Avocado products are made, not manufactured, and thoughtfully crafted with real materials to deliver lasting comfort and support. You can experience them in person at an avocado showroom or premiere retailer near you or shop online at avocado green mattress dot com slash TWS to check out their mattress and furniture sale. Avocado green mattress dot com slash TWS avocado green mattress dot com slash TWS. Period. Look, you know, I don't know when I'm going to leave this job. I don't know when I'm going to stay in this job. I don't know how these things are going to do. But when I do, you should replace me with indeed. You should call indeed when you're replacing me because I listen, this podcast, it requires skills. You need a face. You need a pie hole so that you can open your pie hole and just spout. That's not everybody can do that. And so, you know, if you need to hire, you got to say to yourself, this is a job for indeed sponsored jobs. You can reach candidates that meet your specific criteria like if you want them to have skills or pie hole certifications. Maybe even you want to know what the location is. Only indeed can get that stuff done efficiently. Trust me, you know, you got it. You got to kiss a lot of frogs unless you use one of these services there like indeed. So spend less time searching and more time actually interviewing candidates to check all your boxes, less trust, less time, more results. When you need the right person to cut through the chaos, this is a job for indeed sponsored jobs. And listeners of this show, we got a $75 sponsored job credit to help get your job. The premium status it deserves at indeed.com/podcast. Just go to indeed.com/podcast right now and support our show by saying you heard about indeed on this podcast indeed.com/podcast terms and conditions apply. Hello everybody. My name is John Stewart. Welcome to the weekly show podcast. We got a banger for you today. As you know, the world is hurtling in no small measure towards its utter and complete destruction. And there's a new wrinkle in the destruction of our world. And that is that a lot of the weaponry that we seem to be deploying at the various places around the world are being controlled by not necessarily autonomous, but large language model AI andthropic open AI. The same people that bring you Claude and chat GPT and help you break up with your boyfriend or girlfriend using a rhyming scheme that Drake would use. That's also being used to target and destroy our enemies. And it is incredibly chilling. And just recently, a huge controversy broke out into the open when one AI company andthropic drew a line and said, we shall not. We shall not allow our product to be used in this way. And then another AI company there, what do you call them there? The open AI went, oh, we will. That's cool with us. And but there's it's a lot more nuance than that. It turns out there may not be heroes and villains in this story, but we are going to discuss it all today in an episode entitled, how are we all going to die? And when exactly is it going to be happening? And we have two experts in the field of AI and how it is utilized. And especially within a military context, we have with us Dr. Sarah Schoeger and Paul Sherry. And let's just get to that. Ladies and gentlemen, we are delighted to welcome today on our continuing episode of, I think we're all going to die. Our guests today are experts in the field of how we are probably all going to die. Dr. Sarah Schoeger, who is a senior research scholar at the University of California at Berkeley and Paul Sherry, executive vice president for the center for a new American security and author of four battlegrounds, power in the age of artificial intelligence. Thank you both for for for joining us here today. Thanks for being here. Sarah, let's start with you. You worked in AI, you explained just very briefly your area of expertise as we move forward. Yeah, sure. So I used to be the lead of the Geopolitics team at OpenAI. That was a research team, and we focused on a variety up on a portfolio of topics relating to AI and international stability. And currently at in my role at Berkeley, I focus on new testing and evaluation methods for generative AI models and their potential impact on warfare and military AI integration. Very, very apropos for today. And Paul, for you as well, we're sort of where do you stand on studying AI on military and AI? What's your background with that? Sure. So I've got about 25 years of experience in the national security field. I was an army ranger, did a couple tours in Iraq and Afghanistan, and then I worked for a while in a Pentagon as a civilian policy analyst, actually led the group that drafted the Pentagon's policy on autonomous weapons, which is still in effect today. And then for the last 12 years or so, I've been at the Center for New American Security, researching, writing on this topic, trying to understand how is AI changing warfare, and how do we avoid some of the bad scenarios we're talking about? So this is perfect, because I think it brings in the perspective of, you know, Paul, you've been in the military, you worked in the Pentagon, you understand the ins and outs, Sarah, you've been at the companies that are developing these products. So let's just start for the basics. And I'm going to say this for my audience. Obviously, I understand how AI is used in the military. Paul, very briefly, how does the military utilize AI? And how is that different from their general practices? So it's, I mean, it's not really the military is using it like any new technology that they're going to try to find ways to be more effective, more efficient, much like these computers and computer software and computer networks today. So the military doesn't necessarily see this as something special or different, but really a productivity tool, just like I think a lot of people might use a large language model. An optimizer. An optimizer, right? Just optimizer, if there's something a little bit different. Yes, slightly, but so as you were working at open AI, when they talk about optimizing, are they developing at these companies? Are they particularly developing for military? Or is the technology that they're using just being utilized by military? Yeah. So generative AI models are both dual use and also general purpose. They're dual use in the sense that they can be used for both civilian and military purposes for good and bad, but they're also general purpose in that they apply to a variety of domain. So these are models that can be used in legal applications for software engineering tasks, as therapy bots. We now know some people use them as. So it is not particular, they're not trained for particular use in the military, but, you know, nevertheless, the military, I think, has been a keen adopter in the last year. I think I'd also be remiss if I didn't add that even though most consumers now primarily interact with AI, probably through these generative AI chat bots, AI is in fact a toolbox of methods. It is not exclusive to large language models or generative AI, and the military uses a variety of different AI techniques, such as, for example, machine vision, which is responsible for object recognition, facial recognition. So this is not just, in the way I might use it where I would go on to go, I'm thinking of visiting the Jersey Shore, recommend five different things, and then the AI will say, boy, that sounds like a great trip, because my AI is relentlessly positive, much to my chagrin. And then it'll list me a few other things. They're not just using it in that regard. They're using the other tools of AI, which I guess would be optimizing for anything from targeting to maybe supply chain or any of that. Is that correct, Paul? Yes, you can think of maybe three different types of AI. One is something that's been around for decades, that's really like handcrafted software written by humans. Good example of this would be commercial airline autopilot. We kind of don't think of that as AI anymore, but once upon a time, it certainly was. Military has a lot of things like that in radars and sensors and you know, fighter aircraft that kind of thing. - So already autonomous workings for some of their machinery. - Maybe bounded autonomy, I would say. Like a missile, there's lots of missiles that once you let that thing go, it's not coming back, but the autonomy's pretty bounded in what it can do. Then you've got machine learning systems that might have a narrow application. So they're doing computer vision as Sarah was talking about. Military uses these to analyze satellite images, analyze drone video feeds. The military's collecting more intelligence than it can possibly put human eyeballs on. There just aren't enough human analysts, but the AI can help you then look through these images and find targets and identify things of interest. And then there are large language models, which are these sort of like, much more general purpose text kind of machines where you can feed in lots of data, you can have analyze things, you can combine text and images and other types of data, and that's newer, and the military is also starting to use that as well. Now this is so in the public's eye, 'cause I wanna see if I can fill in the gap between what the public may view this as and what the reality is. In the public's eye, it is sky net. It is robots, titanium robots that can regenerate themselves, that are walking autonomously over crushed human skulls and just firing would appear to be phasers at all kinds of different things. And you're saying actually it's the same shit that we're all using like at the office for the most part. - I mean, for the most part, somewhat different applications, but I mean, it's the same types of things. And look, a lot of what the military does to be fair are back-end functions, right? It's logistics, it's personnel management. - Administrative and bureaucratic. - Administrative, yeah. It's like 95% of what the military does. Now, there's a different component that is actually battlefield capabilities, but a lot of the military use cases are kind of mundane. - So the battlefield, so let's get to that, 'cause that's really where it appears this new controversy is, which is the battlefield. The controversy appears to be, and this began when Anthropic had drawn two red lines. The red line being that there can be no just autonomous kill chains. A person has to be in the kill chain and that the AI cannot be used for general surveillance on the American public or gross surveillance on the American public. Sarah, is that understanding of the controversy? Correct, are those the two lines that are drawn? - So I make a slight adjustment there, which is that they specify autonomous weapon systems not kill chains in particular. - Okay, what's the difference there? Tell me the difference there. - Yeah, so an autonomous weapon system according to the US definition, and it's important that I'm noting that it is in fact the US definition because different governments define autonomous weapons systems differently. Are weapons that can select and engage a target without human intervention? A human can be in the loop, but it's not required. These weapon systems can function without human supervision. The language that's used in the DOD directive 3,000.09 is appropriate levels of human judgment. And Anthropics position was that they don't believe the models are sufficiently reliable, I agree, and that for autonomous weapon systems, they need a human in the loop, which is essentially already US policy. - So the US policy is the human is in the loop meaning. So let's walk through a scenario just to understand a little bit of what we're talking about. Let's say the AI is used to analyze satellite imagery and different targets. A human will then get the results. A human wrote the program I'm assuming to analyze it. A human will then get the results of this data that has been analyzed, make their selections and then give an okay to launch certain weapons that may in and of themselves be autonomous, meaning they'll guide themselves to wherever that target is. And is that a minimalist description of how this might all go, Paul? - Yeah, I mean, I think that's right. I think conceptually the idea would be who's choosing the targets? If a human chooses the targets, then you'd say the human is in the loop, the human's making that decision. If the AI is choosing it, or the AI sort of recommending and the human's not really paying any attention, then you'd say, well, the machine is doing that, right? So one way to look at this would be after the fact, something gets blown up and you'd say, well, who said it was a good idea to blow this thing up? If the answer was all the humans are like, oh, I didn't do it. Well, then right, that's not a great outcome. I assume that'll generally be the answer, right? But like right now, I think we're probably in a case of certainly have no reason to think otherwise, where the humans are the ones making those decisions. Now, the AI might be helping to process information, helping to even maybe prioritize targets for people. But the debate between the Pentagon and the topic is sort of a potential debate about where things might go into future. I don't think actually it's a debate at the moment about using a large language model to like autonomously make these life-and-death decisions on the battlefield and then people are paying any attention. - Tara, does that sound, is it that we're nervous that the computer will just decide on its own or that it will be wrong when it targets? So let's talk about around for a second. Describe how a situation like that goes wrong and where the checks and balances are for that. - So, clawed in the Ravens. - Okay, let me back you up real quick. You said clawed in the Maven. - Smart system. - I'll like to deploy my term, so. - I love the fact that it's named after something, you could name your cat. Hey, clawed. All right, so clawed is what? So clawed is the name that Anthropic gives to its flagship models and which is then used in the Maven smart system. This is an AI-enabled decision support system that does a variety of things, including some of the tasks that Paul mentioned, like helping speed up efficiencies and logistics, but has also been responsible for targeting in Iran. We now have confirmation there as well. And if public reporting is anything to go by in Bloomberg and the Wall Street Journal and others, the first day that the production of 1,000 targets in Iran has largely been credited to the MSS, the Maven smart system. - Now who makes Maven smart system? - Palantir does. - I just, whoa, did you guys just feel the room get colder? That was, oh, the hairs. All right, so clawed, who is made by Anthropic, and that is more of an interface that we are accustomed to using. What is its role in feeding information to the Maven smart system, which is a, I believe a system we are less accustomed to using and is maybe a little less transparent. So tell us how that operates. - Yeah, so the Maven smart system has been used for several years now. The integration of clawed is, I think, relatively recent. I believe in the last, in the last year because Anthropic was able to gain access, go through the certifications to gain access to the government's classified networks. As far as we can tell, clawed right now has been used in targeting. And again, according to public reporting, it seems that it has been used for target selection and then also target prioritization. The Maven smart system itself is designed to pull in different data sources, so from sensors, satellites, and such, and try then, and clawed then makes those disparate data more readable to the human analyst. So it boosts efficiency in that way, but it does also, reading between the lines a little bit, it does also seem to offload a little bit of human autonomy and decision making as well when it comes to that target selection and prioritization process. - Quite frankly, when you brought up a thousand targets, because I have no context, I don't know what I don't know. So I don't know if that's an unrealistic amount of targets, I don't know if that's, I'm understanding that there are target rich environments, there are target poor ones, is a thousand in a day, you know, I don't know how they count it, is that an unusual figure? - Oh yes, I believe St. Com said that it was 2X, the number of targets in the 2003 shock and awe campaign in Iraq, just to have a historical. - So 500 targets in a day was shock and awe, and this was a thousand. Now, I think we have to also take into account Trump math, which generally is like, this is the biggest crowd ever to see. you know in inauguration and it wasn't so how much of that is do you think is trump math and how much of that is an astonishingly high figure i mean it's being reported by bloomberg the wall street journal and the washington post all without and rags rags um i mean they're they're all taking it at face value and and it's acting as though it is seemingly plausible so there is no indication yet at this point that it's not accurate stop paying for too much wireless just because i don't know that's just what i do that's how it's always been that's just my company mint exists purely to fix that same coverage same speed just without the inflated price tag you can change your coverage people mint is the premium wireless you expect you know you're you're unlimited talk you're unlimited taxed your data but at a fraction of what others charge and for a limited time you can get 50% off three months six month 12 month plans of unlimited premium wireless the only thing keeping you from doing it is inertia laziness don't be the bean bag chair be the trying to think of something energetic pogo stick be the pogo stick that's probably not right bring your own phone number activate with e-sim in minutes start saving immediately no long-term contracts no hassle with a seven day money back guarantee and customer satisfaction ratings in the mid 90s mint makes it easy to try and see why people don't go back ready to stop paying more than you have to new customers can make the switch today and for a limited time get unlimited premium wireless for just $15 a month switch now at mint mobile dot com slash t w s that's mint mobile dot com slash t w s limited time offer upfront payments of $45 for three months 90 dollars for six months or $180 for 12 months plan required $15 per month equivalent taxes and fees extra initial plan term only over 35 gigabytes may slow when network is busy capable device required available the speed and coverage fairies see mint mobile dot com so you might say Paul let's say I'm I'm working in the military you you work there and you've been researching this hey Claude I'm looking to take out all the radar installations in Iran what would be a you know where would that where would I do that and how quickly could I get it done and then Claude would interface with maven which has all the data that it's gathered from I'm assuming satellites and then it's translating that data that they understand through whatever intel they've gotten and they're going to place it into real world menus of what you could target would would that be accurate yeah so let me like like explain what we know and then like what we we could speculate reasonably about because it is a little low pay what we know and what we do not know we do that but what do you think it's that so all right we know that um and tropics AI to Claude is deployed on US military classified networks it's integrated through the maven smart system which collects intelligence from different sources and it's been used by the US military in real world operations including the operation against Venezuela and President Maduro and operations in Iran and there's been some public reporting that's been used as I was talking about in target generation and prioritization like exactly how we don't know so now I'm going to I'm going to speculate about what what what might that look like speculation alert for Paul yeah so that could look like you know you're talking to an AI tool saying how we plan this vacation to the Jersey Shore there's somebody who's an as an intel analyst or a targeting analyst who's going to these tools and instead of having to manually go through all of this data that we have of where are the radars and what is the imagery of them queries it in natural language hey develop me uh for example a prioritization of all of the radars that have already been hit and what the current battle damage assessment is of them how much of they've been destroyed or they we didn't hit them again for a follow-on strike how much of them have not been hit yet and let's let's put on a list put in a database let's prioritize it and then let's match it to weapons that would be needed to take out these radars different types of radars might need different weapons and then let's match that to available aircraft to help build a strike package that would eventually go to like an aircraft gets a set of targets and weapons that are assigned to that target and so like the the technology is sort of being used throughout that chain to make it just easier for people to access and process this information so we would be doing that anyway it would just take longer that's right that's right now we're talking about basically replacing the things humans are doing with machines speeding it up making it a lot faster the US military said thousands of targets in a run having the ability to process the information the machine speed is very valuable for the military and then because it's clawed you could say and now give it to me like your earnest hemmingway and then it would give you the targets in short taciturn it would just be very terse and go all there so so sir are we kidding ourselves then that there is a line what is the controversy and how does it break down what is anthropics argument here Paul was saying earlier it's really about the future as it stands right now what is the controversy so I think the controversy in itself is a little mystifying because it sounds like the contract negotiations went south due to some shall we say strong personality clashes if you look at the contract between open AI and anthropic they're actually relatively similar if not the same they've essentially agreed both companies have essentially agreed to both both red lines the contract that they have with the DOD or with with volunteer ah so that's actually we don't actually know about that yet um so so stay tuned all right all right let's speculate some more people um yes stay tuned it's not clear um what model now volunteer might use or if they'll have an array of different models that they can choose from so who makes who makes the the contract is it does volunteer subcontract to anthropic or open AI or does DOD who is the leading role in in integrating these companies together so it's it's not unheard of in fact pretty common for companies to come together and actually combine resources to create a product um especially for defense purposes um so uh you know for instance the DIU trial and the DAWG trial that's the defense innovation unit and also the defense autonomous warfare group have a call for ah building essentially a tritable drones and they've issued that call to industry and companies have in fact responded to that call by combining resources and submitting joint proposals so it's not unheard of for companies to to come into contract with one another and then to approach the Pentagon so they'll do that together. Palantir and anthropic or Palantir and open AI will get together and say we've developed this package using you know our product makes it more readable for humans your product makes it more and so they'll bring it to DOD and now so the $200 million contract that anthropic had Paul do you know what that they had a contract with DOD what was that what was that for and for how long yeah I think so this is where some of the details we don't really know we know that they have an ongoing contract with DOD to deploy their AI tools on classified networks we know they're being used through the maven smart system but a lot of these details of like we don't normally get when defense contractors are working with the government in fact they'd like silver lining to this whole thing is the only reason a lot of these details are coming out is because this whole relationship blew up between anthropic and the Pentagon otherwise normally like they would have some deal about what the tools could and couldn't do we would never go and so that's like you know I think it's unfortunate actually that this sort of few to spill over between anthropic and the Pentagon but it is really the only reason that we have this kind of insight which is even still pretty limited on exactly what the terms of use of these contracts are how opaque are these military contract I know the you know DOD is it's the only government you know agency that's never passed an internal audit but how how opaque are these and the $200 million that they use is that over a five-year period just to use their products on their classified networks yeah I'm not sure that we know oh unless unless there are seen more details that I have uh you know even as an employee I do not have access to contract details it's it's very tented in a lot of these companies and on a knee to no basis now these two are is it is 200 million and on you I mean to me that's an enormous figure you know you're talking about the Pentagon budget in in total obviously dwarfs that one trillion now as they're pushing forth but still It's an enormous amount of money. Do they have that with different companies? - It's not, it's, I mean, it's a lot of money for like a normal person. It's not a lot of money for either the Pentagon or for these AI companies. They're all dealing with billions and billions of dollars. - This is just walking around money. - Yeah, it's not a little walking around money to anthropic, open AI. - It's not quite money under the couch Christians, but like, it's not a mass of amount of money. And the direct cost to anthropic of losing this contract is not substantial to them, relative to like the scale of AI investment that's happening right now in the AI sector. - How much of the contracts for like open AI and anthropic are consumer based, in other words, I pay $11.95 to get your latest model. And how much of it is corporate based and defense based? Do you guys have a sense of that? - Yeah, I mean, I think open AI is right now for 2026 projected to generate about $25 billion in annualized revenue. The majority of that is coming from subscriptions to its models. I think anthropic is in a similar ballpark where they're on track to generate, I think about $19 billion in annualized revenue. And anthropic is a little different from open AI in that it has prioritized enterprise contracts earlier on. But there is, I think opening AI's strategy, this is public, it has been targeted towards generating more enterprise contracts in the future. But I do think that the majority are still coming from individual consumers, developers. - Right. So the reason I bring that up is it does mean 'cause we're talking about their opaque and their tinted, but it does mean that the consumer has some influence here in that the government is not their sole benefactor. It really is individuals. And anthropic says, "I'm drawing a moral line." Whether that moral line is an actual line or it's already been traversed by whoever knows, is a real moral line or not. And open AI says, "I agree with anthropic and we are drawing the moral line here, autonomous weaponry and mass surveillance." And anthropic loses the $200 million contract and that same night open AI announces, "Hey, we just signed a big deal with DOD." How real is that moral line that anthropic drew and how real is the backlash against open AI for suddenly appearing to have turned around and said, "Oh, they won't do it, okay, we'll do it." - Yeah. I mean, like the backlash is real and it's happened from some AI scientists anthropic vaulted after this controversy right to the top of the charts in terms of downloads and the app store. So I think that's happening. The dollar amounts for both these companies are relatively marginal compared to all of the other non-defense investment. The bigger risk for anthropic is gonna be actions that the government is already taking against the company labeling them a supply chain risk and going after them in that way, which would designate other defense contractors saying they can't use anthropics AI tools in the further ends of their defense contracts. And then other steps the US government might take to retaliate against the company. They talked about using the Defense Production Act to seize control of their AI models, for example. So those are probably like the bigger risk. It's not so much the dollar amount of the contract. And Sarah, was it a real line? And it appeared to an outside observer that open AI immediately reversed their moral position given what you guys are both saying is a very small amount of money comparatively for their bottom line. - I'm not sure if there is an actual reversal. - Okay. - I do think that the military usage policies that are often designed by these companies are meant to preserve optionality for its leadership. There was a lot of backlash that I saw in real time, a lot of the AI community still congregates on Twitter and open AI hosted and asked me anything on Twitter in response to that backlash, which I think illustrates the fact that the public can act as a pressure point on these companies. But what we ended up seeing as a result of that AMA was not necessarily an alteration to their previous policy, but adding more language to explain they're already existing, they're already existing position, which in practice, again, doesn't seem to be all that different from anthropic, but I think the communication strategies maybe is maybe a little different. - Right. I don't know if it's the cultural fascination with the so-called great men of history, but I really would resist any kind of narrative that tries to identify a hero in a villain in this story. I'm not necessarily sure that those are appropriate roles for either anthropic or open AI, but to Paul's point, I think part of the sympathy that's been directed at anthropic is because they have been the target of government overreach, and so I think it's possible to hold two ideas in one hand here, which is that anthropic has been unfairly targeted, but at the same time, these two red lines that have been identified by both companies are probably inadequate, and the public does not actually have to accept those two red lines as the threshold of risk. Imagine if you had some kind of like a rewards program, you know what I'm talking about, like a Miles program, et cetera, but it's a rewards program that you pay rent through and then earn points for travel, dining, shopping, et cetera. 2026, if you're still paying rent without built, come on, brother. It's a loyalty program for renters that rewards you for your biggest monthly expense, which is rent. With built, every rent payment earns you points. You can redeem them. Flights, hotels, lifurites, Amazon purchases, so much more. It's a loyalty program that you pay when you pay rent that helps you get out of the place you're renting for like a week and go have fun. And by the way, built members can earn points on mortgage payments. You've got a house, whether you've got a apartment, whether you're sharing it with three of your knucklehead friends. I don't know what your life is. I don't know what you're doing. You can even redeem built points towards your next rent credit, or even a down payment on a home. It's simple. Pay rent is better with built. Well, you pay in rent anyway, get something for it. So join the loyalty program for renters at jointbuild.com/tws. That's j-o-i-n-b-i-l-p.com/tws. Make sure to use our URL so they know we're saying it. (upbeat music) But are we kidding ourselves, Paul, in that, you know, look, if we think through history, there's no human advancement that hasn't almost immediately been sought by the military for advantage, whether that advancement is sonic or chemical or biological, you know, Sarah mentioned two departments over at defense that I would pretty much assume nobody who's listening to this has ever heard of. You know, I think we've all heard of DARPA. But there are development groups. I'm assuming, you know, they said when they went into Venezuela, they used the, you know, the Havana Syndrome liquidation new weapon that like you pointed at people and their insides melt. Like, we are throughout history any advancement that a human can think of, their military wing is immediately going to try and utilize for some advantage. No? I mean, yeah, but look, two of the examples you gave, they're chemical and biological, we do have regulations on how they're used. We have conventions banning chemical and biological weapons. Right, but people still use them. People still use them, right? But not everyone and they've been sort of by, by many states, they've been treated as unacceptable weapons. And you get some prize, you get some outliers, you get people like Sad Moussaint and Bashar al-Assad who are going to use them still. But most states have given up those kinds of weapons. And I think it's better that they have. So the question with AI is not actually, are we going to use AI in the military? None of these companies are saying don't use AI in the military. The question is, should there be any rules? And if so, who sets those rules? Because like the sort of crazy thing about the dispute about autonomous weapons, is this news like until no one is actually saying we're going to use a large language model as an autonomous weapon today. That'd be crazy. If you have a large language model right in email for you, you better fact check that email, right? Because they do weird things sometimes. The question is, who gets to set the rules? And the Pentagon's answer is, when we get to set the rules, we don't want these companies dictating to us. And these companies in many of the scientists working there, they have a lot of discomfort about how the technology might be used going forward in the military. Sarah, you know, when it says about who sets the rules is it the company or is it the military? So we also, and I've read about this group, they are, they're called Congress. We don't hear much from them. there. It's this group of generally older white men who once their past retirement age enter into the legislative house. Are they utterly rudderless here? Are they just overmatched? Do they have any role to play? What can we expect? And what should we expect from them? That was optimistic. Let's start asking questions. I am somewhat sympathetic to this idea that private AI companies cannot be setting the rules in foreign policy. But one of the issues that I see today, and I think this does track with a role potentially for Congress as well, is that AI companies are in fact influencing foreign policy. It may not always be through the back end and through their contracts through the Pentagon, but they're certainly donating significant sums to lobbying efforts and tying those donations to US-China tech competition and arguing and arguing that a low or no regulatory environment is a requirement to quote unquote the China. And they're supporting political campaigns that agree with that perspective. And so this conversation is in fact coming for Congress. And they probably better be equipped at the very least. And I actually think Paul may even be a better person to speak in particular since he is in fact in DC. And I would be curious to hear from him what the general reaction has been from Congress on this issue. But I can say that AI researchers typically are very keen to discuss their work. And I've in fact never met a keener bunch of people who are willing to talk about the risks and opportunities related to AI models. So you can always send them an email. Yeah, I think they're pretty eager to have those conversations. Paul, so what say you down in Washington? Yeah, I mean, look, I'm here in Washington now. I can see the White House out of my office window here. I'm not going to pretend things are super functional in Washington, but you know, I think we have seen government engagement on some of these issues. And there are a lot of tools that Congress can use to have oversight of the military and intelligence communities. One is passing legislation, which may or may not be the right answer in some cases on the domestic mass surveillance stuff. Maybe on the autonomous weapons, maybe not, we might want to maintain some flexibility there. But there's other things. Congress could hold hearings. Congress can get people from the executive. Yes, they could. They could. That is correct. They get a people from the executive branch, come in and brief them and say, Hey, what are you doing with AI? And if you want to keep it classified, Congress can do classified briefings to educate them about what's going on. He said the military. Congress can use tools like procurement and acquisitions. Congress has the money. They are the ones that are allocating money to the military and intelligence community. And so that is a tool that Congress absolutely does use already to fund some projects and not fund others. And so like there's a variety of tools that Congress has potentially to influence these things. I think the model of who should be setting the rules. Maybe it's our democratically elected representatives. That is probably the right approach. Well, that's what I was thinking. But to Sarah's point, you know, look, these guys have more money than anyone. Right now the money is in AI. Now, obviously, they're using a lot of those billions to build data centers that we have sort of no idea where those are all going. But 25 million here, 25 million there, Elon Musk puts 350 million into political campaigns. The amount of money that's flowing from these from the tech sector is like nothing we've ever seen before. Do you think that's had the effect that maybe the AI companies want, which is to regulate us would be they've portrayed as national security risk. They've portrayed it as it would cause us to lose to China. Has that been effective? Or is it that they're overwhelmed by not really understanding the guts and bolts of AI? You mean Congress not understanding the nuts and bolts of the Congress. That's right. Yeah. I mean, I think there's a lot of, I've actually been super impressed when I speak with I mean, you can always find video clips online of some Congress member non-understanding something. I would use them on the show. Yeah, you know, I'm really cute. Okay. But I think like I've been impressed when I speak with members of Congress and their staffs, how knowledgeable many of them are about the technology and what it can do in its limitations. So I think there's always work to be done in terms of improving tech literacy in Washington. But I think some of the bigger challenge is just sort of getting over the hurdles in passing legislation and getting agreement, whether that's around federal regulation of AI or data privacy or social media or other types of, that's just that's actually really hard for Washington to do to pass legislation on these kind of issues. Sarah, how, you know, you spoke of this earlier. It's this great man. Here's why I'm very nervous. I've met a couple of these folks. And they do not seem particularly enamored with humans. I don't want to say outright misanthropic, but, you know, Peter Teal was asked a famously in a conversation, you know, should humans continue and, you know, he paused. I think for a pretty considerable amount of time before he went like, well, you know, and transhumanism, I once asked Sam Altman about the disruption that AI is going to cause to our workforce and that small amount of time in which it's going to cause it. And his response was just, he literally just looked at the question was five minutes long and he just went, we'll be okay. You know, how concerned are you with, with these great men and how great they actually are and what is their connection to, do they understand the damage that they also can do? Or are they megalomaniacs? Well, I mean, I can't look into anyone's, you know, heart and mind. But I would say that if they're able to cause harm, it's only because they are powered by immense, immense wealth and the high valuations of these companies and also by institutions that allow for corporate donations and excessive individual donations as well. So they're essentially enabled by our current institutional structures. In terms of whether these companies discuss the downsides, I mean, I joined in 2021, I left in 2025, there was a period where I think that was the dominant topic of discussion, right? Are these tools actually going to increase productivity? Are they going to replace tasks? Are they going to replace workers? Can they enable the proliferation of potentially weapons of mass destruction? And there were, there was testing and evaluations that began to try and answer those questions. So I think certainly the researchers at these companies have tried to make a concerted effort. But these companies are also complex organizations and there are always factions that are budding heads, right? Some people do prefer a low-to-know regulatory approach. They don't want to see state legislation. They prefer everything at the federal level. And then there are some, some who are at these companies who are actually quite supportive of state level legislation. So it really depends. I mean, I think of open AI and anthropic and frankly other companies is often going through eras, where certain factions went out over others and that's what ends up setting the cultural mood of the company. Do they understand the weight of what they're making? You know, I can't help but go back to Oppenheimer. And I, you know, when you have something that looks like it could be extermination level type technology, positive and negative. I mean, if we split the atom one way, we get energy that can power the world. If we split it this way, you can blow it up. And we all know which one we tried first. And it felt like the people who were making that weapon did it under the crucible of the Nazis. And so they developed it with this idea that, well, if the Germans get it, we're all done for. But it was clear that they at least felt the burden of that. Paul, in your experience, are they feeling the burden of this? Because what Sarah is talking about is, well, they did go through all that testing. We don't really know what the results of it was, and they seem to have gotten past that reservation. I mean, the AI scientists and engineers that I speak with, particularly those in the frontier labs are very concerned about AI risk. They, I think, understand better than anybody. Actually, the downsides of the technology, the way that it could be abused, the way that it could just do sort of strange things that might be surprising. I think one of the challenges here is there are incentives for the companies to move fast, to ship their products, because there's this sort of perception of a winner-take-all dynamic in the marketplace that we have seen in other tech industries in operating systems, hand-sets, and Well, yeah. I mean, in a way, that sort of commercial race to dominate the marketplace, and that does drive incentives. And these companies need a lot of money to build the data centers for trying the AI. So I do think the individuals take it seriously, and I think some of the companies, I mean, if you look at what Anthropics just did, I mean, they sort of stuck to their guns on this decision in a way that is going to be costly for the company. How costly, I think we just don't know, but they decided to do that. So I do think the companies take these issues pretty seriously. And if I can also add, I mean, I just at the risk of potentially misspeaking, the testing and evaluations that were done and continue to be done at these companies, they are often released publicly. But, you know, of course, in certain areas like, you know, Seaburren, so that's chemical biological, radiological, nuclear testing, and then also cyber, there are greater restrictions placed, placed around what can be shared with the public, but there are even reports, summary reports about what that testing looks like. And then a lot of the benchmarks that are used by AI industry are, in fact, publicly available. It just so happens that testing and evaluation of these large language models is still in a relatively nascent phase. And it's not always clear what the best way to test these models are if what we're trying to do is use them as proxies for social impact or risk. And is that, you know, the famous one is now, you know, if you remember the movie War Games and it was, you know, the first sort of kind of dystopian look at what would happen when computers take over was the Matthew Broderick movie from when I was a kid. And it was about a nuclear war game gone wrong and the computer just started launching, you know, nuclear weapons at all the different countries and at the very end the computer said, the only way to win is not to play with AI apparently. It was more apt to launch nuclear war than humans or standard computers. What do you know about that testing and is that apocryphal or is that, did that really happen? I mean, it did really happen. You know, I think a variety of researchers at academic institutions have now managed to replicate the findings. The models have a tendency to escalate more aggressively than humans would. And it's not really clear why the models do that. One theory is that in the training data aka the internet political scientists have a tendency to study wartime escalation rather than de escalation. So that may influence how the models respond to these war game type simulations. But I mean that in itself is of course a cautionary tale around using these models for approving the use of force or for decision making or frankly even for war gaming and simulations. Is it possible Paul that AI because of how adept it is at creating these targets and all these other things that it actually made going into Iran more appealing that before the age of AI we might have been more circumspect about the type of attack that that we launched. Are we seeing barriers to military action fall because of how quickly these models can they bring a sense of false confidence? I mean, I don't think today that's true. Like I don't think AI was a factor in President Trump making this decision. I think it was based in large part on the U.S. strike against Iran last summer against an enrichment program being very successful and limited. And then they'll rate against to grab Maduro being very successful and limited. And it's sort of like okay, having a couple perception of having a couple wins under his belt. I'm using the military. It seems to be effective. No downside. Sure. Right. So I think those are probably bigger factors. I think what you're describing could be a risk going forward, right? So one way and this could be a risk is some of the things that militaries count and try to calculate when they measure military power are things that you could see and you can count. You can count how many tanks somebody has, how many airplanes, how many ships. Then there are some things that matter a lot that are hard to count. We see this unfolding in the war in Ukraine, the morale of the troops on the battlefield. The Ukrainians are fighting for the homeland, the Russians are conscripts. They don't want to be there, the leadership, the quality of the unit cohesion. Those things matter a lot, but they're really hard to measure. So one possibility going forward is you can see a world where as more and more military power gets embedded into software and data and AI. It's kind of hard to measure that and it's like, well, we have this AI and it's amazing and it's wonderful and ours must be great. And there's this, it becomes harder for militaries and countries to sort of gauge what their relative level of power is and you might see more miscalculation. You might see countries sort of assuming well, we have this wonderful technology and we can win and the world will be over quickly and we'll all be home. And if it was not to be true, countries have made this mistake before. That's what happened in World War I, right? It's like, well, we've made it quite a few times. You might have done this. So that's not a, I think that is a possibility that could happen, but we're not there today. Sarah, has anybody studied the confidence, you know, there's a certain thing in bars, like there's a beer courage, you get a couple of shots and you get a couple of beers and you're like, you know, it turns out I'm a tremendous MMA fighter and I think I'm going to, you know, you get a weird confidence from alcohol. I find you get a weird confidence when you use AI. When you use those models, you tend to be much more assured in your decision-making because you feel like you have this kind of infallible being behind you. Has anybody studied AI confidence in decision-making? Does that, I feel it when I use it for the mundane tasks that I do? You know, I'm not sure if I've seen anything like that, but that's a really interesting, that's a really interesting point. I mean, I think what you're referring to, I've heard some people talk about chat bots or frankly any type of statistical analysis that's used to make decision-making as applying this mathematical veneer, right. It makes us feel better because it's therefore objective and it removes the human qualitative or subjective element to it. You know, the issue that I just keep going back to is of course that these models are not always going to be reliable because they are, in fact, statistical prediction machines. I mean, they're useful, don't get me wrong, but they're not, they are inevitably going to output something that is incorrect. And so being able to keep appropriate human judgment and to create a system in such a way that people do not abandon their critical thinking skills is a very important facet, I think, to any type of human machine teaming that we're seeing today in military AI integration. Is that something the military is concerned with Paul because, you know, in looking at it from like, let's say from an educational standpoint, there's been a lot of studies that show that when kids start using this, their ability to do that, to think critically, to reason and all that falls, that it becomes this crutch that when utilized, you no longer develop those kinds of skills and ways of thinking, does this become a crutch for the military to use? And the second part of that question is, is are we ignoring this whole other area which is, hey, Claude or hey, maybe or whatever it is, design me five nerve agents that the world has never seen before? You know, is that another usage that we're not so far, we're only talking about chain of command? Is there a whole other area we're not even really thinking about? Yeah, well, that is, that is certainly a risk, the potential for AI to enable biological weapons and to maybe even lower the barrier to countries, to non-state groups, to terrorists to do so, maybe not today, but that's a concern down the road. I think in terms of military usage, I think the military is actually pretty keenly aware of, for people in uniform, they understand the responsibility that they have. Okay, if they're going to launch this missile, they own where that missile goes. And I think there's a couple of concerns, one would be making sure that they really understand this AI system. What is it going to do? Is it going to do something strange? Is it going to fail? How's it going to work? Ensuring that there's human responsibility and accountability, I think it's actually quite important to the military that's sort of part of the military ethos, but it's challenging for a lot of these AI systems because it's not like a traditional computer program where okay, there's an accident you go back and you say, oh, this is the line of code to cause the problem. Now the answer is embedded in this massive neural network with billions of connections, you know, like why did it do that? I don't know. And so it gets into these issues of trying to evaluate the model's performance, what are some conditions in which it might be biased in certain ways. They tend towards sycophancy, towards basically telling you the answer that it thinks you want to hear. Well, that could really be a problem in some national security application. You're an intel analyst and you're like asking some questions and it's like, well, you know, this is what I think you want to hear, right? So that was Napoleon's whole issue. They were like, sure, boss waterloo, what a great idea. You should go there. Yeah. Now this is going to sound ridiculous, but does it do like what it does with us would Which is, would you like me to give you a 10 day bombing plan? Would you like me to add in other targets that may seem ancillary but might have militarily? Is it that casual when it's describing what it wants to do next and how quickly does it do that? I have never used the Minivan Smart System and so I don't actually know what the personality of the chat bot is. Or is that what they use Claude for? Yeah. I mean, you bring up an interesting point though, right? And that these models can be fine tuned with different personalities to be either, you know, more acquiescing, less acquiescing. We know that users, of course, like to be fond over a little bit. But it's, you know, it's possible that it's not presenting information in the most neutral way out there. We just don't know publicly. I don't think. Do you know Paul? No, I don't know. It's an interesting question. I think one way to think about these models is they're sort of role playing, they're playing a role that's in their training data. And then that can be fine tuned by additional training that they get from the companies. And so that's why you get the sort of personality and different personalities among the different models. So it's an interesting question of like the ones that the military is using or the intelligence community, what are they sort of trained on? And are there hidden biases that might be kind of subtle that are hard to detect? I mean, that's, I think, a difficult problem. Or not so hard. And I just got a chilling feeling that they're training it on the headset. And so they plug something in and the model just pops back. Hell yeah. Let's do this. I told you about my invention, the crumple, the crumple. It is a topographical blanket for dogs, but not the same topography at each time. Every time you throw it on the ground, it changes its topography. It is an amusement park for your dog to find a place of comfort and warmth, but also with interest. It's not the same old, oh right, this is where I put my right paw. And this is where I curl my butt. No, it changes every time. It gives them. It's like visiting, it's like Epcot. It's an Epcot center blanket for the dog. And have I started this business yet? I have not. That's right. To the great dismay and disappointment of our audience and maybe humanity writ large. I have not started my crumple business. And I'm going to tell you why. It's too hard. It's nerve, I don't know how to do this. It's daunting, but you know, you got Shopify here and makes it easy for people. You can get started with, they got a design studio, hundreds of templates, help you build an online store. It can match your style. You can do this with it. They also have 24 hour customer service support, world class expertise and everything. It's the commerce platform behind millions of businesses around the world and 10% of all e-commerce in the United States. It's time to turn those what-ifs into with Shopify today. Sign up for your $1 per month trial today at shopify.com/tws, go to shopify.com/tws. That's shopify.com/tws. So these are like some, I think some really difficult problems with the technology that we've got to find ways to work through, to use it in ways that are safe and effective. And I don't think they're easy answers. I think the technology has some strange and new challenges associated with it. Sarah, you're starting with having a really balanced but also nuanced view of this. What keeps you up at night? Is there something about this that you think about as particularly challenging? Yeah, well, there are many challenges. Let me see if I can narrow them down. Or throw them all out there and we'll go through them one by one. I mean, I think about the challenge related to global governance. I mean, for over a decade now, over 90 plus member states have been meeting at the United Nations to discuss regulating or the possibility of regulating or even introducing a treaty instrument that would regulate lethal autonomous weapons systems. But because of the nature of the forum at which these discussions are taking place, it's a consensus-based body. It's at the convention on certain conventional weapons. It's very unlikely that a treaty-based instrument is even possible in this space. I mean, you can think about how hard it is to pick a restaurant with you and your five friends. Now, imagine that you have 90 plus governments trying to decide what's regulated. Could the restaurant that could kill all of us? But now, how have they been able to do it? Why can't they use the model that they used for atomic weapons? Oh, I see. So, I guess there are a few reasons for that. So the convention on certain conventional weapons, it's really in the name. It is talking about conventional weapons. And autonomous weapons, the conversation around them, has really focused on trying to preserve meaningful human control to discuss whether that's even possible, whether they can actually discriminate between combatants and civilians. And if they can, in fact, discriminate between combatants and civilians to an extent, then they technically could be legal under international humanitarian law, but militaries would still need to abide by the existing international legal order and international humanitarian legal principles. And the good thing about this particular forum is that, though, you know, regulation with teeth is probably off the agenda, most states have been able to have have consented and reaffirmed the norms around international humanitarian law as applying to autonomous weapons systems. So that's, I think, also a silver lining as well. Has anybody kind of gotten it right? And Paul, I'll ask you because, you know, maybe you see ways through this from being in Washington. But, you know, has the European Union done a better job with this? Has any governing body, has any international body? Is there any pathway here that you see that could help establish at least the beginning of guardrails? I think that actually the best avenue we have is starting at the level of AI hardware and then sort of building guardrails domestically, eventually globally, kind of from the ground up. Explain the difference between hardware and the software. Right. So, so the thing about these AI systems that it's kind of amazing is they require massive amounts of computing power to train the most capable models and to deploy them at scale. Now, you can make smaller models that you can deploy on a laptop, for example, or some other kind of edge device, smartphones, but they're not as capable. But the most advanced ones are going to be really big. They're going to have to run in the cloud. They're going to need really advanced chips. And to deploy them at scale as a society, you're going to need a lot of these really advanced chips. Well, these chips are made in one place on Earth. Taiwan. Taiwan. Now, that does not on the face of it seem great that it's an island 100 miles off the coast of China that China has pledged to absorb by force, if necessary, but it is a conservative drawback, I think. That's right. Not the best year graphic position, however, these fabs that TSMC has in Taiwan were the most advanced chips are made depend on technology from three countries in the world, Japan, the Netherlands, and the United States. And without that technology, they cannot make these advanced chips. And so that's sort of starting at the hardware level that actually is like a really narrow choke point to begin to then control the technology. So the deal we just made with UAE to give them the chips, the previous concern had been that they would then sell the chips to China. Did that just blow a hole in the net? Well, I mean, the bigger question is like, what is the global diffusion of this hardware look like at the tail end of the Biden administration, literally the last week when they were in office? They dropped this like very complicated rule called diffusion rule that basically would take US expert controls on the most advanced chips to China, which you've had for several years now. They started the first Trump administration and expand that globally and it's kind of tiered system where depending on which country you could get so many chips, it was a little complicated. Trump administration threw that all up the window. But I do think that like the chips themselves are a way that we could begin to shape who gets access to the hardware, who can build the data centers because they need these chips to do it. And that's a hook for guard rails, right? So you can say, are you want to buy all these advanced chips? I want to see your domestic regulation surrounding making sure that people aren't going to use these chips to make a biological weapon. Like we did it within rich in uranium and the things that you would need to be able to do that. That's actually like not a bad analogy here, right? And we're, okay, you can get uranium for peaceful civilian nuclear purposes, not to make a bomb. And we found ways to separate those two, not to enrich it to that level, right? Right. It would be the same thing. You can use these chips for peaceful uses, basically most everything, but you can't use it to make like an offensive cyber weapon. For example, and put some guard rails on how the technology is used. Right. And Sarah, is there any fear that like by the time we figure this all out? computing is the new standard. And that's pushed us enough. So by the time we figure out, okay, these three chips are crucial to any ability to do that. And then somebody else comes in and says, actually, that's not state of the art anymore. Are we moving so quickly that suddenly quantum computing is the power that's necessary to drive these? And that's a whole different can of worms. I think you're now learning in real time that AI researchers aren't necessarily experts in quantum computing. And I am the worst person to answer that question. We just, because the reason why I bring it up is I just read an article about it and I have no idea what it is. They were to someone was describing that actually quantum computing is going to be wildly preferable to large language models. And I was unable to understand the difference. Is there knowing that you're not experts in this? Is there a sort of remedial version of what the difference might be? Paul, do you have any idea about this? Yeah, I think so. So we are seeing some progress in quantum computing. I think I don't think it's going to like change this picture in AI for a couple of reasons. One, the quantum computing will become valuable over time for like some very niche kinds of computation, but not necessarily everything. And I don't think what large language models or other large, large neural networks are doing today. It's also like the case that we're just, we're not seeing in quantum computing this kind of really rapid exponential growth that we're seeing in AI. So right now the price performance, the performance per dollar of AI chips is doubling about over two years. That's like really grown very, very quickly. That's not true. That's the productivity of it. That's like the efficiency of it. Right. So that's really powerful. That's what's allowing this massive growth in it's one of the factors data and better algorithms are factor two. We're not seeing that kind of exponential growth in quantum computing. It's really hard science. It's like difficult physics. It's it's much more traditional science of people are making incremental gains. I think we're going to continue to see progress. But I'm a skeptic that we're going to see this like transformative leap ahead in quantum computing and say the next five, 10 years, the way that we're seeing with AI right now. So in summation, the drama that we're seeing between anthropic and open AI, that's really the soap opera story. And there's not necessarily a lot of there there. It's the general competition between these companies that are going to try and establish primacy in in the realm of AI models. Military application is just one element of the revenue streams that they're pulling in there. The real sort of where you guys are really looking at is that interface between who are we going to end up trusting more. The humans that are developing the AI models, the humans that are running and integrating the AI models or the models themselves. Would that be kind of where the real tension is going to play out? I mean, I think it's fair, but I would just add that it's not going to be only one technical, you know, it's not only just going to be safety through the technical stack or only safety through the law or safety through regulation and policy, right? It is truly going to be an all of society effort. And in part, because AI again, general purpose, and it can be used across a variety of applications. So a one size fits all approach to safety is probably not going to work. Is it a kin to the battle against climate change? And if so, that we haven't done a great job there. So is that does that give us a pathway not to follow? I mean, I think any pathway towards AI governance is going to be through cooperation. And I don't want to be overly cynical here. And so I'll try and draw on a positive, a positive example. No, go for go for cynical. I'm going to go what? I'm going to give you one positive example, just one. I think I've been coming. Come on, Sarah. Hit us. So under the previous administration, there was a, they launched the declaration on the political declaration on military on military use of AI and autonomy. And that was a voluntary declaration with principles and norms. And around 60 countries signed on to it. And in that declaration, it really centered international humanitarian law and also civilian protection. Those conversations can resume. Those diplomatic conversations can resume. Really that what's stopping right now is political will. And that process can in fact happen alongside the existing UN processes as well. So there isn't really a way out of this that doesn't involve talking a lot to other people. But that there is something there to build on. Is the cynical version of that that international norms and rules seem to be in disfavor with the current sort of, I guess what you would call large power politics that seem to be playing out. Would that have been, is that your downside? Yeah, I mean, I think that's probably fair. Or we're dancing around lots of things. It is. But you know, at the same time, people can continue to demand this through Congress. We mentioned Congress earlier. I see a role here potentially, right? If they want, if they want to do something, you know, if they're, you know, I'm counting on your students, doctor. I'm counting on your students, a burglary, to be able to come up with the, yeah, away through it. Paul, what what keeps you up at night and and give us a nice balance between cynicism and optimism on on the way forward that you see. Yeah. Look, I think the reality is this technology is going to reduce a lot of challenges. How is it used by the military? What are some of the risks in cybersecurity? It's like a little bit about the risks of AI empowering biological weapons. I mean, there's a lot of, there's a lot of risks of the technology. And that's just didn't like the sort of national security space, not to mention things like job dislocation. I think my takeaway from this fight between anthropic and the Pentagon is that these decisions are too important to be left up to any one of these entities on the round, right? For-profit companies or the government deciding on its own. I think like yes. We all have a stake in this world that we're living in, not just on some of the civilian uses, but even military ones. All right, so okay, we're not the ones building the killer robots, but if people build them, we're going to live in that world. You know, we do have a stake in what that looks like. And so there's, there's, you know, democratically elected representatives. All of us, your listeners, you know, have a role to play in weighing in on this debate. And if there's a silver lining of sort of this controversy we've seen in the last couple of weeks, it's it what would have been a private conversation is now happening publicly. Kind of messy. A lot of personalities involved on all sides, but it's airing this issue. And then we're all sort of debatable. What should be these red lines? You're holding a second. That's a good conversation to have. And I'm encouraged that we're having that discussion. Fantastic. Guys, thank you so much for joining us on this. Thank you for having me in. Thank you. Thanks for the discussion. It's great. Should I, did I take the wrong? Should I not become her? I, yeah, my hair is still on fire. Sorry to say. It did not calm me. Did it help at all that because they were still putting it through a process that they were, they still wanted to filter the problem of AI through international cooperation or legislative process or you know, government incentives for that, rather than saying, look, we're at one second to Doomsday. Somebody's got to step in. I think I was kind of calmed by the idea that like we have these models for like other sort of disarmament that have worked, like what you said about like nuclear weapons, like the nuclear arm seals, but also like, you know, the Iran deal. The one he used was biological weapons and chemicals. Yeah, exactly. Like I think that that was encouraging to think like we have these frameworks that we could look at as models. And like this isn't totally uncharted territory. And then I think I'm just reminded that we're not doing that. So that's where the nerves come back in. I also think freaking people out too much is not conducive to getting them to act as we've seen with climate change. I think it's really hampered people's ability to organize. So I did appreciate that. I also really appreciated this is just a personal thing, but over the weekend I did notice a lot of people framing a topic as the good guys, which I thought was really odd considering all the reporting coming out about these Iran strikes about the Maduro capture. That's already been used. Yeah, and I really appreciated just that we had someone who's worked at one of these companies like breaking down that it's not a binary that there's so many considerations for these people to make. And as you've said, they're not, you know, perfect actors. Everyone makes mistakes. The technology itself makes mistakes. So I just appreciated that nuance. I also like that what they talked about was, you know, in terms of the usage, it really is, in some ways, a kind of cousin of the way that we use it, in that it's just collating data more quickly and spitting out those pleasantly formatted, you know. Yeah. That did not make me feel better. Here's five great places you could bomb. But, John, how do you use AI? Oh, like, I'll go in to AI and be like, okay, I want to find the best, like, who's got the best pizza in blah, blah, blah, like, generally I use it for, like, those types of recreational, like, I want to try this sport, you know, what's the stuff I might need? How would it be hard to get into it, like, that sort of shit? And it's, it's effective, you know, here's five places you could go to get started with, you know, paddle tennis, you know, that, and then the government asks, what's the best pizza and then bombs those places? And I don't know if this had make me feel better, you know? No, but here's, so here's why though, here's what I'm gonna say. So in the same way that I look at autonomous cars as, like, dystopian, almost everything I've read about it is that it would make it safer. That human error is actually at a higher fraction than the other. Now obviously letting it just make decisions on its own without any kind of interaction makes me uncomfortable, but I guess the point is like, how great are we actually? Not driving? Not good. Because we bomb, we bomb shit randomly before computers ever happened, like, how, what was our track record on bombing? Like not so fucking great. Like, we dropped two atomic weapons on Japan, would the computer do worse than that? Like that, that's my only point is like, are we elevating humanity to a higher status than we've earned? I think that the issue is that it makes doing these things so much faster, so maybe it would have dropped five atomic bombs on Japan. I don't know, you know, like, but if we were to look at the charts, that seems to be the way that it would go. Right. Also in the waymo case, but there was reporting recently that people in the Philippines were intervening, you know, like, we're just not there yet. Oh, really? Okay. Yeah, I didn't know that. Yeah. I'm assuming that. I guess what I was saying is sometimes in the battle between man and machine, we tend to look at man a little bit more favorably than maybe man has earned, but I absolutely get that. To that point, one of my biggest fears about AI continues to be what appear to be the pathological personalities of the people that run those companies. Same. Yeah. I was thinking about that in terms of the attitudes and the personalities of these chapats when you were talking about that in the conversation and just remembering like six months ago, though or whatever company is above Grock for Elon, but yeah, made a contract with the government for like 42 cents for like a year and a half, they could integrate Grock into government. And apparently there was like posters around DOD with headsets, you know, AI generated mugs saying we want you to use AI, like, they really want to get government hooks on, you know, their product and I just imagine like someone in government being like to Jillian's point a little bit like, okay, there's flooding in Texas. What do we do? And they're like, well, Hitler is the best person to do with this, you know, you're thinking that they contracted with mecha Hitler as opposed to just normal Grock, no, you're right. And those guys manipulate algorithms and they are ideologues. They have a lot of them are transhumanists like they are leading us down a path that is not favorable, I think. Yeah. When Sarah said, I don't know the personality of Maven, like I, my stomach drops, oh my god. We can't be talking about the personality of weapons. That's so dark. Or when she talked about Sam Altman's heart in mind, I was like, does he have either of those things? Right. Hearts or minds. But it is like, I don't know the personality of the Palantir generated war autonomy system. Wild shit man and not going away. But I loved how measured they were and I loved how they sort of helped us through there. Brittany, what are the people have for us this week? Sure. John, we're still going to get Greenland, right? I think we already have it. We've already won. Like everything else in the Trump administration, we've already, not only it's like with the Iran war we've won and we're doing more, it's, we are Schrodinger's country. We exist in all different, we have Greenland and don't have it at the same time, but they respect our unique and, you know, unparalleled power. And so absolutely we have it and don't have it and could do whatever we want with it and won't because of our largest and I don't know. It's like how the Iran war is almost complete, but also could go on for as long as it takes. We live in this middle space. Yeah. Almost complete and never done. Yeah. And we are going to only stop at unconditional surrender and we've already stopped. We are, we are Schrodinger's country and it is only the beholder that determines where we are on the existence plane. We obliterated the nuclear program, but they're one day away from it, you know, hard to keep up guys. Very hard to keep up. Is that it for them? One more. One more. John, why does everyone ask you where to get pizza? Because I'm considered one of the world's leading and this is recognized around the world. Any, any of the larger pizza conglomerates, the pizzas, they recognize, you know what? I think because of that rant I did on deep dish pizza in Chicago. I think that's the only, oh, and we did something on Trump eating it with a knife and fork. And so those two things, you know, there is no real accreditation other than the port noise rating system for pizza. So oftentimes non-experts are elevated to that position. I mean, little do they know you're just asking the AI, I was about to say your deals earlier. Can I tell you the truth? Like, my world there is so small I go to Joe's on carmines if I want a slice and I go to John's on bleaker if I want to pie and that's kind of my, like, as you guys know me, my world is small. I am not, I am not a man who is out there. It's the same clothes I have eaten. I shouldn't even be telling you guys this. I eat the same lunch every day when I go in to work at the Daily Show and I've done it since I've been back, the exact same lunch. Well, what is it? I'm embarrassed to say. What is it, like, Girl Lunch? What's a Girl Lunch? It's girl dinner. Just like little bits of everything. I'm just very curious, trying to prompt you. Yeah. We are not ending a podcast until you tell us. That's what a Girl Lunch is. It's a little bits of everything. All right. Yeah, you don't have to really cook. Yeah. No, I order. I don't make it. Let me just be very clear when I go to work at the Daily Show. I don't cook. I call out and I get a bean and cheese tostada. Okay. Wow. We have to stop talking about lunch during these recordings. With all that setup, I know that that was a bit of a let down in terms of, I should probably be more particular, like I get every day the same thing. A quarter of a lime spritz lightly on steamed cod. It's a bean and cheese tostada. The only difference is it comes with jalapeno and I generally say no jalapeno and I've done it every time for three years. It's very Jennifer Aniston of you. Is it really? Did she get a tostada lady? Well, not tostada. Did she look like a tostada lady? Like a hard lady. When they were doing friends, she would eat the same lunch every day. Oh, is that true now? What did she get? It was like a chef salad. But I actually know exactly what it is and I'm not going to fix it because I don't want to look at crazy things. I am the Jennifer Aniston of late night. I think people have always. But you guys know that. On my 50th birthday, the daily show bought me, we had one of those staff, all hands meetings down in the studio and they had a box sitting on a table and I opened the box and I pulled out. It was a t-shirt, a long john's shirt, khaki pants, hiking boots and a thing. And it was exactly what I was wearing that day and I was flattered and humiliated all in the same moment. But I am a creature of very lame habits. But I hope, man, what information they got today. Very, very, very nice, lovely program, thrilling and chilling and nerve-racking and all those different things. Brittany, how do they keep in touch with us? Twitter, we are Weekly ShowPod, Instagram Threads, TikTok, Blue Sky, We Are Weekly Show Podcasts. And you can like, subscribe and comment on our YouTube channel, the Weekly Show with John Stewart. Beautiful. As always, guys, thank you guys so much for. The incredible preparation you did on this episode, Lee Producer, Lauren Walker, producer, Brittany Mometovic, producer, Jillian Spear, video editor and engineer Rob Vittolo, audio editor and engineer Nicole Boyce, and our executive producer is Chris McShane, and Katie Gray. We will see you next time. Wait, wait. (upbeat music) The weekly show with John Stewart is a comedy central podcast is produced by Paramount Audio and Bus Boy Productions. (upbeat music) (upbeat music) Paramount Podcasts. (upbeat music)