Autonomous Weapons 101 + Anthropic v DoW

Mike Horowitz of Penn University, formerly with Biden's DOD. We didn't get enough on Monday on autonomous weapon systems. This whole around war thing gotten the way. So we both thought it would be who the audience to do a little bit of a one-on-one on what these things are, how they kill people and just how autonomous the world is in 2026 and perhaps beyond. Mike, take it away. So Mike, how would you characterize like where the fear lies in the well-meaning researcher or head of an AI lab who thinks their technology used for certain types of autonomy would be a bad direction to go and maybe contrast that with how this stuff is used today in Ukraine and Iraq? I think that the average sort of maybe Silicon Valley, AI safety researcher, AI safety researcher who's worried about autonomous war bots is probably worried about AI making the decision essentially about who lives and who dies. And it's, you know, thinks that that's a, like that's some dystopia that they don't necessarily want, that they don't necessarily want any part of. And so get worried about the incorporation of AI into the like pointy end of the sphere for militaries, especially when it comes to, you know, potentially selecting and engaging targets. What I think sometimes gets lost in the conversation is the substantial degree of autonomy that already exists in modern weapon systems in that the US military and basically 40 militaries around the world have deployed autonomous weapon systems since the early 1980s. These are often automated systems that are, we're using essentially deterministic good old-fashioned AI, like more or less, that are on ships, like these enormous gatling guns called the phalanx that can operate by algorithm. And so if there are too many threats that are coming and say too many missiles about to hit a ship and operator can basically flip on the algorithm, which can automatically target and hit those kind, can automatically target and hit those incoming threats. Or, and you also have things, you also have semi-autonomous weapon systems that fall into the category of fire and forget munitions. Think about how a radar guided missile works. So, you know, a pilot sees that, you know, believes that there's an adversary radar, that's a legitimate target. They, you know, press the launch button, the radar guided missile fires. When it go after going a certain distance, it turns on a seeker, it detects a radar, it goes in, and it destroys the radar. There's no human supervision or control of any kind after that weapon is launched. And like, hey, maybe that radar is on top of a school. Maybe that radar is on top of a hospital. And so that's the status quo in some ways of autonomy in weapon systems. And those kinds of technologies have been used since the 1980s, and we tend to think that they're way better than what came before, which was the, you know, like area bombing, essentially, of World War II. And so there's already a lot of autonomy and weapon systems, which then makes this conversation about what we don't want AI to do in the weapon space, a lot harder because it can sometimes be challenging to talk about it without inadvertently, in some ways, wrapping in all of these existing kind of weapons, which we generally think are good, more or less, like in the world where we support military action, because they're both more effective and are more accurate, making things like civilian casualties generally less likely. I was reading to command the sky, the battle for air superiority over Germany, as well as fire and fury by Randall Hansen, and like people forget that those planes, when you drop the bombs, you would be like lucky to be within miles of the thing that you were trying to hit. So imagine doing the Iatola, like actually succeeding in doing the Iatola, like compound explosion that we saw over the past weekend, like would have caused tens of thousands of people to die as opposed to like 50 or 100. So you would have dropped like tens of thousands of pounds of weapons that were, you know, from a couple dozen aircrafts, that yeah, like hundreds or even thousands of people would have. And, and so yeah, like precision strike, drones, all of this stuff just like tightens the radius of the thing that you end up exploding, and then even with drones, like what we saw with what Israel pulled off of like, you know, you're going into specific windows and apartment complexes. So anyways, all right, but let's take the narrative forward from the 80s to the to the 2020s. And I think that's kind of getting a little closer to, you know, the contemporary Iqfactor on the stuff. So now a thing that is a doable do in the context of weapon systems is imagine a deterministic algorithm that is trained on a very exquisite data set, like a data set, say of Russian tanks or Chinese fighters or something very specific. You can now essentially train an algorithm that can go on board, on board, you know, some kind of weapon systems, you know, maybe a loadering munition. And that kind of and it can launch, go to an area, turn on a seeker, and then look for Russian tanks. And in some ways it can then use an image classifier to say like, is that a Russian tank? No, all right, move on to the next image until it finds a Russian tank, at which point it will destroy a Russian tank. So, you know, that's a weapon then that is launched by a human who in theory then is trained in how the weapon works, understands its upsides and limitations, et cetera. And but that weapon then is after it is launched, not just operates autonomously, so you can't recall it, you know, like a radar guided missile or something from the 80s, but is now using an algorithm as the basis for destroying a target. And you see like early days for this a little bit in the Ukraine context. There's so much jamming and electronic warfare in the Ukraine context. And so Ukrainian FPV pilots, know, there are one way attack drones were getting jammed constantly by the Russians. And they're coming up with different concepts of operation to try to get around that or they're working on like connecting fiber optic cables that could stretch for like kilometers to be able to hit a target. I like what if somebody cuts the cable? So, there are, you know, now some Ukrainian weapons that essentially have last mile autonomy where you, if there's jamming that occurs in the last kilometer and the data link goes away, that weapon then, again, trained on an algorithm that maybe has a target library of targets that's allowed to hit, then can still continue on to the target and hit the target. And that then becomes an absolute necessity for militaries fighting in electronic warfare heavy environments in trying to operate if you don't have access to say satellites or your equipment gets jammed. So, why don't you give the generous reading of the Anthropic case where they say that or actually, it was not familiar like that. - I have no, I actually have no problem with what Anthropic said Thursday night. I think we can cut this, but I actually have no problem with what Anthropic, like I think that they do everybody a disservice when they use the phrase for late Thomas weapons because nobody knows what they mean. And then everybody picks it up because it's Anthropic and it would be better if everybody used similar like words mean things and it'd be helpful if people used the same terminology and we're talking about the same stuff. I think their position is actually like very reasonable which is LLM's aren't ready. - What are reasonable concerns model providers should have as their models kind of get into the ecosystem that's spinning up weapons like the drones that have the last mile capability? - So I think part of this depends on what you want the role of the human to be in the context of using weapons and what you, in some ways what you're most concerned about. And one of the things that I think thinks think it's lost in the conversation about autonomous weapons systems, at least for the United States, is that for the United States, the United States has a policy on autonomy in weapons systems but also has both domestic legal obligations and international humanitarian law treaty obligations that essentially require that there is human responsibility and accountability for the use of force. And that's a requirement that exists, whether you're talking about a bow and arrow, whether you're talking about a radar guided missile, or whether you're talking about an autonomous weapon system. And so when you start from there, I think things start to fall into place a little bit. The issue is that if you don't start from there, and what you are worried about is, AI systems making decisions about whether somebody is a waffle combatant on the battlefield and like turning into, and turning into killbots, and you think that that will happen without a trained commander making the choice to deploy that system in a context that they believe is legal and where they've received legal approval, then you think about it a little bit differently. But if you start from the premise that there's always human responsibility and accountability for the use of force, and you believe, although people might have different views on this, that the Pentagon will follow its own rules and the law on these issues with regard to the use of the use of force, then in some ways, it becomes a question of when we think autonomous weapon systems of different types are ready for prime time. By ready for prime time, I mean systems that are as good as or better than existing weapon systems, since nobody wants their weapons to work more than militaries. And because weapons that are not reliable and aren't safe, by definition, don't work well. And that means military commanders and operators, where the use of these things will determine whether they live or die, are strongly incentivized to get it right, essentially. And what this means is the incentive for the military has been to incorporate autonomy in ways that they can validate works well and works better, again, as well as were better than existing, existing weapon systems. And so once you start from that proposition, in some ways, you're already starting from a place where some of the worst case fears than about what, what inthropic calls fully autonomous weapons and what the Pentagon calls autonomous weapon systems, some of those concerns, I think, then become less, less of a broader moral and ethical issue and more of a question of, can the weapon do the thing it's supposed to do? All right. So let's, let's spend a little tie walking through the sort of legal strictures that require humans to be involved in this. I mean, it's a nice directive, you wrote, Mike, but like there have been a lot of Biden era over regulations, which have been thrown away over the past few, or I guess the past 15 months. So what else besides that directive are kind of keeping humans involved in these sorts of decisions? The thing that keeps humans involved in decisions on the battlefield actually has nothing to do with the Pentagon's directive on autonomy and weapon systems. The Pentagon's policy on autonomy and weapon systems is about the process for developing and fielding semi-autonomous and autonomous weapon systems. So the precision-guided weapons of today, the autonomous weapons that have been used for decades, and then what future autonomous weapons might look like. The, whether a human is actually involved in an involved in a substantive way in making the decision about the use of force, is actually governed by separate Pentagon policy. The Pentagon has guidance on the use of force, written by the lawyers that say when you're allowed to use force or when you're allowed to use force or not, and that's connected to treaty obligations under international humanitarian law, where commanders and operators have to ensure that uses of force are, you know, that there are proportionality and distinction and all of those good legal requirements. And so this is not a case where it is, you know, necessarily, like if what you're worried about is like the robot deciding, this is not the case where it's like Biden-era policy standing between like us and the killbots. It's a broader architecture on of law and regulations surrounding the use of force that again, isn't even specific to AI. You can think about Pentagon policy. - Yeah, I mean, I just think, the question is like, when you have a secretary of war who's telling commanders to kill everybody when they see you vote, like does any, and there's no like inspector general thing that exists anymore? Like who cares? Like does it, if you're thinking about selling something into the, you know, into the system, like, like how much can you hold your hat on any of that stuff? - That's not an AI issue then. That's like a Pentagon follower. That's like a Pentagon following the law issue. And so the, like, one can believe that, and like, you know, and like, you know, musters of evidence for it. But like if you believe that, that's not a reason why, like autonomous weapon systems are good or bad or like different AI uses are good or bad. Like that would be a reason, you know, in theory not to do business with the Pentagon at all. And or why, and that could be true, you know, like, much more broadly, much more broadly beyond that. So like, take the point, but that's not a, that is, that would be a reason why a company might choose not to do business with the Pentagon. Not an argument about autonomous weapon systems in particular. - All right, so back to autonomous weapon systems. This seems like an inevitable force of history that, you know, we're gonna go from one mile to two miles to five miles to one person controlling one drone versus five drones versus 10 versus 50. I mean, like, you know, is there, like, our, what are the reasonable, you know, if the drones are actually better than the sleep deprived, you know, like on their fifth cigarette human being, in picking out the, the targets, like, what are the legitimate ethical concerns around the war bots? You know, it's, it's this, if we're, if the analogy, if the actual analogy is like, Waymo versus a human driver. - Yeah, I mean, in that case, I think the, the ethical arguments against autonomous weapon systems are not that persuasive, frankly. If what we are talking about it, that is a weapon system where there is still human responsibility and accountability because there is still the human that makes the decision to use it, that would be responsible and accountable at the end of the day if something goes wrong and you are telling me that that system will be more effective at hitting a target than, you know, a specific target that you, that you wish to hit, then the, you know, like, 18 year old on their fifth cigarette or something like that, then the, that's seems to be like a better weapon system. And one, again, like in the world where we, where we're like, yay, using military force, that, that makes sense then for the military to acquire. Where, where this gets tricky sometimes is the sort of objection to these on sort of ethical or moral grounds gets, can get conflated sometimes with what are, I think like, can be pretty legitimate concerns about like how, like, whether they would actually work. And that's part of what, you know, anthropics, a beef with the Pentagon was getting it, was their belief that large language models like Claude are not ready for prime time when it comes to incorporation into autonomous, into autonomous weapon systems. As a side note, I wish that anthropic would not call them fully autonomous weapons because that is not a term of art and it's not clear exactly what they mean by that. Relative to the government entity that they're dealing with, which talks about autonomous weapon systems, which have a clear meaning in policy, but like putting that aside, the, the, I think in tropics actually, probably correct about the limits of large language models in powering autonomous weapon systems, which is also why the Pentagon isn't doing it right now. It wasn't talking about doing it, you know, like one of the many reasons why this, like whole blow up between anthropic and the Pentagon was so needless. - We had a fun riff on second breakfast where I think it was Justin was talking about how usually when you're selling into the Pentagon, you're promising the moon and you sell them like a piece of cheese. And that is kind of like the general dynamic that you're getting, but in, in, inthropic, they're actually, you actually have a provider who's like, no, like we cannot do all the things that you imagine we can. This is maybe like more of a moral philosophy question, but like is that, I don't know, just what, what's your sense of kind of going to them and saying, hey, you know, our thing isn't ready for prime time. We're not sure we trust you to not use it in a way, which is like gonna be unsafe for service members. - This is part of the issue, is part of the part of the issue, I think, that the. It is not being clear and from what I know, not even true that the Pentagon was trying to get Anthropic to develop autonomous weapon systems like fueled by LLS. That this was a theoretical concern about like a possible future ask from the Pentagon. And you know, Anthropic even said, I think it was like last Thursday that like, they think autonomous weapon systems actually make sense. They just think their technology is at ready for prime time on this. And further said, they were willing to work with the Pentagon to make it happen. And so the, you know, which is part of why the Pentagon's concerns in some ways are more philosophical than anything else. But like part of the challenge here is so all military systems and especially weapon systems go through a testing and evaluation process, which is how the military figures out that whether a system is reliable or not. Because again, as I said before, if it's not reliable, even if it somehow got through the approval process, it's not like commanders and operators want to use it, like they want to use stuff that works. But it's been, it's challenging to figure out how testing and evaluation for these large language models should work, especially in like a safety critical kind of use case like a potential weapon system. And so in some ways, there's like work on the back end that needs to occur to validate these systems, like in addition to like whatever advances in the systems themselves that anthropic things needs to happen. - What do you think about this cloud versus edge distinction that bubbled up this past week? - I think the, I'm actually reasonably sympathetic to a cloud versus edge distinction as important. But that's because I am very anchored on the Pentagon's definition of an autonomous weapon system. Remember that definition is, the Pentagon's definition of an autonomous weapon system is a weapon system that after activation can select and engage targets without human intervention. And unless that system, unless there's human oversight of that system continuously, which generally there would not be, because then there's a data link that someone could hack or jam. Like by definition, there is no cloud access. So the effective autonomous weapon systems, in general, aren't going to have data links and cloud access. So if you have a system that only operates through the cloud, then it almost by definition can't be used to power an autonomous weapon system. You could use it to do lots of other military operational kinds of things related to the battlefield about like planning military operations, directing thing. Like there are lots of things you could do with that system. But if it's cloud-based, that means it can't operate on the edge in an autonomous weapon system, which means it can't be used for an autonomous weapon system. So I'm actually reasonably sympathetic to that distinction, at least at a high level. And I think it's a reasonable case to make. - So the idea being if you just think doing autonomous weapon systems is icky, but you still wanna like help out with command and control and just to send back office stuff, being in the cloud API access provision spaces, like a relatively neat way to make that distinction. - It based on where the technology is right now. And also keep in mind, I mean, I think anthropic is correct that this tech, just that LLMs aren't ready for prime time in terms of incorporation into autonomous weapon systems. And so even if you could somehow put them on the edge, the, it's not clear, like it'd be, it's tough for me to imagine those kinds of systems surviving the Pentagon's own review process for those kinds of things. And so, in which it's very similar, frankly to what anthropic said, but I think that the way to protect that from happening, even if you are, even though again, I'm saying that you shouldn't worry about it that much because it's not just that it wouldn't be a good idea, like wouldn't make it through Pentagon testing processes. But if your system can't operate on the edge, it can't be in an autonomous weapon system, like period dot. - Let's talk about command and control with all this stuff. I mean, it just AI smarter than me, man. Like how can it obviously-- - No one is-- - Nobody's knowing this anymore, but nobody puts Jordan in a corner. You're very smart. But not as smart as these AI's. They know so much. - I mean, I think the question asks is like, you know, it's like, I mean, I think like the real conversation to happen in some ways is like, okay, like Mike, you can like policy in legal, you can like policy your way to like persuading me that an autonomous weapon, that like autonomous weapon systems are like basically fine. - I see what you're saying. - Because they only happen with you, because there's always human responsible, like whatever, like all right, can see your point. But like, is it really worried about the like, AI's gonna like do all the really scary stuff before that? - Yeah, okay, so I mean, I think with the autonomous weapons, like the range of how bad things can get seems to be like relatively narrow. Like a thing can blow up a school, or it can like turn around by accident and decide to like, you know, blow up the base that came up from. But once you start handing over more and more range, not rains, not to just like a drone swar, or actually know what is kinda scary. So let's talk about, okay, so we've got like the rogue drone swar seems like really not great. But one or two levels up of like the rogue like, brigade or battalion are like, you know, combatant, I mean. - Here's what, I mean, let me take two things here. Like here's, I am not that worried about the rogue drone swarm. And the reason I'm not worried about the rogue drone swarm, in some ways is because I believe that military's when it comes to weapon systems tend to be relatively like little sea conservative institutions. And because of the incentive structures that I laid out before and some of the challenges in developing like good testing and evaluation procedures, I think the notion of a like super unreliable drone swarm that like causes like major chaos like out there out there in the world, like maybe there are militaries we should worry about that for, but like even in the current context, I'm not like terribly worried about that in the context of the United States. The United States has been way too slow, frankly, in incorporating AI into military systems. Rather than to, rather than to quick. But the, but if, but again, that's also because even in the case of the drone swarm, the drone swarm would have to be activated by a responsible human who would be accountable if something went wrong. So there's, there's a responsible chain of human command and control from the decision to use force which would be by an informed person, all the way to the varieties of points of impact. And military's actually pretty good at accountability when stuff goes wrong. But sorry, if we want to talk like at the operation. - I think, I think the thing is though, right? Like when I've been using cloud codes for the fast few weeks and it asked me, do I want to let it do, and I say do a thing and it says, okay, well, you need permissions, press two to give permissions. I've been pressing a lot of two over the past few weeks and then at some point I googled, how can I stop pressing two? And then the internet said there's a setting you can put into cloud that says dangerously accept permissions. And now I don't have to press two all the time and it just does the thing I want to do. And it's been totally fine so far, way more efficient, way more effective, less time. You know, it's, it's, it's, it's succeeding for me, right? So like, I just, I mean, maybe, maybe all we have to hold our hat on, Mike, is this idea that these are these slow and pure, and pure credit institutions that still have like paper trails and human beings with like legal liability if they screw stuff up as well as like, you know, the moral weight of killing the wrong person. But there does seem to be something kind of inevitable about more and more parts of your work. You're just handing over to a thing and like, I don't know, I mean, maybe our, our, maybe like test and evaluation can catch it. But when it's just like, you know, when you have, when you have a secretary of war saying, I want you to use AI and this is just like the broad ethos, like that's going to come across, that's going to come in contact with, you know, it may not always be this sort of inertia that that wins out here. And you could end up getting to a point where, you know, we end up handing over too much. Totally. And no, I think the, like I am far in some ways, like I believe that the self interest of commanders and operators and having things that work will generally, will make some of the worst case scenarios that people like worry about, surrounding autonomous weapon systems a lot, a lot less likely and that it's possible to capture in some ways a lot of that, a lot of that upside without some of those worst case scenarios. If you want something to worry about more, I think it is this question of operational decision making. The idea that we already have now tools like maven smart system, which are again, not in and of itself a large language model. That is a thing I think people need to get their heads around more. The risk here is not necessarily connected to whether it's a large language model or not, but you have a platform like that that's designed to essentially be a dashboard for commanders out in the field, especially going up the chain to the combatant command level to try to understand what the world looks like around them. What are the enemy's forces? What is information from open source looks like? What are information from classified sources? Like aggregating those things together and churperting that information in a way that may generate increasingly specific recommendations to commanders for courses of action. That the commanders would then take and be responsible and accountable for. And the risks then I think are in some ways more prosaic than we sometimes talk about. Like one risk then is automation bias where people trust algorithms more than they should given the reliability of said algorithm. They're all sorts essentially of like behavioral decision making biases that can then get triggered if you're just offloading more and more like cognitive judgment to the machine. And you can the military can develop standard operating procedures and training to try to hedge against that as much as possible. But there's a there's a point at which you've just like given a lot than to the given a lot than to the machine. That's a real thing to worry about. So but is that okay? So like what is the I mean we're really getting we're really getting some maven smart system. We're getting our money's worth over the past week or so. I imagine the stuff was literally made for like figuring out how to you know degrade an air defense system and find missile launchers and blow them up or whatever what have you. And you can totally see like a anodine way where this technology helps you you know find out who needs to fly wear and who needs to be bombed with what faster and more effectively than like people doing it with paper paper. So I don't know what's what's what's there to be concerned about. This seems actually like not super scary just like that humans could have done it better or there's some you know bug and they end up bombing the wrong thing or. Yeah like it's like a joke in the pentagon the number of like staff officers that it takes to like generate operational plans and like generate operate you know like like idea like concepts for you know for senior military leaders in decision and the if you can like automate a bunch of that process and and especially the process of generating alternatives and like in helping like estimate some of those risks as long as there are responsible humans involved like monitoring those feeds and like ensuring that they're that they're basically right. There are tons of like real efficiencies you can gain from this that are be like super useful. The problem is if you start using these systems and but like don't really understand how they operate and don't have people that are really well trained on them. And like that's when you you generate like a bunch of that kind of risk and it might be frankly that we are in a transitional period where you have these technologies coming online that can be increasingly powerful and a workforce that wasn't raised on these technologies. And so it might be more inclined potentially towards automation bias or some kind of you know like some of those kinds of things where when you know like Gen Z is like when you have Gen Z three stars or something and they've been like raised on this tech and understand its limits intuitively because they've been working with it all their lives. Maybe some of those risks become maybe some of those risks become a little bit less likely and then the question becomes like how do we get from here to there. But none of that is about a ton of weapon systems. Which is like the thing that's been in the news. So this is so I think this is like. Yeah, so the the worry about the command and control stuff from your perspective is like, okay, if we were at a six in effectiveness and the promise of doing this well gets us to an eight and a half, then if we trusted too much, then maybe we'll be stuck at a seven or a six and a half. But you're like really bad actually. But these are not like doomsday scenarios, Mike, right? No, they're not. I'm not a doomsday guy like the. No, I just mean like the I think that really just not exist like what's there to be worried about? You're kind of saying like nothing. It'll be all fine. I'm not saying it's fine. But 10% scenario. What I what I'm essentially saying is that I I tend to think that the process that a military like the United States has for evaluating capabilities, whether we are talking about edge capabilities on the battlefield or operational level capabilities, it is not perfect. There are accidents with every military system. There are mistakes with every military system like those are in heaven and those will be inevitable in the age of AI as well. What I'm what I'm suggesting is that that process generally works pretty well and that the military is pretty resistant to fielding stuff if they can't know whether it works or not, which actually will screen out a lot of the things that people are worried about the most and make the like worst case kinds of accidents a little bit less a little bit less likely. And so I think on balance I'm again like more worried about the Pentagon going too slowly with these technologies rather than rather than too quickly, but that's also because of how much one how much autonomy I think is already in existing systems. And two, I all of the mistakes and errors that humans make today. So I think maybe then the most the concern I can I can sell you into the most is like the what are we calling this the autonomy bias. Yeah, the automation bias at the strategic level where if you if you believe that the that the tactical and operational people have enough kind of like bureaucracy in their way that they're not going to adopt something which is like totally not ready for prime time because it's their bot it's their physical bodies that are on the line. Like having the sort of you know super cool AI simulation of how like oh this like this like war that we could start would just end in an amazing fashion because like of course Mavin will have like done all the nine dimensional chest to figure out how it's going to work out for you that in fact that is the thing which is which is maybe more scary than SkyNet is just like a president or secretary or you know someone on the joint chiefs being like okay the AI has just got this like and have a higher confidence that wars would work out. Yeah, I think like senior decision makers uninformed about AI trusting AI tools too much and like guiding their decisions if you want like a war like how does stuff like really go or I it's like less because the pointy let's because of the AI at the pointy end of the spirit it's like much more at the strategic level and that I think is an absolutely legitimate concern because all those like standard operating procedures and training and incentives that I was talking about don't necessarily apply to senior leaders so if they really want to worry like that's where I would worry. So it's it's the president and it's a defense secretary just like Chan with their mill dot AI scheming up who to bomb next which is the who even know. Okay anything else Mike I think this was helpful I I'm feeling a little bit better. You should feel a little bit better but most of all it would be helpful if everybody just used the same words or at least that would make my life better if everybody used the same terminology to talk about this stuff. This weapon systems AI decision support systems like automation bias. We could all use the same words for the things that we're talking about it would be like it would be like easier at least like have some of these debates. And this is I think this conversation is why I think the the thing aside from the personality stuff the thing that really tripped up and thropic and the government was on the domestic survey on side because there like with this autonomous but like you can you can fudge a solution especially if you're willing to do like 95% of it already but I would imagine I would imagine that in traffic was just like look we don't want to be involved in finding illegal immigrants and they were like fuck you this is you know we we were truly elected to do this stuff why aren't you. letting us, letting us through. So anyways, but I'm sure that will come out in the next slide. Totally reasonable to be, totally reasonable to be worried about AI enabled master balance. I don't worry about it most from the Pentagon. I worry about it more from like other agencies, but like totally legit. All right, let's call it there. Thanks for dropping by me. Thanks for having me.