PodMAPS 3: Normative proposals on military AI – What comes next?

In warfare, AI is no longer a future concern. It's here, guiding drones, processing intelligence, and accelerating decisions that can determine who lives and who dies. The question is no longer whether militaries will use artificial intelligence, but how we govern its use before it outpaces our ability to control it. Welcome to MAP's Dialogs, that's military AI peace and security. I'm Ulysses Richard, consultant with the UN Office for Disarmament of the Feds, and this is Episode 3, normative proposals on military AI. What comes next? Over the past two episodes we've explored the opportunities of AI in the military domain, the risks that keep experts awake at night and prospects for international cooperation and capacity building. And today we shift the focus from diagnosis to prescription. What governance frameworks are taking shape? What's happening where and where do we go from here? We're witnessing what some describe as a Caledos Cup of Governance initiatives. We have multilateral resolutions, state-led declarations, regional frameworks, and multi-state order dialogues, all emerging in peril. Some see this proliferation as healthy experimentation, but others worry that it fragments efforts when coherences desperately need it. And to help us map this terrain, I spoke with Dr. Jasmin Afina, who is a researcher at the United Nations Institute for Disarmament Research, who offered an overview of what's emerged so far. If we look back on how the normative governance of AI in the military domain is emerging so far, I think that there is a growing understanding that the governance of AI in the military, but also wider security domains will be multilayered. And by multilayered, I mean, we of course have an ecosystem of initiatives and processes at the multilateral and international levels within and outside of the UN that have emerged. So there's the first committee resolution on the military domain last year, but also the one this year as a follow-up. But we also have a number of states-led initiatives, such as re-aim or the political declaration issued by the United States. But then there's also a growing number of discussions at the regional level. So with, for example, at the ASEAN level, the defense ministers have adopted earlier this year, a joint statement on cooperation in relation to AI in the defense sector. And then at the national level, of course, you have a number of national strategies emerging or at least an increasing number of states that are conducting efforts to develop these national strategies. And you have also more thinking on military frameworks, such as updating rules of engagement when regards to AI in the military domain, procurement processes, documentation to really add more granularity on how AI in the military domain is being governed. But of course, beyond this policy frameworks, you also have an emergence of multistakeholder efforts and initiatives. So for example, track to dialogue, led, for example, at the U.S. China level with Qinghua and Brookings. We also have, within Unity, a bit of a self-promotion, a roundtable for AI security and ethics or raise, which we also launched last year in close partnership with Microsoft and a number of experts, including national sort of all the P5. But also, for example, we have Kenya, Singapore, and maybe Ecuador, India and Pakistan. It's really a way for us to get multistakeholder perspectives at the sub-government level. So if you look at it, I think the best way to describe it is a kaleidoscope of governance, framework, and initiatives. And I think seeing how this is going, this is basically going to be the future when we have the governance of AI in the military domain. And at the center of these efforts, it's the UN General Assembly, which in 2024 adopted a resolution specifically addressing AI and the military domain. The resolution in question was led by the Netherlands and the Republic of Korea, and it established several important markers. Here's running a holdback from the UN Office for Disarmament Affairs on what this resolution set in motion. In 2024, the Kingdom of the Netherlands and the Republic of Korea, which had previously hosted the two Rian conferences, submitted to the first committee of the General Assembly, the first draft resolution on artificial intelligence in the military domain. While the General Assembly had previously adopted a number of resolutions on the peaceful applications of artificial intelligence, this was the first time it considered the implications of AI on international peace and security. This led to the adoption by the General Assembly on 24 December 2024 of resolution 79-239 on AI in the military domain. The resolution set out a number of principles, perhaps most importantly a clear affirmation that international law applies to AI throughout its life cycle. In the resolution, the General Assembly was also mindful of the need for appropriate safeguards and recognized the need to narrow existing digital and AI divides in societies. The Secretary of the United Nations was encouraged to facilitate knowledge sharing and raise awareness of the implications of AI in the military domain for international peace and security. This provision is the basis of the map dialogues on which this podcast series is based. More practically, the General Assembly requested the Secretary General to seek the views of member states and observer states on the opportunities and challenges posed to international peace and security by the application of AI in the military domain and to submit a substantive report on that basis. International and regional organizations, the International Committee of the Red Cross, Civil Society, the scientific community and industry were also encouraged to submit their views. 42 states submitted their views. The resulting report was issued as Document AAT78. It contains a summary of the views of states and observations and conclusions of the Secretary General. The Secretary General's headline recommendation is for states to take concrete steps with a view to the establishment of a dedicated and inclusive process to comprehensively tackle the issue of AI in the military domain and its implications for international peace and security. In addition, he suggested that particular attention be paid to the use of military AI in situations involving the use of force. He also urged all states that possess nuclear weapons to agree that any decision on nuclear use be made by humans, not machines. In October 2025, the First Committee recommended to the General Assembly a second resolution on AI in the military domain. If adopted, the General Assembly would mandate three days of informal exchanges in Geneva for states to further share perspectives on observations and conclusions as presented in the report of the Secretary General, including opportunities and challenges, existing and emerging normative proposals, and next steps. These informal exchanges would be an important opportunity towards the establishment of an inclusive and intergovernmental process on AI in the military domain as recommended by the Secretary General. The second resolution has since been adopted and mandates three days of informal exchanges in Geneva, which is an incremental but significant step towards the sustained process that the Secretary General envisions. But governance is now waiting for consensus to emerge in the UN General Assembly alone. And outside of the multilateral system, a peril track has been developing and is one that compliments into governmental discussions with multi-stakeholder dialogue. The responsible AI in the military domain initiative, or Re-Yam, was launched in 2023 in the Hague and has since convened summits in Seoul, and its global commission brought together an independent group of experts to develop what they called strategic guidance, essentially a framework built around the concept of responsibility by design. So I spoke with Sophia Romanski, one of the penholder behind the commission's report about his core findings. The report overall is guided by, I would say, two central tenets, the central tenet of responsibility by design and human centricity. We start by presenting three guiding principles. From these guiding principles, emerge core recommendations for all stakeholders. And so this involves anchoring the responsible development and use of AI in the military domain in relevant and applicable principles of international law, so that we put as the very first one. But then we also, as our second core recommendation, argued that at a legally binding level, the decision to authorize the use of nuclear weapons must be preserved with humans. Subsequently, the third recommendation states that national policies must be implemented by guarantee human responsibility across the entire AI system life cycle and recommendations for and five specify the international level actions that can be taken. So the creation of a permanent inclusive multi-stakeholder and multilateral dialogue on responsible AI as well as the creation of an expert network. But then we go one level deeper and we look at the life cycle itself. We provide specific recommendations that expand on recommendation three about responsibility across the AI system life cycle and this primarily concerns new activities or adaptations of existing activities that militaries and developers should undertake. And finally, we give also sets of specific recommendations for states, for militaries and for industry. And this is under recognition that these specific actors also have specific responsibilities depending on where in the life cycle they intervene, but also kind of in this more global conversation about responsible AI in the military domain. And so overall the findings are that a lot of thought has to go in at the outset and not be tapped on retroactively when something already is implemented. And what stands out in the commission's approach is its emphasis on practical cumulative recommendations. It provides specific guidance for different actors at different stages of the AI life cycle. So from developers designing systems to operators deploying them. your needs. also contributed its own evidence-based roadmap. A ten-step framework that synthesizes years of research and dialogue. And unlike proposals that focus narrowly on multilateral outcomes, you need to recommendations span all levels of governance, international, regional, and national. And yes, Minoffina walked me through the logic behind their approach. - That Uniter paper builds on our submission to the UNSG in response to the first committee of resolutions of 1929 from last year. And the idea was to summarize everything that Uniter has done in the past few years on AI in the military and security domains to really issue distance that recommendation. So at the multilateral level, what we have recommended was to establish a process to discuss AI in the military domain within the United Nations. The reason why we are recommending this is because of course we recognize the fact that there's a lot going on at the moment, but we also need coherence. I think that at some point, of course, there is value in having different processes and initiatives, there is value and having a multi-stakeholder platform for responsible animal to the main such as re-aim. There is value as well in seeing different specialized processes such as, for example, at the BWC level looking at the convergence between bio and AI because it provides a really focused discussion. So to speak on AI and bio. But then at the same time, amidst all of this emergence and proliferation, almost of processes and discussions and initiatives, you need something centralized in order to intro coherence and to coordinate everything that's going on. And that's why we are recommending for the establishment of these discussions within the UN on AI in the military domain because I think something that we also need to take into account is also somewhat the perspective, the perceived need and almost sense of urgency by some actors to have this kind of discussions. And there's no better place for inclusive dialogue than the United Nations with 103 member states. And I think there's also a lot of value in being and providing this neutral platform within the United Nations for such discussions on AI in the military domain. So that's globally our recommendation at the multilateral level. But then of course, if you look at the 10 step record map, actually most of our recommendations are at the regional and national levels. So they really get as well into the things of leveraging regional discussions. For example, to look at AI in the military domain, developing national strategies on AI in the military domain, but also updating and looking at other military frameworks for governing these technologies and so on and so forth. So yeah, very comprehensive. And we're really hoping that these 10 step road map is something that states can be used as a point of reference for the way ahead. Now let's zoom in on two areas where governance debate has been concentrated and where the stakes are among the highest. First, lethal autonomous weapon systems or laws. So for over a decade, the convention on certain conventional weapons has hosted a group of governmental experts grappling with how to regulate weapons that can select and engage targets without human input. And there's a common misconception here that this is primarily an AI debate. It's not. Now, of course, AI may enable greater autonomy, but autonomy doesn't necessarily require AI. And Julianna Hallouv underberg from the UN Office for Disarmament Affairs explained how this distinction matters. The CCW group of governmental experts are the GGE has been one of the more practical and technical international conversations about autonomy in weapons systems for the past decade. But GGE doesn't actually seek to regulate AI at launch or AI weapon systems as a whole. The systems under discussions may incorporate AI, but AI is not actually prerequisite to their functioning, meaning that the GGE is not an AI governance forum. And what it does is that it focuses more on function so how the system processes information, selects targets and engages them. And of course, the very center of the debate is the question of the required role of the human or human machine interactions, which is where I think the GGE has been the most influential. And importantly, the discussion of the GGE is very much grounded in IHL, international humanitarian law. So the risks that we discussed, they are framed around whether a system can comply with core legal obligations about effects on legal compliance and warfare and not just technical characteristics. Now, all that said, we shouldn't deduct that AI does not at all play a role in these weapons systems or in the discussions of the GGE. So while not all autonomous weapons systems require AI to function the integration of AI still remains critical for achieving some complex high levels of autonomy. So there, I think the debate is very much about increasing integration of AI comes with an amplification of the risks that we see in these autonomous weapon systems. And the concerns associated to them in the first place, especially if there is no regulation around them. And Juliana argues that what the lowest discussions have produced are governance lessons that extend far beyond the battlefield. For example, the concept of meaningful human control, which has been developed over years of expert deliberation, offers a template for frankly any high risk application where humans must remain responsible for consequential decisions. Take human control as an example. This has been a central element in the discussion of the GGE since its inception. And while there's still no agreement on some central questions like what is the extent of human control that is needed or when is it needed or even what terminology should be used to reflect it, the GGE has been able to develop a very rich set of ideas around appropriate or meaningful human involvement in the use of force. So this is why the GGE now increasingly refers to human control and judgment to really reflect the idea that sufficient human intervention requires more than being able to just push a button or intervene in a worst case scenario. It means ensuring that there is legal judgment by a person and that ethical considerations are being taken into account. So with that, the GGE has been able to build some of the most mature language on meaningful appropriate or sufficient human control. And these concepts can translate almost directly into any high-risk AI system. And they can provide a template for defining the role of humans and setting the necessary boundaries. The idea there is essentially really simple AI or even autonomy for that matter can help, but humans must ultimately stay responsible for the decisions. These ideas are also incredibly relevant as AI moves into places like hospitals, transport systems, workplaces, even our own homes. The laws debate has already given us a vocabulary, meaningful, a context-appropriate human control and judgment. And importantly, a way of thinking about this concept. Another example that is also linked to this is a question of the assurances-- AI insurances-- that GGE has been discussing the need to ensure that the effect of laws are predictable, reliable, traceable, explainable, to those responsible for their use, and also the need to prevent or mitigate unintended bias and how to address automation bias. All of those are basically the foundations of AI insurances. And a lot of questions and answers can be translated into this broader AI debate. So while the laws debate is about weapons, I think the governance lessons that we see are much broader. If we use them, it could give us this very, very made set of building blocks on human judgment on control accountability, predictability, that can essentially help ground the wider AI discussion very far beyond the battlefield. And if autonomous weapons represent one governance frontier, nuclear command control and communications, N-C-3 represent another. The idea is that decisions to use nuclear weapons must remain under human control. And here, the re-Aim Global Commission's report singled this out as its only capability-specific recommendation, which was a striking departure from its otherwise technology agnostic approach. So if you were a monstice, explain why. Among the five core recommendations in the Strategic Guidance Report, there's clearly one that I think would jump out at anyone. And that is recommendation number two, which reads, agree at a legally binding level that the decision to authorize the use of nuclear weapons should remain under human control. And why this one stands out is because it is the only one that specifically makes a recommendation at a capability level. Whereas with all of the other recommendations in the report, we were very broad in the way that we formulated the statements with the intention that any state, regardless of the type of capabilities that they have at their disposal, would be able to implement the recommendations. However, we still thought that it was highly necessary to include this among our core recommendations, because it is something that very naturally emerged from the methodology which we used in the writing and the analysis found within the report. And that is that a lot of our structure is based on a use case approach and capability-based approach to AI in the military domain. We focused on two categories of uses, that is non-operational and operational uses. And then we also developed four criteria to evaluate the priority that should be given to these specific use cases, where not every single kid. case, code or even should be regulated at a global level, or even governed through international norms. And so based on these criteria, which just to name a few, looked into dependence of the system on specifically machine learning or considerations about systems of systems, the danger that arises that if something goes wrong in one part of the system, there will be significant ripple effects. We also then arrived at a collection of priority use cases that then the rest of the report looks like. But specifically the intersection of artificial intelligence with nuclear command and control, which is also highlighted in a spotlight box in the report, really stood out as a clear red line that needs to be drawn just for the sheer potential impact. So we had low, medium and high scoring on each of the four criteria that we provided. But if we could create a fourth category, beyond high, this one would score like that everywhere. And so it is also something that simply came up in every single conversation that we had when meeting with the Global Commission. And so it was decided that this is something that we as an independent group of experts really can highlight as if there is to be agreement on at least one specific case and the boundaries of that case that this would be it. And that we are also well positioned to make that statement. And I think in a way, a lot of the members of UC Raim, and especially well, I certainly came to view the Global Commission as a kind of simulation on small scale of many of the considerations and conversations that are happening at a global level in international deliberations. And if we were able to arrive at this kind of conclusion with all of our different views, all our different disciplines and cultural backgrounds, then our hope is that also at the actual, in the real world, so to say, a similar conclusion could also be reached. And in 2025, the UN General Assembly adopted a resolution led by Mexico, calling for human control and oversight to be maintained over command controlling communication systems of nuclear weapons, including those that integrate artificial intelligence technology. And interestingly, one thread seems to connect to these initiatives. The recognition that governance of the military AI cannot be one size fits all. Different capabilities present different risks and different context demand different responses. So the challenge here is building coherence without sacrificing the flexibility that this complexity requires. And the technical risks are equally concerning. AI integration into nuclear systems introduces vulnerabilities that could destabilize the turns itself. And the phase two researcher at the Stockholm International Peace Research Institute Cyphery outlined three dimensions of concern. So the risk here are actually pretty well studied. Let me just briefly share three key dimensions to how AI integration may affect nuclear deterrence and the strategic stability. First on AI and sister integration, there are concerns over reliability, transparency, attention manipulation, so data corruption and human oversight. Human control, human responsibility and accountability is central to a variety of initiatives in military AI governance. But when you look closely, it's not always clear what that actually means. And even if human operators retain authority over final decisions, a critical concern remains how can states ensure that decision makers do not become overly dependent on AI generated recommendations and options. So this is a phenomenon what we call as automation bias. Nuclear balance and the second strike, AI integration, convening to perceptions of technological imbalance that can intensify security dilemmas. The fear of being left behind may push a state to pursue greater integration of automation and AI into its nuclear forces. And this may prompt a nuclear arm state to diversify its nuclear forces or to adopt a destabilizing poster such as long-term warning or lunch under attack to preserve deterrence. And this isn't just about nuclear weapons. AI enabled non-nuclear weapons pose additional challenges to nuclear stability by, for example, targeting an NCC infrastructure. This weapons with strategic effects could be especially attractive also to non-nuclear arm states that face resource constraints. So in some ways, AI present an opportunity to develop asymmetric capabilities that could disrupt or neutralize the adversary assets. And nuclear arm states have also begun to incorporate this red posed by advanced conventional capabilities in their nuclear doctrines. From a UN perspective, the focus is less on whether deterrence holds and more on the heightened risks of use itself. Through unintended escalation, compressive system making windows, and the dangerous speed that AI could introduce to nuclear command and control. What's notable here is that a path forward exists. For example, the United States conducted a fail-safe review in 2022. Several nuclear powers have endorsed human control over nuclear decisions in principle. For instance, we have a November 2024 agreement between Chinese President Xi Jinping and former US President Joe Biden. So the challenge now is translating these national positions into binding international commitments. One prominent thing that would be for nuclear arm states to sit down bilaterally or multilaterally and exchange on specific attributes of AI systems, DC as decibilizing, including from the perspective of deterrence. The state could then consider how such AI-related risk-mete manifests across different nuclear command and control functions and talk about the means to mitigate this. State could also consider elaborating principles specifically for nuclear contacts, but not just ancestry. An additional principle could descend around prudent and responsible development of AI technologies, and this would address the common concerns the both nuclear arm and non-nuclear arm states have about AI reliability and the vulnerability. And at the national level, efforts could be done for conducting independent fail-safe review such as the one the United States did in 2022. The state could also consider implement domestic oversight and AI-specific standards to address reliability through the nuclear life cycle. And to address capabilities and actions, a relevant approach could center on renouncing or limiting the use of AI technologies in certain weapons systems. And in our research, we suggest drawing inspiration from the two-tier model that has been proposed for governing autonomous weapons systems. And this could involve a first step with an overarching principle such as the one we have, certain concerns that are retaining human control over nuclear decision-making. And then the second tier with more adaptable and system-specific regulations and norms, aiming at constraining how and where AI is integrated. And in the short to medium term, there's also a lot we can do with confidence building measures, enhancing and adapting existing crisis communication outlines, or even consider bilateral hotlines dedicated to addressing AI-related operations. And finally, also to back to my previous point on human control, it is really important to keep pushing for explicit confirmation and real information from the P5, and potentially for all nuclear arms states on this front. And especially in today's two political environment, having that on the record itself will be a stabilizing signal. So whether this leave us, well, the governance landscape for military AI is both richer and more fragmented than even two years ago. We have a UN General Assembly resolution that affirms that international low-piles and calls for sustained dialogue. We have the re-emprocess building on multi-stakeholder consensus on responsibility by design. We have the CCW's GGE, grinding through the hard questions of autonomous weapons. We have regional frameworks emerging in ASEAN and elsewhere. And we also have states developing national strategies and military doctrine. What we don't have yet is coherence. Different processes address different slices of the problem with different participants at different speed. And the risk as unity of warrants is duplication or even competition between initiatives at exactly the moment when coordination matters most. But perhaps the more fundamental question isn't which process will prevail. It's whether any process can keep pace with technological change. And if governance always arrives after the fact, are we governing at all? That question and the perspectives of those who see reason for both concern and hope is what we turn to next. This has been Episode 3 of the Map's dialogues podcast. I'm Ulyss Richard and I thank you for listening. Also UN Episode 4.