Episode 158 – AI-powered Investigations with Marta Bo and Benjamin Thorne

(soft music) Just to say that if you are enjoying this podcast and you'd like to give us a bit of extra support, you can head over to our supporters page where you can give us a tip or you can follow us on Patreon and you can download our newsletter and we really appreciate everybody who gives us a bit of extra support there. - Pa' don't kid is a very interesting example because it's being heavily criticized as we all know. - I think what this example also illustrates is how this technology can move from military context to law enforcement and criminal justice context. The way it started to enter Ukraine was primarily for military purposes and then it transitioned also to other uses. (soft music) - No one can act with impudence. - These are crimes that are beyond the pale. - We are talking about human beings. We're not talking about numbers. - We need a court that's as simple as that. - This is asymmetrical haircuts. Your International Justice podcast with Janet Anderson and Stephanie Van Dember. - All right. - Hi Janet. - Hi Steph. So do you spend any time reading those really glossy annual reports that come out from the International Criminal Court? You know the ones from the office of the prosecutor and they get launched at the Assembly of States parties every year? - A boy do I ever. I do read them and I try to parse them for every little bit of information that I can possibly glean and it's not very much. - Yeah well good luck with that. Yeah they're so glossy that they just gloss over everything in my view. But do you remember that there was, I think it was about two years ago there was this line in one of them that said the office of the prosecutor was going to become a leader in technology and they were making a big investment in a particular tech called project harmony as far as I remember and this was a way they were going to improve the management of evidence. Did that? - Absolutely. I actually did a deep dive into it because I wanted to do a story about how they were going to use AI in parsing through evidence. But they, what I remember from it was that it was very general and that they didn't want to tell me very much more in detail about what they were doing. And so I couldn't in a way sell it to my editors to do a story about it. So I tried and failed. - Yeah. - Well here's the chance for redemption. How little we know we will put on show. But yeah, welcome to our Lives as Journalists. They're asking questions and getting only minimal answers. So that in itself has been at the back of my mind for a little while. And then when somebody about six months or so ago mentioned a thing called relativity to me, that's a system that's apparently being used by some organizations. It belongs to Microsoft and put a link to that in the show notes. And that's all about how AI artificial intelligence is being used in connection with legal data. So that started me really thinking about what are the systems that everybody is using? One of the ways that AI is being used, what do we need to know about and how is machine learning coming about? - One of the things I'm really interested in is how do you tag these databases? How much does this cost? How effective is it? They made a big deal of kind of overhauling and modernizing the OTP evidence presentation also during trials. But of course, we also know that now there are issues with Microsoft and the ICC and they're moving away from it. So what are they going to do with these systems? Is another kind of more current point that I'm sure we will get to? - Yeah, we're actually recording this podcast as the second part of our tiny little toe-in-the-water series about artificial intelligence and international criminal justice. - Yeah, I really enjoyed our previous episode where we talked with Jessica Dorsey and Elkashlarts about AI targeting. - So this time we're going to talk specifically about investigations and AI. To help us out, we have a friend of the pod and former podcast producer herself, Martha Boe, of the Assert Institute. Hi, Martha. - Hi, Daniel. - Hi, Steph. Thank you for having me. - And we have a new person on the podcast, but very journalist-friendly Benjamin Thorn, who's from Reading High Benjamin. - Hi, very for you. And thank you for the very kind invitation to participate in this conversation. - The way to kick this off probably is to start with the same way we did our previous podcast to ask, to ask you, Martha, to define what do we mean? What is AI in this context? - That's a great starting point, Steph, because AI is very much an umbrella term. If we don't unpack it, we risk talking past each other. So on your one hand, we have a root-based systems. They operate on a base of if-then logic. So if we imagine an AI system that is programmed, for example, to identify words like attack or orders or weapons in document, and then based on this identification, it would flag a certain document as relevant for a word crime investigation. So this is very different from machine learning systems. A machine learning system in this context would be instead trained on the basis of documents that are labeled as relevant for word crimes investigators or not relevant for word crimes investigators. So it would learn from patterns on these documents. So instead of identifying a keyword, it would learn also that, for example, ordering to carry out a new operation and not only an attack would basically be the same thing. So it would identify such document as relevant. And machine learning systems are very good at pattern recognition and they're much more efficient than rule-based systems. At the same time, they're more opaque, and I'm sure this is something that will come out more as a problem of these systems in this podcast. Now, on top of machine learning systems, we also have LLMs, large language models. So the technology that is behind the charge-bq, for example. And these models, they analyze language, and they provide analysis of legal language, for example. Based on LLMs, we also have an increasing role of a genetic AI, so AI assistants that can help in a variety of tasks. So they do not only react to crime, they can actually perform blunt steps. And this could potentially be very useful for investigations as well. Okay, so there's a lot of different details in there. What's the most important thing for us to understand as to how this relates to international criminal justice investigations? Yeah, I think what is important is to what task in criminal proceedings, these or different types of AI that can often be combined are applied to. So international criminal proceedings, as you know, are very complex proceedings. They involve hundreds of victims, witnesses, hundreds of pages of witness testimonies, video footage, intercepted their communications. And investigators do not need to only prove that a single murder has been committed by any individual soldiers. On top of the underlying crimes, they have to prove contextual elements, as you know, so that crimes were part of what spread or systematics attacks or that they were carried out in the context of an uncomfit. So this is very much linking exercises. You need to link several crimes, victims of crimes in different geographic parts of certain countries. So you have to connect the patterns of crimes and ender, you have to connect individual perpetrators also to high level perpetrators. In this linking exercise, AI can be potentially very useful. So AI is very good at the particular recognitions in these linking exercises and in processing, analyzing, categorizing, huge amount of evidence. These are not just hypothetical applications, but this is how AI is being used right now. So in Ukraine, the office of prosecutor general has been used platform provided by Palantir to integrate different sources of evidence, satellite imagery, documents, so this is what this platform are very good at integrating data. And in the context of international security and prosecution, this can be very helpful, for example, to mapping command structures or to map the patterns of crimes. So Martha mentioned some of the investigators of some of the offices who are working on things with these models and with this AI assisted kind of searching searching through the evidence. Benjamin, are there any other investigators worth mentioning who would you would like to lift out? Sure, so I think obviously the the IECC, like many things to do with international criminal justice, takes up a lot of space and perhaps oxygen and perhaps rightfully or wrongfully. So, but I think in these conversations when you're asking about sort of different types of investigators, we are talking about core international crimes and international justice, but I think this is also overlap to kind of the connected field of transitional justice as well, because there are a number of organizations. And some of that might be in terms of criminal prosecutions, but some of it might be also to document human rights violations that also might relate to core crimes such as crimes against humanity. Can I refer to this in some of my current work as this digital accountability ecosystem. And I can know we're talking about investigators, they are a number of different actors with sometimes shared purposes, perhaps also sometimes competing agendas as well. So, I think when we're talking about this ecosystem of investigators and those who are interested in AI technologies and digital evidence, I think it's not necessarily a homogenous whole, but I think it's perhaps more useful to think about it as these different component parts. One being investigators within the OTP. And, but they're not all moving in the same direction and this ecosystem is more fragmented, perhaps certainly diverse and at times contested. So, in terms of more narrow criminal investigators, we have people like just mentioned that ICC, but also you adjust also UN relevant units such as the double I double arm and the triple I M and up until 2024 units had as well, which is where Karen Khan came from before we took up his posters chief prosecutor at the ICC, but we also have domestic war crimes units in different European countries. Who to varying degrees may be using some sort of AI tools, but then we also have as well, well known, own since organizations and broadly related human rights organizations that have their own investigative teams, definitely some overlap just to give some examples, owns it for Ukraine, that in cats, just accountability unit, although that clothes, I think it was late 24 or 25, these witness air was is an interesting one, particularly in this context as they gather a lot of geolocation data from the better word, but they don't gather it for specific purpose in terms of a tri, well, my understanding is they gather more to gather it and then it could be for a variety of processes, but they quite publicly talk about how they made a choice not to use AI and I was at a talk where someone from air was was thinking last autumn and they were saying that because these central violations are very much a human process, the process of analyzing those that data should also be a human process, partly linking that to dignity, so it's quite interesting how there's a lot of AI going on in this kind of investigation processes by a variety of institutions and organizations, however, there are a few and I think air was is one of those that's kind of resisting in some sense, the use of AI, I also have organizations like eyewitness to a trocity who refer to themselves as a close source investigation organization, another one that I've become aware of relatively recently called earshot, so we often think of digital investigations and the use of AI around the visual kind of what we see whether it's through geolocation data or a social media post and around kind of visual verification of that, but also audio is a really big part of that is organization called earshot that does ballistics for example, so I mean that's kind of kind of interesting component of that and they use AI within some of their work, I think another part of this is just kind of often sits at the margin is in rotations consultants, so particularly human rights organizations who are involved with a lot of this work directly, they will sometimes want a better word outsource some of the analysis to external consultants, so whether we think of those consultants, perhaps or perhaps not as part of who we consider investigators, then we have organizations such as what I refer to some of my writing as activists archives, you may well have know the group, the monic who kind of is number, a lot for archives such as Syrian archive, Yemen archive, etc. And interestingly we think of owns in organizations is kind of this big thing that's happening in terms of documenting and contributing to evidence for possible trials, but organizations such as the Syrian archive kind of predates a lot of the work that a lot of the owns in groups are doing, but groups such as the Syrian archive, I kind of have been for more than a decade, documenting and now using AI within that documentation process to prepare that material for a justice yet to come, and I don't know if we can even include perhaps journalists as investigators, so I think you might agree or disagree with some of that, but I think it is useful to try and nuance what we mean by investigators and also the kind of work that they're doing. I think for this podcast we're probably going to focus mainly on the kind of courts that we follow and direct investigation of war crimes just because we need to narrow it down. I wanted to ask Martha specifically because you refer to that the Ukraine Office of the Prosecutor General was using AI and specifically volunteer for some of their war crimes investigations, which is remarkable to me because volunteer, we talked about it also in our other podcast about a our target. It's been criticized for its kind of deep involvement in the military, so I wonder if there are concerns around some organizations more than others with AI and what you should use as a kind of diligent war crimes investigator, we're talking about that group of people. I think Paronquire is a very interesting example because it has been involved in the science context, the law enforcement context and being heavily criticized as we all know for what its involvement in many of our eyes operation. I think what this example also illustrates is how this technology can move from military context to law enforcement and criminal justice context because the way it started to enter Ukraine was primarily for military purposes. So for military targeting and then it transitioned or so to other uses, but these of course in terms of legal safeguards present some issues because the guard rates in terms of human rights, etc. that you would have in a law enforcement context would be different compared to like a licensed context. But if I can go back a bit to what also Benjamin highlighted, which I thought it was extremely interesting is that because of its inherent characteristics, international crimes involved so many institutions, because they involved just so many communities because of their gravity impact. So ideally, it would be desirable to have more organizations also in terms of how AI is implemented within this context because you would have been standardized procedures for accessibility, reliability checks, etc. Instead the tendency is very much the opposite. Every organization, every national authority or are developing bespoke in AI systems that work within their environment. This is also to protect confidentiality of course because in criminal law context, this is a paramount concern. So we have this tendency to develop AI assistance working only in Italy for example and within the Italian judiciary. And of course one evidence is processed through one tick-tune AI pipeline that categorized, translated, summarized, they may arrive in another institutions with in a format that is incompatible or partially processed or maybe metadata is not available. So this just creates an enormous amount of compatibility problems and the admi stability problems in proceedings for which by definition institution should be cooperating, exchange, evidence as much as possible. Step picked up Palantir, I wanted to pick up the thing that I mentioned at the top of the pod which was this thing called relativity because I'm wondering whether that's becoming some kind of standard, understanding that it's been developed very specifically for judicial purposes. And the context in which it was spoken about to me was me asking where is all the unit ad evidence, all the stuff out of Iraq that was gathered, which you know some of it is going for various trials across Europe. And somebody said to me, oh don't worry, it's in New York, it's been entered into relativity. The problem is that there's no person to push the buttons in order to extract. it from relativity and to send it around in its different formats to the different spaces that it's needed. You need a budget to be able to run the relativity, but the evidence exists there. So I was wondering Benjamin, since Marta did volunteer, do you want to do relativity for us on any other systems that you think we need to know about? So it's relatively, it's also Accenture as well. So if you think about Project Harmony, this is kind of pyramid, if you like, main organizations that are that partnered with ICC. So we have Microsoft, we have Accenture, and then Avernab. Avernab is kind of consumed within both of those. It was partly funded and by Accenture. But then within those kind of free partners of Project Harmony, you do, as you rightly say, a big part of that is Relativity. Maybe it's just helpful, brief, is to say that Project Harmony has free components. It has the OTP link, which is this kind of portal comes under kind of article 15 communications, where in principle, any stakeholders or anyone can upload information. That's increasingly being kind of potential evidence if you want to call it that. So that's why I think we were talking at investigations a few moments ago, why, or actually, like owns and who writes are quite important because they are, they are doing that. So they are uploaded information to OTP link. In terms of Relativity, they kind of, as I understand it, and again, as Janet alluded to it, the very top of the podcast, it's difficult to understand what's happening because it's very secretive. It's my also my my my experience. In addition to OTP link, these two parts, these the Evolt, which is more of the Microsoft part, the the story that people started panicking about a lot when there was the sanctions and kind of access. And it was reported that within the OTP office, they were apparently some people were printing off dossiers because of fear. They want to go access that. That's the Microsoft part. The sanctions that you're referring to Benjamin are the US sanctions against certain individuals at the ICC with Stefan is amazing reporting out of Reuters to suggest that there is actually a sort of Damakles held over the whole institution and who knows. So I assume that's what you're referring to. Exactly. And what I think is increasing, perhaps maybe kind of a web of sanctions that people have fear of auto trip. And so this idea kind of over compliance. Yes, interestingly, that's kind of died down a little bit. Those conversations, it's not something we're not going to come back, but it has kind of settled down a little bit maybe since the very end of last year. In terms of, I mean, actual sanctions being coming out of the US administration. But that was certainly a big issue. And then you have to have either discovery, which is where some of the AI tools in terms of the analysis. And that's as I understand it, where relativity comes in. Relativity as kind of as alluded to by matter, some of the stuff comes from the domestic criminal justice setting. So this is a, for example, relativity tools is used within the criminal justice system in UK, for example, willing to police body cameras also in court video technology as well. So I think again, give up to your comments earlier, Janet, when you're saying, these is this big claim of the ICC will become a global leader in accountability technologies, which was part of its strategic goals as 23, 25. I mean, that's just the usual ICC chest beat informed better word, perhaps. But I think it does also have real world implications. A bit like sometimes it might over claim to do things around victim participation. I think, yes, you can just say it's just ICC being the ICC, but it does make these big claims and people do interpret that. And maybe that is kind of again, over promising to back to old old issues with the ICC. But going back to relativity, they are these connections between the domestic criminal justice system, and that will be in kind of you. So it's not to say that ICC in the future won't perhaps develop its own technologies or that are those, but at the moment and lots of what's happening from my understanding, it's kind of been taken from the domestic criminal justice and being adapted or adopted in the variety of ways, perhaps slightly more bespoke ways going forward. And in terms of master mentioned in kind of military connections, my understanding of Accenture is a beast of a technology company. If you have a look at it, it's always different. It has metaphorically lots of fingers and lots of pies, and they have a lot of contracts in the US military. I think my understanding is also in NATO. So they are, they're involved in this, it also kind of sees that connection between military application and kind of to justice application as well. A few conversations I've had around relativity with people inside the OTP, and again, these are informal conversations rather than formal research interviews, but it is that the systems are used are a bit more fragmented. So if we're thinking around story analysis and disclosure, they are a few different systems and they're not all compatible with things such as relativity. Now I still have, although I understand what you're saying, my very basic question is, what does relativity do, what kind of program is it in should I envision it as a kind of, it's supposed to be like an operating system or like this kind of one stop shop, what Microsoft does with office, is it the idea or with relativity that they offer some kind of suite of programs that you as a legal officer or an investigator could use and it includes, I don't know, a word processor or a filing system and all these things are, is it AI specifically, because I still don't know what relativity does. I understand the court management system. If you have a court case, I know there's all those e-court things and this is, you know, you share documents and all those, but what is, you know, what is, what is the product that relativity is in a very basic sense? It is a kind of e-discovery. So I think that has a variety of tools, how it's used at the iECC, again, I'm not certain from my conversations, it's used to do with things such as facial recognition, object recognition. So if you see the particular object in a image, then it can compare that to a similar object in a different image as well. As I understand it all also has some text-based analysis. So again, in with transcripts, for example, it can then map certain words as well. So those, as I understand it in the iECC, that's what it is, but yeah, to do with facial recognition, identifying words within text and transcript as well. So the iECC use this one best of word, cute language of evidence management platform, but even I think in that process, before we get to kind of what we might think of the conventional analysis, such as facial recognition, there is also, my understanding is there is AI involved in kind of the sorting and cataloging. And then that might potentially create issues, or have issues relating to bias, or also just things if a document's being kind of titled or an odd title, why whoever did it externally, is there issues there as well. So I think AI, yes, in the analysis, using tools provided by relativity, but also the platform that also has AI, and I can really think about that as part of the kind of AI terrain as well. Marta, quite a lot of what we're discussing, sounds me very mundane to be perfectly honest. I mean, in I think we need to put something on the front page of our podcast to say we use AI, for example, for transcription. Sometimes even when I'm transcribing something in Ukrainian, it does an automatic translation for me, which I don't necessarily think the translation is perfect, but you know, thank you. That's good, isn't it? So at least I know I've got the basics of what somebody has to say. So I kind of think of this as like, you know, yeah, that's normal, and it's just like an assistance tool in that sense. But I understand that like the last podcast that we did on AI, we specifically discussed the ethical issues. So are there a lot of ethical perils, let's say, I understood that one of the judges are to meeting that you were at to describe them as ethical perils, Julia Motoc use that term. So what do you think? What are the what are the problems with this? I think there are both ethical and legal problems, because once we have been discussed things so far, for example, are still uses of AI that requires some sort of human verification, right? So we're thinking about facial recognition systems that helps scheme through thousands of photos, and basically at the end of this process, you would, in principle, have also someone verifying the whole process. So for me, the first problem, in relation to the idea of human verifications that we would still use AI with human verification is basically that often to maintain this human judgment and this human capability to understand what these systems are producing and are recommending to us, it's quite difficult because there are proven studies that prove that there are cognitive bias. When we use the systems that might lead to over-reliance on AI output. Now this means that the errors that AI itself can produce such as that for example, missing key information, missing, exonerating evidence, exonerating information that the prosecutor has the duty to investigate might be missed. So there are inherent errors in what AI can do and can produce. And if we couple this issue with cognitive problems and the automation bias issues, these might need to critical errors in investigations, in decision making. So on one hand, AI can help process, process massive amount of evidence and large volume of data, but my core argument is that these tools also introduce serious risk in terms of missed identifications, missed interpretation of data, failures, to detect crucial evidence, including exonerating evidence. So under article 54 of the statute, the prosecutor has the duty to establish the truth and to investigate both incriminating and exonerating evidence equally. So if an AI systems lead investigators to overlook evidence or materials that might be exonerating, this obligation might be not fully met. Now, this is not a fully fledg violation of these fraudules but could still have serious consequence for investigations. Now, if we think about this and if we consider the transparency problem around what type of AI is being used, for which task, for which purposes, for which uses basically AI was being implemented. And if we consider that AI is often difficult to understand unexplainable, it might be extremely difficult for the defense to understand and detect, for example, if something had gone wrong, if something had gone missing, and if there was a violation of the duty to investigate exonerating evidence. So the main takeaway for me is that we should not be rejecting the use of AI but its use needs and spaces around what has been used. And only after that, we can start thinking about safeguards, guardrains, disclosure protocols, exacting, etc. In order to safeguard the rights of the defense. - My question is, can we ever expect transparency from AI systems as a kind of general because they are commercial systems that kind of secret sources is hidden and not public, it's run by commercial companies. It's not an open source situation. So Benjamin, take it away. - I think these are still human decisions whether there is transparency or not, whether it's in the source or in the tools and how it's also being used, it's still always, these are just systems algorithms if we're going to boil it down to the essence. So I think it's always a human decision, the choices made or not made when it comes to transparency and other ethical related issues. And I think a lot of this conversation around transparency comes to legitimacy as well. And if we don't have transparency, it really potentially impacts on legitimacy, legitimacy into interconnected ways is the main way I see it. Legitimacy of the process of how AI is used but then also legitimacy of the outputs that that process produces. So if we don't have transparency of what is that case tools are, how they've been used. And also the algorithms were embedded within those tools and systems. I think it really raises legitimacy concerns. And so I think yes, we can say, oh, these are not open source systems, but I do think to some extent there is a responsibility of institutions such as the ICC selling and not the only ones. But I mean, it is a responsibility for them to engage more with transparency. So these are kind of ethical related questions as I see them and transparency is one of those ethical related questions. And I think they do have a responsibility. I don't think it's good enough just to say, oh, we can't discuss some of this because it could affect investigations. Maybe there are instances where they can't talk exactly about what they're using or the algorithms behind it. There are times where I think they should be talking about that. And I think they hide behind this secrecy. I would just add to that, they are not the only organization that's quite secretive around how they use AI tools. And I think this goes back to, I think I said near the beginning of the podcast, is there is some kind of competition between different institutional organizations who are in some ways directly involved with documenting human rights violations that relate to core ancestral crimes. So that secrecy and lack of transparency goes beyond the ICC. In terms of accommodations I had with a couple of defense councils working in the ICC, one of their concerns is they don't know how the OTP is using AI. So again, it goes back to transparency. So that that is a concern for them that they feel very much out of the loop as well. And that does raise questions to do with procedure and human rights. Also, things we talked about that content itself that is AI generated or modified and then how does institutions such as the ICC own AI tools recognize AI generated content. So again, this relates partly back to something like the OTP link platform where external organizations or stakeholders can submit potential evidence, some of which maybe AI modified or generated, then how accurate can they tools that the OTP is using identify that? I mean, that's one issue. Also saying that Master very accurately talked about how we have this kind of these different organizations institutions as ECC system as I put it. And she was talking about how the technologies they use, they're not always compatible with other organizations and institutions. So that is a big concern. I think another related concern to that where evidence moves between different organizations and institutions is what I refer to as kind of the layers of AI in terms of transparency. Let's say you're a just who are using their relatively new system analyzed some potential evidence and the views some AI tools within that. And then it gets submitted to the OTP, for example. There is not an audit of any kind showing the different types of AI that we use say at Eurojust and at the IECC. So I think we have these these layer ins or layers of AI, but very little transparency about that in specific context of how evidence moves between the cross different related criminal justice, intentional criminal justice, institutions and organizations. And I think a part of that, a big ongoing issue is this lack of a more joint approach. There's also just a very practical sense, duplication of efforts and resources as well that we might have two or more organizations or institutions doing the same sort of analysis, at which I think is problematic at all, particularly in terms of the resources, arm and efforts that these things take to do anyway. I just want to pick up something that Marta said around human verification, which I completely agree with, just to add to that a little, I think there's one conversation around there always has to be human verification. We hear this language of human in the loop. There has to be space for verification, human intervention, et cetera. But my understanding is that some of these institutions keeping with the IECC as an example is sometimes a first layer of initial analysis that gets done with AI tools. Then the human investigator then looks at that. But the human investigator isn't necessarily looking at all of that digital information. They might look at all of the digital information that AI has kind of identified as potentially relevant. But there's sometimes a first layer that a human hasn't looked at all the potential effort. And so I think we're talking about verification. I think for me, AI tools are useful when the human is involved throughout the whole process. And yes, this means it's a long process, but then I think there are risks relating to kind of biases that come within the AI tools, but also biases that potentially become embedded within the patterns of the AI analysis itself. When we're basically looking at digital evidence, using AI tools, but they need to see all the evidence, which some might say is an unrealistic call. But I think otherwise we're having an initial layer of analysis where the human might not see a piece of potential evidence, for example, a piece of evidence that AI has disregarded because it doesn't meet the algorithmal patterns it has. But that might be useful, whether the prosecutor or perhaps for disclosure reason. And I think in terms of transparency, I've asked, after that, has a few conversations around are there any intentions to have a sort of OTP policy on the use of AI and the kind of the ethical use of AI within its work, which I think is much needed? And my understanding, is that there is not. And I think that continues to be, for me, deeply problematic. There's not a willingness to have a policy on the ethical use of AI if the OTP is going to continue to use AI perhaps increasingly so. I think that is saying that should be addressed. All of this I think is to do with transparency, lack of transparency, and also not assume that AI always needs to be used in these processes. I think sometimes they have a very useful role to play, but I think also it's important for investigators to always be asking why use AI for this particular task. We talked about a lot already and then we have touched on a lot of different aspects, but the kind of journalist in me is always looking for the more newsy bits. So I'm really shame on you. I've been hammering away at these fascinating material. And I'm listening to it with great interest, but you go ahead and be a journalist. I want to ask though very specifically about these sanctions because we're still we're talking very specifically about this much of this AI, if not all, is a US base. There is some China base, but that has its own problems with these ICC sanctions. How realistic is it that the ICC and other kind of European justice institutions can keep using this relativity software or these Microsoft based systems and store potential evidence on US servers of commercial companies that are in part beholden to what the US government orders them to do. And I'd say it's not only relevant the ICC, it's also irrelevant to all of the other actors that Benjamin has run through because they're all having to think about whether they use US based stuff. So who wants to take this as a last question? I think some of this is that there is like an appetite appears in Europe to try and move away from some of this. I think it does seem perhaps that and I get perhaps talking within but also slightly outside of the cruel justice context, but I think these things relate in terms of how we might be seeing or as I said the ICC moving away from these things. But there is an appetite to take more risks. I think some of the systems, I guess I hear I'm talking a bit more about cloud systems rather than the tools, but those things are sometimes interconnected anyway. Some organizations and also domestic and regional governments are looking to take risk with untested at scale products because of what's happening. I think also as Janet alluded to, they are a number of organizations that have to consider this, but I think sometimes particularly some owns and organizations who are working specifically on gathering evidence for criminal trials, relating to a trustee crime. They're well aware of the issues with engaging with US and perhaps Chinese tech. However, their priority is to gather and analyse this information. That is their primary objective and sometimes it comes down to really practical things like resources and maybe they haven't got the resources to look and invest at this moment alternatives, which I think I mean, I deal to say everyone should be moving away from US and Chinese tech perhaps, but I think if you're a small scale on the organization, it's a very different kind of calculation compared to the ICC. Just on the ICC and Relativity, a lot of project harm in the whole big part of it was funded by a European commission, I think it's around 7 million euros and also some voluntary donation by state parties. So there is a question there, if the ICC is going to move away from Relativity, how they're going to pay for it. I mean, maybe they'll find a magic money tree or something similar as they sometimes do, but that is a consideration I think, even with a big organization like the ICC is how they would fund the move away from my understanding the contract with Relativity run from 22 and it's due to expire 27. I'm not sure if that's being used and again, maybe one of you know news more about that. And also my understanding is that the money that came from the European commission and voluntary donations from from states was not enough for project harmony, so they had to kind of dip into the kind of the regular budget a bit for that as well. I think there are both at the small scale, there's these different kind of considerations whether to move away from US tech, but also for the ICC, they're also I think perhaps slightly different, but also for financial considerations. But we don't know, I mean, I think it's the bottom line here, when it comes to ICC at least, because it is so closed, we talk about transparency in terms of the tool that just transparency that ICC I think is maybe something we would maybe all to some extent agree upon the lack of transparency and that I think becomes a big issue when we were talking about things around tools which have direct impact on the immiscibility of evidence that could affect the prosecution. Marta, why don't you take us a bit inside the the black box of the ICC or wherever and tell us more about what we need to know. I think because at the moment, higher level, what is important to think when institutions start incorporating these technologies is that they don't really buy a product, I think, but the buy a service and this creates a long-term dependencies. Okay, in terms of also, if we think about the updates, configurations that are necessary and the expertise that must be continuously supplied by AI suppliers to the end consumer. So over time, this really creates structural reliance on private companies. So this is also, you know, like blending what criminal justice system is and who has control over court task of criminal justice and investigations. So of course, one solution could be European technology, technological autonomy. And I think we are hearing this a lot in the defense context. Europe has the TORIARM to develop the AI is now at the core of the RIARMEMENT efforts within Europe. And you could make the same argument for the judiciary, but this would require enormous amount of expertise, a massive amount of investments. So I think to think a bit more on the shorter term, what is needed is internal knowledge, internal capacity within institutions such as the ICC, to have internal expertise, in-house expertise, to understand the limitation of this system and to understand at least the basic elements of how they work. Okay, explainability is a core problem of these systems, but it's basic understanding of what are the inputs, what are the data that has been fed into the systems, how output are produced, so that users have a meaningful understanding of it. I think it's necessary also to kind of decrease the reliance on industry. Especially since we are going more and more towards co-development of these systems and which then requires the relationship of trust with industry and training of the end users, which means ICC staff and investigators on how these tools should be used. We're definitely going to have to bring you both back in because we had another series of questions that we're not going to get to, but we're just going to ask our final question, we're not going to ask all of them. The last one is always is there anything that you have been reading or listening to or watching in the last wee while either within the field or something completely different to get away from it that you would like to share with the audience that you think it'd be interesting for them to know about. Matter, why don't you kick us off? So my final point is since we've been discussing many applications and very limited AI assistance, for me it's just the final comment on how cautious we should be, investigative food be about the use of chatbooks and AI systems because all the risk that we have been discussing so far especially around over-reliance are more and more present with this agentic AI new triangle and judicaries around Europe have started using chatbooks and AI assistance for quite a trivial task like a drug part of the judgment or but this should be much more awareness and training around the use of the systems. I think my final point is about a risk that is often not discussed enough but it is killing essentially and this is highly concerning. To me there are already many studies around how the user of AI assistance basically is killed as and slowly we lose our brain capacity and bring capability to solve even even extremely easy task so I think the problem of this killing should be put there. Yeah there is a wild article from today actually yeah that is very much about how actually it would be much better if a IAS system would tell us how to do things instead of giving us fully cooked replies because these are effects on cognitive functions. And if we think about okay, all your generation of investigators and judges, yes, they can still perform tasks that they've been learning from here. But if we think about young generation of legal officers at the ICC that are just enjoying an institution and and start using these tools without having first build their own knowledge on how to do investigation, how to do the mapping of cases, how to do these things, I think it is very dangerous. What about you Benjamin, do you want to add add in? Just so I think master point about these skill and is like one of the most important things I think going forwards. I've just quickly responded to what I've been reading in this context. In terms of AI and maybe it's come across here in my response, this is some of your questions. A lot of the literature I engage with is the kind of more critical literature on AI, both in the context of international justice, but also outside and then thinking about some of those arguments that's made more broadly around ethics and AI and seeing how they might apply to what we are talking about, international people who are all in justice. So one book I read recently and I would recommend anyone who's interested in AI in this context, outside is a book by Shannon Valer who's a ethics and AI philosopher based at University of Edinburgh and a book, the AI mirror. I don't know if any of you have read it or not, this AI mirror, how to reclaim our humanity in the age of machine thinking and she talks a lot about choices when it comes to how governments and institutions are using AI kind of a design stage and how AI is designed and that's about choices for it to act in be certain ways and these other choices that could be made. So in terms of thinking perhaps critically about some of these conversations we haven't today, that's a book I've read and I would recommend. Which is another way of saying let's be concerned about bias and I think we have spoken about it but not so explicitly during the podcast but a lot can be reconnected to the bias problem so what Benjamin you discussed at the beginning about the bunch of investigators and people involved in investigation. We have also to think about how all these preliminary material that has been documented, collected by NGOs then is also used as training data for these systems and how these then the bias can be essentially fed into the systems and be replicated over the course of the proceedings. We'll just end it by saying thank you so much and sorry for having to cut off the conversation. I mean there's so many aspects to this but really Martin Benjamin thank you so much for coming on and giving up your time and explaining some of the issues to us. Thank you very very much. You're welcome to play you. This was asymmetrical haircuts, your International Justice podcast created and presented by Janet Anderson and Stephanie Vandenberg in partnership with the Hay Humanity Hub. Music is by Audonotics.com. You can find show notes and everything about the podcast on asymmetricalhaircuts.com. This show is available on every major podcast service so please subscribe, give us a rating and spread the word.