Episode 15 | How AI is Transforming Legal Workflows and Document Review

Welcome to Legal Finance Insights, the podcast that takes you inside the legal finance industry. I'm your host, Pip Murphy, CEO of the Association of Litigation Funders of Australia. Join me as our interview key players and cover the latest trends and provide invaluable tips and tricks to accessing, obtaining and using legal finance. We hope that legal finance insights becomes your go-to resource for enhancing and improving your understanding of the legal finance industry. Thank you for joining us today for the next legal finance insights podcast. In today's episode, we're speaking with Lauren Anderson, an e-discovery specialist and director at Kortimentha. Lauren is admitted as a legal practitioner of the Supreme Court of Victoria. Her qualifications include a combined jurist doctor and master of laws from Monash University and a Bachelor of Arts from the University of Melbourne. With over 12 years in the industry, Lauren has extensive experience managing complex e-discovery projects for large-scale litigation, transactions and regulatory reviews. She has provided e-discovery services to law firms, government, global and remote clients, including supporting public and private organisations through multiple royal commissions, government inquiries and class action activity. Previously, Lauren has worked at several nationally and internationally recognised firms, including FTI Consulting, Herbitsmith Freehills and Minterellasson. During her tenure, she managed large-scale e-discovery projects, transactional data rooms and multi-disciplinary digital transformation and migration projects. Kortimentha is a proudly independent founder owned and led advisory firm that has supported some of Australia's most high profile and sensitive projects. Joining Lauren today is Skide Getty from Link Legal Advisory. Skide was directly involved in applying NewX's new AI functionality in a live matter and brings a practical perspective on how these tools are transforming document review under real-world time pressures. Welcome today, Lauren and Skide. Well, thank you for joining us today on our Sleegle Finance Insights podcast. Today we're going to talk about Kortimentha, the development of its application of AI and how technology is transforming legal services, document review and dispute resolution in the Australian market. Before we begin, Lauren, would you mind telling us just a little bit about yourself and Kortimentha and your role within Kortimentha? Absolutely. Thanks for having me, Pip. I'm a director in the Forensic Service Line and Specialty of Forensic Technology. You'd Kortimentha, I specialise in e-discovery specifically, which focuses on identifying, preserving, reviewing and producing electronic data for litigation investigations or regulatory matters. So essentially, I'm assisting lawyers with managing their document obligations for review or disclosure. Can you tell us a little bit about what sets Kortimentha apart from other forensic technology providers in the Australian market? I think one of the biggest things that sets Kortimentha apart is that we're Australian-based with a really strong forensic capability. So that means where a matter involves collecting devices, accessing local data or just needing to be agile, we can get people on the ground really quickly. We've also got a really diverse team, so that includes lawyers like myself, former police officers and IT specialists. And another strength when it comes to our platforms which we'll discuss in a second is that we're platform agnostic. So we work across all major review tools and we let our client needs drive the choice rather than pushing our own preferences. So obviously probably the strongest discovery of forensic team I've worked with. So you've been at the forefront of developing workflows for the new Gen AI tools integrated into the Nuix platform. And we've been fortunate to collaborate with you on this initiative and we're really proud, Sky and I, to have been the first law firm to apply this AI functionality in a live matter. We'd love to explore the journey behind this features development and discuss with you what it means for users moving forward. So I've worked with Nuix Discover for quite a long time, which is now actually Neo Discover. And it was actually the first e-discovery platform that I've ever used. And our clients at Cautimenta, they rely on it basically to extract and display data for review in a way that's compliant with the formal document production frameworks, regulatory protocols and similar things like that. At Cautimenta, we host a substantial amount of data in Nuix and we actually provide the expertise around it. Nuix itself when it comes to generative AI and actually many of their new features, I've taken a measured approach to bringing it into the platform, to market and even to the beta testing, which is a decision that they've sort of explained as being rooted in a commitment to deploying the technology correctly and also to have the right safeguards from the get go. So they've launched their JIA feature in beta and they only worked with a handful of select providers, Quattimenta, among them, although despite being in the beta, I think you'll agree that it was actually a pretty smooth experience. We had our collaboration stood out and by that I mean Quattimenta and Link Legal Advisory was our agility. So I guess, you know, PIP itself and I, we'd already been having these conversations about working smarter and what we could do to leverage technology to sort of enhance that client value. And I think that mindset drove us forward when Nuix reached out for us. So when Nuix came to us with this beta opportunity, I brought that opportunity to you knowing you were open to it and the new technologies and because we'd already been discussing how to focus review and legal expertise. While we know others in the market were running test cases and perhaps focusing on its usage in discovery, we saw an opportunity to use that tool differently from the outset and in a real setting. I think, you know, we were agree that was a mix of innovation and a bit of a calculated risk because we knew we weren't using the AI to find documents. We were using it to analyse what we already had, which meant we just didn't have that risk of inadvertent under or over disclosure. And I think, you know, the attitude was we should test this if it works, huge advantage, if it doesn't, when we're no worse off. And, you know, what was great was like level advisory, you know, you're willing to invest the time and effort. And also Nuix was fully involved as well in the background giving us the support we needed for this testing. Yeah, I mean, I absolutely agree. It was extremely valuable for us and a very interesting exercise. Can you talk a little bit about how it actually works in practice? So at its core, it's a fairly simple process. What it's doing in the background is complicated, but it starts by writing a brief prompt about the issue that we wanted the tool to analyse. And then we pointed it at a document set. The AI then would generate a score, which indicated how closely the document aligned with the prompt that we gave it. And then also a summary, so a word summary of why it did or didn't meet the criteria, completed with citations that cited pieces of the documents where it had got its information from. For our specific approach, quarter-manther and link legal advisory collaborated on crafting the prompts, tailored to the matter and the specific questions that you wanted to interrogate the data for. And then we use those AI generators' scores to rank the documents from most to the least relevant. I think you'll agree that method worked really well. We only really did just prompt the data once, with quite minimal iteration. And we saw the value almost immediately. The tool very quickly identified key documents, accelerated the review process. And the reasons in the cited references are really helpful. It was quite a bit more effective than keywords, which were a bit of a blunt instrument. Yeah, I mean, I agree with absolutely everything you've said there. And I think that's a fantastic overview. And thank you for breaking it down so clearly. We're also lucky to have Sky Getty from link legal advisory with us today. Sky was directly involved in applying this AI functionality for our live matter. So Sky, I'd like to hand over to you to just share what that experience was like from a law firm's perspective. How did you use Nuik's and what impact it had on your workflow? Sure, thanks, Pip. Look, honestly, from our perspective at link legal, the integration of AI into Nuik's was a real turning point in how we approached our document review. So traditionally, when you're dealing with thousands of documents under pretty tight deadlines, it can feel like searching for a needle in a stack. But this functionality allowed us to shift from manual keyword heavy searching to a much more intelligent process. So targeted prompts, for example, allowed us to surface the most relevant documents, which is something that Lauren touched on earlier, which were complete with reasoning and citations. So that level of precision and speed was really invaluable for us. It meant that we could focus our legal expertise where it mattered most, rather than spending hours turning through the relevant material. But I will say that what impressed me most about how intuitive the process felt was once we understood how to frame the prompts, the AI did the heavy lifting, such as ranking the documents and explaining why they're relevant. So for our live matter, that was a real game changer. It kind of gave us the confidence that we weren't missing any critical information and freed up more time for me and you to focus on our strategy and the client outcomes. So yeah, basically in a nutshell, it wasn't just about being more efficient. It was also about enhancing the quality of our review and delivering [BLANK_AUDIO] better results for our clients. - Thank you, Skye. That's a great insight into how this worked in practice and it really shows the impact these tools can have on real matters. Lauren, building on that, I'd love to hear your thoughts on what's next. How do you see this AI functionality evolving over the next 12 months? - Yeah, I think we're gonna see, I think a lot smoother integrations as these types of technologies start to become the norm in the various platforms. Perhaps even exporting out results for indices, creating chronologies for council review, things like that. There are some interesting interfaces as well that almost behave like chat GPT or co-pilot where you can analyze documents and get answers on the fly, which is quite an interesting idea. And those real-time responses I can see being very helpful. I've also seen some prompt validation tools, which I think would be great for those that are iterating through prompts. Sky and I use this project co-pilot to help refine the prompts that we put into New York's discover and that worked quite well. We'll also see the large language models themselves. They're continually improving and that's the technology that these tools are running off. So that's fantastic that their output continues to improve. And finally, I think the validation tools are gonna be really, really important. In our case, we didn't use the tool to sort of rule out any documents, but if you think of something like discovery, it's gonna be increasingly important how to vet those outcomes and address the prompt biases when they arise. - So do you have any advice for people considering using generative AI and their matters? - Yes, I have a few key pieces of advice. First and foremost, is security. You absolutely can't be copying and pasting client data into public tools like chat GPT because that information actually goes out into the world. So doing that can immediately breach certain obligations. For example, the harm and undertaking. So if anyone's gonna be using generative AI, it really needs to be in a controlled environment. And if you don't have a robust IT team or you're unsure whether your use of it is locked down, it's certainly something you need to check. Second, it's really important to check the relevant practice notes. So Victoria and New South Wales, for example, have guidance on AI use and New South Wales in particular, has some prohibitions where you shouldn't be using AI and not allowed to. So it's really important to make sure that you're operating with those boundaries when preparing data for quarter, even in your preparation for things like expert evidence. Finally, I think quality assurance as well. We need to be thinking about that any time where using one of these tools, what do we expect from the tool and how will we verify it? For some, that looks like some internal policies around AI use. For example, using it to draft documents, you should always be proofreading the output and verifying the citations. In a large scale review, we also use techniques like data sampling when we take a look at maybe what's left behind and get some metrics out of that. Tools like New West Discover are really helpful 'cause they've got those tools to use. But also the LL Large Language Model tool that we use, the Gen AI, that provided citations for the reasoning. And I know that Sky actually checked those citations when she was reading those summaries and also read the content of the documents they were generated off, rather than just taking it at face value. And that's a really big advantage over some other tools that can be a bit of a black box where you don't know how exactly the output was generated or what it was leveraging us. So, in short, I think really making sure you're in a secure environment that you're compliant with the practice notes and that you have a robust validation process in place. If you have those three things, you'll generally be on the right track. - Yeah, and you talked a little bit there about sky checking the citations and reading the content rather than just taking at face value. And I think we've had some experiences in recent times with other AI products where there has been, the hallucinations of information and in particular where we've created chronologies and the AI has just made up huge slabs of information that just didn't exist. And so we've learned that you have to be very, very careful with that quality assurance program that you're talking about to make sure that everything is checked and triple checked before you actually start relying on it. So I think that's really important. So following on from our last episode, which was with Phil Smith of FTI Consulting, we discuss the growing use of AI and litigation, what the biggest challenges have been in the last five years, for example. But looking ahead now, how do you see AI shaping the legal services landscape in Australia over the next say three to five years? What developments are you most excited about and also concerned about? - Yeah, firstly, I think it's a really exciting time and I'm definitely seeing a very real shift in what I'd see as a client led push for AI. Even a year ago, it was sort of us introducing and sometimes pushing these tools as providers. But now we're finding law firm clients and their end clients are really proactively asking for AI solutions. And that's quite a big change. Now if we do take a step back and convert it to the last major AI wave, which would have been TAR, which is technology assisted review, where I could sort of say the adoption of that was very much necessity driven. So it wasn't so much, I want to use this tool or I'm finding a way to use this tool so much as I've got a million documents and I have to review them and I have to adopt this technology 'cause that's the only way I can do it. I think now we're sort of seeing a mindset of, wanting to eliminate low value document review so that lawyers can really focus on applying their expertise and seeing AI as the tool to do that. Looking ahead, I think we're gonna see deeper integration between eDiscovery providers and legal teams. In this particular matter we worked on, I had a great session with Sky where we were finding the prompts together and we really combined that legal and that technical expertise and it was really effective in the results we got from the AI. And I think contribute to us not needing to prompt too much to get the results that we wanted. I think that kind of collaboration will become the norm and it's really exciting 'cause it makes the technology smarter the process more efficient. And of course I think we'll see some of the risks, well the risks be realised. We've already seen some of those in recent times where people are citing hallucinated case citations and I think obviously that's a risk that's kind of well known but I think another one is probably case theory bias or prompt bias which is maybe training an AI or prompting an AI to find everything that supports your argument and overlooking documents that challenge it. So that could have quite serious implications if people don't account for that type of thing. And I touched on it before but the other area we're gonna need to watch the security. There's a really strong appetite for new tools feels like there's something new out every day but not everything secure or it's not purpose built say like new Excel or relativity. So if someone sort of rushes to onboard these sort of shiny new apps or load sensitive data into them, you know we could see instances where people exposing themselves or their data to vulnerabilities that they may not appreciate. Basically I'm really excited about the potential for these smarter workflows and particularly that closer collaboration but I am cautious about that bias and security risk. It's definitely gonna be an interesting few years. - No doubt about that. Innovation in legal tech moves fast as we know but complex regulatory and ethical considerations often accompany it. How do you and the team equipment the strike the right balance between pushing the boundaries of what's possible with new technologies like this new AI functionality but also ensuring that compliance, data security and ethical integrity. - Yeah this is absolutely core at what we do. Especially when you think about the volumes and the sensitivity of data that we handle. We also do assist government on certain matters as well. So there's those sorts of obligations that arise. At Quattiment though we're ISO compliant and we have very robust security protocols for any data we intake and host. And when we're deploying tools like this like AI they're operating within the secure environments about platforms and not in open systems where the data can disappear into the ether or be easily infiltrated. We also take a really proactive approach when we are using generative AI tools. These tools aren't just switched on available to users in a way they go. We make sure that users understand any risks of the tools they're using that they know that and they're proactively making a decision. Yes, I'd like to use this. For us the risk of those because we are in a secure environment isn't the data exploitation although of course we're aware that legal data is always a prime target for bad actors. But when we're talking document review and disclosure usually human error, poor prompting or misunderstanding the process or weak quality assurance those are the things that we really keep an eye on and guide our clients through. So I guess if we're encouraging the AI which we absolutely are, we're also encouraging validation at the same time. So if we're using AI for your discovery, if we're using it for review, how we demonstrating that it was an acceptable methodology to discharge our obligations to the court, do our lawyers understand enough about it? They could include it in an affidavit confidently. Are we prepared to grow so far as defended in court? And I think for us if the answer to any of those questions is no, then we're back to the drawing board. And that's really the balance. We're pushing those boundaries, but ensuring as a first step that that compliance, that security and that ethical integrity is there. So the innovation's absolutely exciting but more important for us as always is that it's defensible. Yeah, and I suppose that's the really hard balance isn't it's crap. having those new and exciting opportunities and trying to, as you said, push the boundaries, but doing it slow enough that you can make sure that all those security issues, ethical issues, compliance issues are all addressed as part of the process. So I think it's certainly a link legal. We took the view that it was very important to partner. We had called a mentor to help us with all of those issues and it was certainly an exciting and interesting experience using that AI for the first time. So thank you for the opportunity for that. And thank you for today, obviously, for you, Lauren, and for you, Sky, for your time and your insights today, talking about new X and explaining to the listeners how that all works. Thank you, Pip. Thanks for having me. Thanks, Pip. It's been great to be part of the discussion. Hi there. That's a wrap for the Legal Insights podcast for today. We hope you have picked up some useful tips and tricks and enjoyed listening to all things legal finance. If you want to continue the conversation, please reach out via email or via the website, associationoflitigationfunders.com.au. We would love to discuss what you are seeing in the legal finance industry and what we can do to enhance and improve it together. Thanks for listening and tuning next week for another fabulous conversation.