Bert Vries - Lessons from one of legal's biggest AI deployments

So Harvey, two and a half years, compared to Harvey today, it's like a Fiat 500 compared to Formula One car Ferrari. My name is Hinder Bandsman and you're listening to my conversation with Bert Fries recorded during the Legal Week Europe in Amsterdam. Bert is director of Innovation, Knowledge and Technology at CMS Netherlands, where he leads the firm Digital Transformation, shapes the innovation agenda across CMS globally and drives the standardization and automation of legal surface delivery. Today we will talk about coordinating AI adoption across jurisdictions, why this wave of generative AI is fundamentally different and how law firms should measure the real return on their AI investments. We started back in 2023, so we are two and a half years down the road almost three years. So it's the longest project, if it's still a project that I've done. And it's still ongoing. Before Gen AI, most of our projects were a little bit like, yes, we still have some users involved, but now it is not that the users sit next to us, but we sit next to the lawyer. And that's quite different. If you are Columbus and you want to find a way to the West instead of selling to the East, so you are exploring the uncharted areas, then it doesn't make any sense to think a lot about what could happen. So you need to test it, you need to sell on the ocean, know where the drift winds are and explore it. Take a little bit of risk also. I see a lot of other firms, law firms who are doing a lot of analysis and they sort of stick in their analysis parallel. They analyze ten tools and they wait and they don't do anything. It's a little bit like compounded interest. And that's also what you do with learning, so step in in your boat, sail out and experiment, her other experiences of your colleagues and grow. And you grow faster if you go together. Bert, nice that you're here. Thanks for making the time. Yeah, thank you. It's nice to be here. You just had your talk on stage? Yes, it's correct. How did it go? Yeah, I think pretty good. So it was, I had the opportunity to talk a little bit why it is difficult to get going with the Genai projects in organizations, what are the main reasons that these projects fail and also some tips how you can kickstart initiatives and how you can take care that you are not at the wrong side of the border and that your projects are fainting. Yeah, and if we talk about projects with CMS, you just did a very big project in a sense of rolling out Harvey. Harvey is of course a legal AI tool and you picked Harvey as the one with CMS to work with. Correct. And you need to roll it out. But in the firm with a lot of lawyers, they get a new tool and you need to learn how to work with this tool. Quite a job. It was certainly quite a job and I would say a little bit also what I said in my presentation. This is a typical type of project. So is it still, we have the discussions, are we still in the project phase or is it business usual? And that's what I was also referring to normally at the end of the project, you hand over the keys and the project team members, they go to the next project. But this journey is already two and a half years, we started back in 2023. So we are two and a half years down the road almost three years. I mean, it's not going. It's not stopping. No, so yeah, we rolled it out to all our lawyers. Also, internationally, we rolled it out to the lawyers. It doesn't stop there because the project evolves, but also we find more and more interesting use cases. And what kind of use cases did you found where you were like, okay, this is very interesting also to use Harvey with. As I said in my talk, we started doing the generic use cases like analysis of dog commands, summarization, etc, etc. And at a certain moment, we said to each other, okay, there's now the time to show you switch from, let's say, the horizontal term wide approach to a vertical approach where we are really focusing on the practice area groups. And you got deeper in those practice areas? Yeah. So banking and finance will use Harvey in a different way than, let's say, labor law or the corporate them in 18. So by diving into these practice area groups and what they are doing and how they are doing, we also see different use cases. Sometimes you see use cases which are used in multiple areas of our firm, but a lot of times mostly the use cases are very specific for the type of work there. And I can imagine that in your role, that's also having a lot of individual conversations with those practice groups. Hey, we now have a very powerful tool that's to do a lot, but for your practice, these kind of function are the most interesting, for example. Yeah, it's dead, but also we are getting closer to the lawyer. So let's say before Gen AI, most of our projects were a little bit like, yes, we still have some users involved, but now it is not that the user sit next to us, but we sit next to the lawyer. And it means that we need to understand what type of work they are doing and how they are doing. And then we can align the possibilities of the tools with their daily flow of work. And how do you make sure that you really understand what kind of work they are doing? Talking, talking a lot. Yeah, so it's really being honestly interested in what they are doing, not being judgmental, but open, how they tell, I've these type of clients, I do this type of work. And it's not only that we talk to with the partners, we talk with all different layers in the organization because partners are doing different stuff than let's say the more junior people. Yeah, do you see a big difference there? We see a difference of course in the work they are doing, but also in the usage. The higher you come into the organization, then it's less delivery and it's more about quality assurance. And so it's not only the pyramid which is taking care of the leverage, but also the delivery. So you have a lot of young people doing delivery work, but it's also the other way around the pyramid is quality assurance. So a partner will use Harvey Les for delivery type of work as he or she is more focused on quality. And I think partners, if they make an investment and you make an investment, that's a firm to say, okay, now we're going to do a worldwide rollout. It's a big investment. They also want to see a return on that investment. Certainly, certainly. How do you measure that? Because I think within one year legal AI tools, they went, they skyrocket. I mean, they're all growing very fast. We now have a lot of firms who are in the beginning area of their subscriptions and partnerships with those vendors. How do you make sure that you find where that return of his investment is coming back? Yeah, so our initial sort of calculations were more focused on usage. So we first need to get the people going. So are they using the stuff? And now we have found that using it? Yeah, yeah, using it more and more. And I have to say, it's quite amazing that we are now in sort of tipping point where even skeptical partners are saying, this stuff is so powerful. We saw a lot of change in the last two and a half years. So Harvey, two and a half years compared to Harvey today, it's like a Fiat 500 compared to a Formula One car Ferrari. So it's a huge difference. There's a funny example if you talk about this where they had a wheel Smith video and they made it with AI, not with Harvey, of course, with others. So, and then you have the video like three years ago or four years ago and then every year again, they make the same video. And now we have Hollywood quality already. And it's 2026, but it's crazy how fast it's going. It's amazing. If you create a picture, so I don't create videos, but I created a lot of pictures and people really have five fingers. And we lost a lot of fingers in the previous versions of AI. But coming back to your questions, yeah, they want to see a return. But they also know that there's not a quick return. So it's not about saving euros. It's about changing behavior of the people in the organization. And that takes time. We need to change the behavior. But we already have some anecdotal evidence where we are saying, okay, in the historic way, let's say the 100% manual way, we spend so much time for this task. And if we measure it now, so do a sort of a B testing. We see quite a difference. And it's not every task is the same. So in certain tasks, you see a lot of productivity improvement, all the tasks you see less. Which task do you see a lot of improvement? It's the usual suspects. So if you are doing a lot of repetitive work or you do a needle in the haystack kind of works, you discovery type of work. or we had a matter where we needed for certain cases. We needed to refuel 10,000 invoices to see certain differences in those invoices. Yeah, that's not the type of work lawyers are saying, oh, that's why I learned law. - Give me those invoices. Oh, give me more. So, but this type of work is now really becoming much more easier. Those are the use cases focusing on, let's say being more efficient or I always like to phrase is more productive, but there are also use cases where lawyers see that AI helps them to be more creative, to be more complete. So, if you are saying, okay, I need to make a plea note and this is sort of the structuring of my story and these are the arguments, then you can also validate which Harvey or whatever tool you're using. What other arguments could be valid? And if these are the arguments, what could be the counter arguments and could we create counter arguments for the counter arguments? So, that's more on the quality of services we deliver. - Do you also get back from lawyers that their work is more fun when using these tools? - Yeah, if you are saying, I like to refuel 10,000 invoices, then it's not fun to use Harvey, but if you think this stuff is really annoying then yeah, I'm always saying sort of, if you sort of create an abstract perspective of the type of work I'm doing, then they are gathering information, analyzing information and advising the client. And we see a sort of shift of this, the first two categories being reduced in time, which means you have more time to invest in quality. Of course, it's dependent on the client. So if the client we are doing, let's say a procedural work and a client is priced sensitive, then the client is saying, oh, this 20% of the total time invested in advising is good enough for me. So just give me the benefits of using AI for gathering information and analyzing information. But on the other side, if you have clients who are doing sort of brain work, you bet your company type of work, then they're saying, oh, this is interesting. I want to reinvest the free-up budget into better quality. So in between those ends, there's a lot of different positions. - But it's still very client centered. - Yeah, the client defines what they want, how they want, so what type of advice they need. And for certain type of work, the client says, this is good enough. So if it's good enough, then you don't need to do more work. And then we're saying, okay, it's great. If this solves your problem, then we have a happy client. In general, happy clients are coming back. Instead of them looking to the invoice, they did so much work, which is not valuable in solve. It's too overengineered to solve the problem. - But this is a discussion, which is going on very happily right now in the market. The billing model of law firms. Of course, we know the original billing model. You work one hour, you write one hour. And now we have AI, you can do work faster. So people say, okay, we need to skip the whole billing, billing hourly model. - Yeah, I'm more a fan of sort of nuance approach. Although I love technology, I'm not in a binary mindset. So it's not like we only do time and material or we only do a fixed price. I think different problems. So if you do brain work, if you do gray hair work or procedural work, are asking for different solutions and also different pricing. So it could be the case that for one case, we do 100% time and material other case. We can do fixed price because it's very easy to sort of estimate how much time we need for that. - But that's quite a change for lawyers. In the sense of they need to also see their work maybe differently also from a pricing model and then think, okay, I'm not doing this work. Then we have this pricing model and if I do this work, then I have that pricing model. - Yeah, so of course, doing one approach, one pricing approach is mentally easy. - You see. - Yeah, so of course, but it is not a big shift. And we are helping our lawyers to understand what the different pricing models are. What kind of situations they can choose for what type of pricing model. And we have tools for that which help them to price it in a smart way which suits the situation and at the end makes also the client happy. - So technology is also solving that problem? - Yeah, we are supporting our lawyers. So not doing on the back of an envelope calculation, but a sophisticated way of pricing. And if we look at your own career, you're already quite a long time, you're active in this field, also the field of technology. You have seen the internet exploded. You have seen other things exploded and now we have this whole AI movement. And we are today at Riego Geek. We see all these young vendors, these young people, this fast entrepreneurs, people want to grow, they're working very hard. Is it giving you the same feeling as when the internet exploded say, end of the 90s? - Yeah, back in 2000, I was 30 years old. I was still a youngster, but it was a vibrant period. And yet a lot of startups, a lot of young people are saying, okay, we are going to conquer the world, we are going to disrupt this industry, this type of work. And so there was a lot of energy flowing. I remember that we were almost partying every week, somewhere in Amsterdam and it's certainly the period which I drunk the most champagne. So it was an exciting, exciting period. And of course, in time you always remember the good things, there were also some bad things, but in general it was a very vibrant period. And I'm saying to my colleagues, it feels like I'm back in 2000 again. I'm older, I'm more gray hair. - You don't go to all those parties anymore? - No, I have kids, so I need to take care of them. - I'm always in pain every day. - Yeah, and that way it's a little bit different, but I see the same energy, I see the young people who have dreams and want to change, let's say, an industry which is not very mature in digitalization. - And if we look at CMS again, I mean, you're not quite far in the whole process, you just said me, took a two and a half years of experimentation, of working together with different vendors testing. Now you choose Harvey, but we still have firms in the Netherlands, but also outside of the Netherlands, who are way more at the beginning of that process, and you learn a lot of lessons in those two and a half years. Can you give a few lessons for firms that are way more at the beginning of that whole process of maybe picking a vendor or how to make sure that you pick the right one? - Yeah, what I lately say quite often is like, if you are Columbus and you want to find a way to the West, instead of sailing to the east, so you are exploring the uncharted areas, then it doesn't make any sense to think a lot about what could happen, so you need to test it, you need to sail on the ocean, know where the drift winds are and explore it. - Take a little bit of a risk also. - Experiment, that's what I'm saying, and test something and what we say, you want to do not, you don't want to put everything on black or red, so we call it no regret moves, so you test something, and if it works then great, you test another step further down the road, but it's certainly a period where you need to experiment. It is, I see a lot of other firms, law firms, who are doing a lot of analysis and they sort of stick in their analysis parallel, - And they analyze 10 tools and then they wait and they don't do that. - Yeah, because the Mega-N-A-I is coming to the market. - I know, you're launching your own firm. (laughs) - And then that could even be a better tool, so why invest already now in an existing tool? - But that's of course, there are so many options that maybe people also have to feel like, yeah, but this tool maybe is better, this tool maybe is better, and then they don't pick. - Yeah, I have to say how we choose Harvey was, in a way, quite simple. I filled in my email address on their website, and it started with somebody from Harvey calling us, and we were just starting to do a pilot, so it wasn't like in that period of time that it was all crystal clear, which vendors were the best or what you needed to look at. If you would select a CRM system, you have a long list of functional requirements, technical requirements, there were no requirements, the technology was new. So we just sent an email and somebody picked up the email and responded to us, and we started experimenting with Harvey. It's not only Harvey, so we experimented with a lot of tools, but Harvey was one of the first, and people became enthusiastic. People said, I really love it. it's it's it's apps and value. you to the type of work I'm doing. Just start experiment. Yeah, it's a little bit like compounded interest. And that's also what you do with learning. If you can build on every learning, then I would say, and also if the group is getting bigger, then in general, a group of people learn faster than one person. And that's certainly what I believe. So step in in your boat, sail out, experiment, her other experiences of your colleagues and grow. Thank you, Bert. And enjoy the rest of Ligelgeek. I will. Certainly. Thank you, Edith.