What Does It Mean to Be an AI-Ready Lawyer? Farrah Pepper and Kirsten Talmage on AI Competency

and the one thing that resonated throughout that group was a lack of a shared understanding of what AI competency or AI fluency means not just for individual attorneys, but also for law firms. And what they were seeing was that attorneys would go from one firm to another with very different understandings of what it meant to have competency in AI. So based on that, we decided to really evolve this competency framework to be something that could be applied, whether you were in a law school setting, whether you were in a law firm setting, whether you were in a legal group within an organization so that it could be this really shared language about what having AI fluency really means. This podcast is presented by the Practicing Law Institute and the American Arbitration Association, bridging world-class legal education with a global leader in dispute resolution. Welcome to AI in the future of law, where we explore how artificial intelligence is transforming the legal profession. I'm Bridget McCormick and I'm Jen Leonard. In each episode, we'll explore how artificial intelligence is impacting the legal community. Join us for thoughtful conversations and practical insights to help you navigate what's ahead. Hi everyone and welcome to the AI in the future of law podcasts, the podcast where we explore all of the fascinating dimensions and changes happening in the world of artificial intelligence and discuss what they might mean for the legal profession. I'm your co-host Jen Leonard, founder of Creative Lawyers. Here as always with the wonderful Bridget McCormick, president and CEO of the American Arbitration Association. And we are delighted today to be joined by two fantastic guests, Farah Pepper, who is legal innovation partner with Harvey and Kirsten Talmadge, who's chief strategy officer with Practicing Law Institute. Welcome to the pod, both of you. Thanks for having us. Absolutely and we are going to learn all about the AI competency framework that PLI recently released and which Farah Kirsten and myself had a role in working on. But before we dive in, in every episode, we ask our guests to share their AI ah-ha's, what they're using AI for in their personal or professional life that they find particularly interesting. So I'm going to kick it to you Farah to share your AI ah-ha to get us started. Well, here's my ah-ha and I think it's kind of a small one which is sort of the point. I do a fair amount of presentations and talks and get to go to a good number of events, which I love doing. And I take very seriously that if I'm a presenter, I want it to have my voice, I want it to be material that is interesting, that give people some sort of ROI nugget. So I have my own way of putting together materials. And it takes a while, at least in the before days it did. And so my ah-ha has been leveraging generative AI as a thought partner and as a style designer so that as I go into different sessions, I can do the part that I love which is thinking through what I want to convey and how I want to do it and have this partner who will basically take things like slide formats and do a lot of the messy work to shape my ideas. And then once I put together something that I feel good about initially, which takes a fraction of the time that it used to as a cumbersome human working alone, I then also use it as a sparring partner to try to think about what might be the hardest or most interesting questions I would get and to be ready for that. So I've basically turned generative AI into my prep partner for things I have to do. Very cool. I love that. And definitely I know Bridget and I both see a compression of time using AI to get ready for things like that right Bridget. It's definitely like part of my day-to-day workflow now that I don't know if someone told me they had to take it away from me, I think I would find a new career maybe. I don't know. I don't want to find out either and I guess that's the nature of progress, right? We can't imagine the world before this innovation came about. It's very cool Farah. And Kirsten coming to you, what are you using AI for these days? Yeah, so taking Farah's example and putting it on the macro level, if you will, my team has been working with our content division to streamline the way in which we create our program. So I think many people are familiar with PLI's multi-hour to multi-day programs. They actually are quite lengthy to produce. So it takes about a six to nine month planning period to plan these programs. And so we've been building orchestrated agents that actually reduce that program time substantially so that we can now go to speakers. So for instance, if Farah was speaking on one of our programs, we could go to Farah ahead of time and say, look, we have all of these assets already produced for you, look through them, review them, put them in your own words, but we've done a lot of that work that would take the constant back and forth over the six to nine month period. We've reduced the actual time to create those assets to 15 minutes to an hour and it gets us off to a running start. And I should say this isn't the experimentation phase still, but it's been very interesting to build these agents that can really do a lot of this work for us and take a lot of the burden off of our speakers who are all volunteers. That's really cool because it's not only saving yourself six to nine months. It allows you to kind of totally rethink how you go about recruiting speakers and even programs is my guess, right? Like you've opened up a lot of room for creativity that probably was cramped when you had that long planning timeline. That's awesome. That's exactly it. And it allows us to use our human experts to the highest their capabilities. So it removes that road work as well. I bet it's easier for speakers to say yes as well when you come to them with that. We hope so. We're still in the experimentation phase, but yes, I think so because I think a lot of the time when we have speakers who can't participate in a program, it's oftentimes because they can't set that timeline into their schedule. Now, Kirsten, I'm so into what you're talking about and I love that we're already kindred spirits. I shouldn't be surprised, but as someone who has over the years spoken on my share of PLI programs as a volunteer, I could not be more jazz to hear if this is the direction you're going because it is actually a commitment of time and resources. And I expect that this is going to make everyone so happy. I love it. We've already created and produced innovation already. We're not even through AI a haze. Bridget, this conversation's making me think of when we were teaching a class together in law school about AI and how difficult it could be because every time we got together to plan for like a fall class six months away, we just gave up because everything would be different by then. It's impossible. I feel the same way with ABA panels. ABA panels want you to give them the slides 18 months before the panel. And when you're someone who speaks about AI, I just can't do that. So you're going to have to find somebody else who is dumb enough to give you slides 18 months before I'm speaking. Yeah, they're like this exciting new development called chat GPT. Well, I love those AI a haze. They're very cool and they will make all the work that we do better in so many ways. As you both said, it elevates our ability to add the human element to it. And so today we're really excited to have you talk about our main topic, which is how do we build those skills and emphasize those human skills in a changing world and get some clarity on what actually goes into being competent in the AI era. So we're going to talk about an AI competency framework that PLI has released and it's the result of coordination across experts from across the profession. But Kirsten would love to start with you and hear a little bit about that origin story and what the AI competency framework actually is. Sure. So to go into its creation a little bit, it was a very I think intentional collaboration amongst three different groups. So we have PLI's Innovation Council, which Ferris member and Jen you share. It's comprised of experts in the field of innovation in the legal industry. And then there was a second group, the professional development consortium and their board, the third group being creative lawyers. So these three entities came together based on a workshop that was held in the late fall of 2025 and we sat down with the board of the PDC and asked them really what was on their mind? What could we help with just very blue sky conversation? So based on that, we decided to bring these three groups together and we had a series of workshops and sessions and we iterated off of what the feedback was and really evolved this competency framework to be something that could be applied whether you were in a law school setting, whether you were in a law firm setting, whether you were in a legal group within an organization so that it could be this really shared language about what having AI fluency really means. And so in the end, what it is is a framework that has five pillars. It has 20 competencies under each of those pillars and it has three levels of proficiency. And when we were making it, we very intentionally did not attach a certification to it. There's no linearity. You can be fluid in one of the pillars and lacking fluency in another. It's really so that you can take it under or a PD person could take it and apply it to their group. of attorneys or to their firm and understand where you are in each of the pillars and how to get to that next level of proficiency. That's so needed. I'm really excited to learn more about it and it feels like the time is so right for it because so much change in the industry right now and Farah, you're kind of an avatar for that. Like you started working on this framework when you were in-house at Marsh, right? And then you finished it now at Harvey and those are two different jobs, I believe, although you tell me, I think you probably do different things, but they probably give you great perspective on why the framework is useful, especially now. What is the fluidity that we're seeing in the legal profession right now? Mean for the moment we're in and why the framework is useful. And then secondly, separate question, what skills do you build working at a legal tech company that are different from those you built practicing law in-house? I love this conversation so much. I'm an accidental avatar. Let's put it that way. So yeah, I did. I started on this framework and joined the PLI Innovation Council last year. It wasn't that long ago, but at the pace of change, the velocity that we're seeing in the market, it actually is a long time ago. A lot has happened because when I started, I had been a long time in-house council, most recently at Marsh, which is a fairly large organization that had more than 600 members in the legal and compliance team, which I was part of as Chief Legal Innovation Council. So from that perspective, I joined. And by the time I left just a few months later, I was in my current role, which is a legal innovation partner at Harvey, which is a different role, but definitely draws upon a lot of the same skills. And what I love about this is that I'm learning and I'm growing and I'm seeing that law firm in-house and tech companies are not really three different careers anymore. They're three different vantage points, three different perspectives. And the more you see of each of them, the better your judgment gets, just your legal judgment. That's been my experience thus far. So I'm only four months into my Harvey role. And that role involves being a strategic advisor to Harvey's customers who are on their AI journey, being a strategic advisor inside of Harvey, bringing that in-house perspective. And I also once upon a time spent over a decade in big loss of bringing that to bear too. And also getting a chance to engage in things like this framework because this is the moment for us to all as a profession figure out how are we going to teach this? How are we going to learn this? How are we going to shape the next generation? And I think that in some ways the framework is so useful because we don't know what to call people anymore. Like we're hiring people now in these roles that we're just making up titles for because they didn't exist six months ago or a year ago. And they're straddling tech and the GC's office in-house in my organization. And so we're just like making up job titles. I don't know if we're doing a good job or not. But in some ways we care more about the competencies than we do about like the name of the job, right? That's so dead on. My last few roles have all been made up. They were the best. Why? Because they come from a place of need. It means something is needed that wasn't there before. And I was basically on with a community from a law school just a week ago. We were talking about this kind of progression and people want to know what does this mean for the future. And my position, which I believe heartily, is there's never been a more exciting or better time to be in legal. Because my job that I'm in now didn't exist five months ago. And there's going to be plenty more that didn't exist five months from today. Yeah, it's really cool. And we were talking about how I started my career in legal education as a career counselor. And 10 years ago when we would have alumni call who were thinking and making a move after only having been somewhere like two years, we would say like absolutely you should not do that. Like it's a major red flag if you're bouncing around. And now it's just like everybody's bouncing around all over the place. And Farah as you were talking, I was sort of thinking about the council itself and how many changes have happened in that group, including one of our talented colleagues, Jen Reeves, who's on the council who was drawn to Bridget's leadership and is now part of Bridget's team. So I agree. Like I think it's so freeing and so fun to be able to take advantage of new opportunities and not be constrained by these like mindsets that you can't do things when you really want to be doing things for old fashion reasons. So Kirsten, you talked a little bit about the challenge that the framework is designed to solve this lack of shared language around AI fluency. And Farah, you had mentioned that there's a parallel in our recent past as a profession in thinking through frameworks and their importance. Would you care to share how you were reflecting on something you'd already been through that looks a lot like this? Absolutely. So when we were thinking about this, I'm just struck by the parallel to what was in my career, the first big wave of tech that I experienced. I came of age when e-discovery became a real thing. And people did not know what to do with themselves. They were used to approaching discovery in a certain way. There was no shared lingo. There was no predefined workflows. It almost sounds quaint today because so much of that has become routinized and clear. But that's because the blood, sweat, and tears of a lot of fearless volunteers went into defining that and coming up with the common language and flows. So the parallel that struck me is that this framework is very similar to the e-drm model in kindred spirit ways, meaning they touch on completely different things. But it was an attempt by a lot of very dedicated industry experts to say can we agree that there's one way to talk about this because we can't even have meetings or trainings or anything because when someone says collection or review, they mean different things. And so they reduced everything onto this model, which obviously has a lot more going on, but it could be put on one page. And I, to this day, any training that I will do that touches on e-discovery, the very first place you start, that very first slide is always the e-drm model. And so the hope and aspirations that I have for the AI ready lawyer framework is that that becomes what e-drm was to e-discovery, a common way of looking at things and thinking about things and talking and teaching about things. It's interesting. I mean other professions do this somewhat regularly, right? It's sort of like how the internet was built because companies building it couldn't wait for the government to figure out what the shared language would be, the shared framework, but lawyers don't have a history of operating that way. So I love the, like, let's get in front of this and figure out how we can have a shared language and a shared understanding of what AI ready lawyers need to know. Is that part of the goal? It's absolutely part of the goal is to get ahead of it. And one, make sure that it is not a static framework, so it is ever evolving as things change, as they are changing every week, it seems like at this point, we can evolve the framework, but it is also not tool-specific. It is also not innovation-specific, so you can apply it at any point in time, but also we can evolve it. We really intentionally wanted to make it that way, and I think the other thing is is that the framework is actually teaching us, as an education company, about what we're missing. So the first thing we did when we finalized the framework with creative lawyers and with the Innovation Council on the PDC board is we applied it to our own training. So we took a look at our catalog, and we made sure that we had programs that could fulfill every area of the five pillars, and not just the five pillars, but all 20 competencies as well, and that we had a progression there so it could hit those different proficiency levels. We found some gaps, and we're currently producing programs to fill those gaps, but it was certainly a learning tool for us as well, even though we were involved in the creation. So it's been an interesting experience, and quite a good one, I think, for everyone that's been involved in just the different perspectives and points of view. Coming in, you know, we have the PD leaders who are struggling with what AI training looks like, what innovation training looks like, and to give them a guide or a map is incredibly helpful, but you know, you could also as an individual attorney, take a look at where you fall, and where you needed help and guidance. And we've been sort of focused on the big picture here and the goals of the framework, but maybe Kirsten, you could start us off on like a cliff notes version of what the framework actually focuses on. So at a high level, what are the types of skills that it's concentrating around? So the first pillar is AI, fluency. Do you understand what the tools are? Do you understand how to use the tools? And then the second pillar goes into application. So now that you understand these tools, can you actually apply your own workflows to these tools? And not can you use the tool in an AI first fashion, but as an enhancement, because the third pillar actually goes right along with that second pillar, and that is making sure that what AI produces to enhance your work, not replace it, is actually the right thing. So you're ensuring that there aren't any hallucinations, that there isn't anything that wouldn't be what you would produce as an attorney with judgment and skill. And that's what we are ensuring is not replaced with this framework. It is really the ability to fluently use AI. And then of course, there's the governance piece that goes on top of it and ensuring that all attorneys understand the ethical implications of using AI and their daily workflows, or as a thought partner. Farah, what about at Harvey? I know Harvey Academy has its own education modules, training modules. Is there some way that those will be mapped onto the framework or aligned with the framework? What's the thinking? The fact that Harvey has Harvey Academy is a microcosm of what's going on right now and why this framework is so important. So obviously high quality material coming from many different sources, but I think when Harvey Academy was created, this didn't exist yet. So what I hope we see and the spirit of this is alignment so that a discipline-wide framework starts to exist. So the job now I think is translation because if you get a certification inside of Harvey Academy and a proficiency level inside of the PLI framework, the idea is that's actually describing the same growth and we don't want a lawyer or a legal professional's development to be trapped inside of any one vocabulary. So that's why this is so important. I'm hoping that going forward as materials are created that there will be that alignment so that we don't have death by a thousand training certifications. Definitely. And what is so cool about the framework, one of the things is how it really is designed for any stakeholder across the profession to take and use in their domain. So what if your inside of firm or legal department and somebody passes this to you forwards it to you and says like here figure out how we should use this in our organization, how would you actually execute on it? Sure, I actually recommend doing exactly what we did at PLI which is first an audit of your current content. So what training content do you have for your attorneys or other professionals within inside of your firm or organization that fits within the different pillars and that addresses the competencies within those pillars? And then where are the gaps? Where do you need to fill the holes in the framework? And then what resources are available to you to fill those holes be it internal trainings or you know bringing in a consultant or a training partner that you have. But I would really recommend going in and just first of all taking a look at what your current training looks like and also your organizational structure or law firm structure I think is really important as well because if you're sitting in an AMLA200 firm versus a midsize firm you're going to have very different needs and resources available to you. I'll also add because this is something I've seen before I've spent years volunteering in clock as an organization and one of clock's major contributions is what I lovingly call the clock wheel but it's that visual of different competencies in almost a pie and at first if misinterpreted can be quite overwhelming because people say wait you want me to develop all 12 core competencies in legal operations simultaneously and they throw up their hands and they're done but that's not how it's intended to be interpreted it's intended to show the range of things and give people choices so similarly in this framework there's 20 competencies that's a lot to tackle at once so I did want to stress that that's not the intent it's to show people what the end state can look like but that you have to put it through your own filter your own priorities where value is going to exist for your organization where your weakest where your strongest and then tackle a little bit of its time I had a dear colleague at Marish who for many years before he retired would say you just got to take one bite of the elephant at the time so while it's not actually an elephant the visual is yet camped off more than you can chew at wads PLI is the distributor of the framework what has been the response so far like who are you hearing from what has the reaction been yeah it's been overwhelmingly positive within the first week of its release we heard from I think it was over 35 law firms just spanning the gamut so Amla 200 all the way down to smaller firms that have specializations in specific areas and what we're seeing is it's been particularly needed and in play in mid-size firms because as I alluded to earlier they don't have the same resources and staffing that the Amla 200 firms have so even though law firms in general have a way to go in terms of AI fluency and having you know the standards which is framework will certainly help proliferate the mid-size firms are basically thanking us for releasing something that can help them structure their AI and innovation training because they don't have a director or a chief innovation officer they don't have access to the same tools that the larger firms have and so they found this incredibly helpful as a guide and a start to assess where they are and then what they can do with the resources they do have to set up a structure so that all of their attorneys are at least fluent in AI so do you guys think it's going to be just as useful for small firms or even solo practitioners what's your view on that absolutely and I'll let fair answer this as well but I mean it was intentionally designed so that me as an individual lawyer could go in and figure out exactly where I have gaps in my own knowledge and use the AI within my practice or for somebody who was responsible for the training or who had innovation at a firm to take it and overlay on top of the firm to assess weaknesses at strengths and see where they perhaps need to spend more of their time with their attorneys so both and I'll second that and say I've been very focused on the in-house experience since I joined Harvey working with different teams of different sizes and one thing that's interesting to me is that smaller teams can actually benefit from AI more because they are so resource constrained and similarly may actually benefit more from this framework because it is the rare large legal department who actually has in-house training development learning resources in most in-house teams at best you have some resources that are company-wide who occasionally try to customize things for legal audiences but most often in-house teams of every size are relying heavily on law firms and groups like PLI to give them the education that they need so this is an incredibly potent basically tool that is being handed for free to the community to leverage as everybody's trying to get to that same level of AI fluency and competency so we've been talking about how practicing lawyers can be using the framework but they're not the only group that we were thinking about in designing this and I actually have to confess that when I was thinking about legal educators I really was thinking about law professors who were trying to modernize their curricula but it occurred to me as we were prepping for this podcast that if you're a law student or you're somebody who's thinking about going to law school there are actually ways that you can be using this framework to make the most out of your educational experience so maybe Farrah will stay with you for a second and ask you know what do you think about law students and prospective law students and how they could use the framework? Yeah I love this one because I kind of put myself in my little time machine and thought back to when I was in law school and what not everyone knows is that just like back in the day you would get a Westlaw or Lexis license so you could do research a lot of students are getting exposed very early on to gendered of AI Harvey has a law school program other companies have law school programs and so students have these tools in their hand and so if I had the power like that back then I probably would have used it for any number of things one that immediately springs to mind is thinking about future employers and doing an assessment of what people say they do in terms of leveraging AI and comparing that to what they're actually doing you know whether the talk and the walk meet as you're selecting where you want to spend your future and then there's also just a lot that I would probably deploy it on as I was you know doing things as simple as picking my course schedule and figuring out what my daily life would look like so it's just incredible to me like we were talking about right at the beginning that all of these capabilities are unlocked they were never there before and there's going to be generations of law students who never know a different way who say what do you mean you had to sit down and just talk to people and hope that they were telling you the truth during meet and interview and you used to let humans drive those cars why did you do that what were you thinking so dangerous yeah here's some what do you think about law students and prospective law students yeah I think there's two things I would add for prospective law students I think it's a great way to assess the AI readiness of the schools to which you've been accepted so you could take the framework and apply it to the programs and training that the law schools offer in the areas of AI and innovation and then the second thing just as Farrow is speaking there's an ethical and governance area here because I think as Bridget said you know there's going to be a time when law students don't know that AI didn't used to exist you know they don't know their daily lives without using AI and I think there is the risk of complacency and familiarity there that sort of loses the ethical obligation that we as attorneys need to keep in mind and so I think using the framework in that manner throughout your life cycle of from being a law student into your practice when you pass the bar and become an attorney I think it's really important to keep that part of the framework in mind because I think it can be lost as we become more and more more comfortable with AI and I think that goes also to using it as the thought partner and not a replacer for an attorney and for a lost unit as well. I appreciate that it sounds like it's built in a way that's got flexibility. So as the technology and the skills that lawyers need to learn change, I think the framework probably can change with it right and can be updated and you can imagine like that being a reason not to do it right like well we don't know yet what it's going to look like and I feel like a lot of lawyers have that reaction to the technology generally and so I love that you guys just were like nope we can add value by putting together where folks can be now. What's next? Like is it possible that we can get a so I really would like to go do it and win? I would like to win the framework. Can you give me something if I win? Is there any certification assessment anything like that coming? Yes, so right now we're in the experimentation phase with a few things, one being an assessment because for every framework or every competent matrix if you will, it's always helpful if you are taking an assessment that is on biased and you're answering questions based on your own knowledge and so you could read the framework and identify where you are but wouldn't it be wonderful if you could take looks use quiz that's a little more exciting than assessment. Although assessment is the proper turn you can take a quiz and find out your areas of strength and weakness and then even what to hone in on we could recommend trainings based on those areas of strengths and weakness so it could be a whole unified ecosystem of ensuring that attorneys are constantly rising to that next level of proficiency for each of the different pillars. And then certification is something that we have tossed around I mean we have CLE and that's been our main focus but certainly there is something around the certification without creating the linearity in the framework because that's something we really want to avoid. Well and I'm going to give an unsolicited fan girl glowing review of what the team at PLI is doing. I've been fortunate to be part of the PLI Innovation Council and as I mentioned faculty for years and years and what I love about what PLI is doing is they are very much walking the walk. They are an old organization that people might think about as only doing one thing people might just say oh I get my CLE from PLI and then that's all they know but I've seen firsthand that they are really experimenting and leaning hard into AI and all sorts of other creative ways of teaching and learning and that's exactly what the industry has to do too kind of shed old images shed what you think you know because everyone's in this period of reinvention and the winners Bridget are going to do exactly what you want them to get in there try it and win. I want everyone to win the whole profession and if you're out there listening and you want to be part of this and learn more about it here's some where can people actually find the framework. So they can find it on the homepage of PLI so www.pli.edu is where the framework sits. Do you have to have some kind of membership or subscription to access it? You do not it is free to download anybody can download it. Awesome we got to get that word out to law students because I do love that if I were a law student I would love that and also law students are very good at advocating internally for what they want so when they discover that not everything in the framework is on their course schedule or available to them they can advocate for change from their own perspective and position which I love. Well in Kirsten we didn't explicitly say this but I think implied is that if anyone is listening to this you review what you see in the framework and you have strong views you either think something should be different or you want to actually bring it inside your organization and deploy. Reach out I think the group that put it together would love nothing more than to engage with people who feel similarly passionate about this topic. 100% we are always looking for feedback. In fact I keep a note on my phone of feedback for the framework so I am happy to talk to anybody who has opinions on it it can only get better. And it only well get better and we're so grateful to both of you for spending your time to share this exciting development with I'm sure many people who weren't familiar with it and are now eagerly typing into their web browser to find a copy of the framework. You're both fantastic and it was lovely to talk with both of you and we're so grateful to everybody who is listening right now and this ends another episode of AI and the future of law and we look forward to our next conversation which will be totally different because every day an AI era is totally different. So until we speak again we hope that you are well and we'll see you next time. Do you want to turn AI insights into opportunity? Hi I'm Bridget McCormick president and CEO of the American Arbitration Association and co-host of the Triple AI podcast with Zach Abramowitz. Together we talk with experts innovators and builders who are putting AI to work in dispute resolution exploring how technology can drive fairness efficiency and trust. Each episode delivers insights and strategies to help you stay informed inspired and ahead in the age of AI. Listen at ADR.org