Why AI Governance is Relevant for Consultants and Business Leaders - with Christopher Pavlak of Deloitte

[MUSIC] This is Daniel Fajella, head of research at a Merge Artificial Intelligence Research, and you're listening to the AI Consulting Podcast, the home for AI strategists, advisors, and consultants. If you're looking to start or grow an AI consulting company, you found yourself in the right place. Our focus this week is the topic of AI governance, and why it's relevant for consultants and business leaders. Our guest this week is Christopher Pavlak. Currently serving as a specialist master for AI ethics and AI governance with Deloitte, previously was in the defense world. He has his master's degree in technology intelligence and cyber space from the National Intelligence University, and he also is a JD, a doctor of law, and he is also a JD or a doctor of law. Before working at Deloitte, he was a research staff member at the Institute for Defense Analysis, focused on laws, policies, and their applications to cyber space, AI, and cloud computing. So his transition to AI ethics and AI governance is from a unique angle, but is somewhat of a natural transition. Chris this week speaks to us as a consultant himself about what AI governance is at a practical level, and how consultants and business leaders can think through AI governance as a framework to help them build more successful AI adoption strategies to cut through some of the red tape, avoid some of the risks, and be able to see more projects through to success. We're also grateful to have Christopher as one of our many Emerge+ members. If you are not already an Emerge+ member, Emerge+ is our online platform with our full library of Emerge AI use cases, white papers, and best practices and frameworks. Again, we're grateful to have Chris as one of our many members in the Emerge+ community, and he has a unique enough angle here with governance and being a consultant himself that I knew we would have to find some time to be able to pull him onto the show. So we're glad to have Chris with us finally here on the AI Consulting Podcast. Without further ado, this is Christopher Pavlak with Deloitte. You're on the AI Consulting Podcast. So Chris, I'm glad we're able to catch up. I think we connected first either on LinkedIn or through the email list somewhere we're glad to have you in the Emerge+ community, and you're doing some interesting work at the intersection sort of AI and ethics. A space that's obviously become hotter and hotter over the course of the last three years or so. And I think for some folks they have a vague idea of maybe how they should think about AI ethics in terms of a business context or listeners or business people. When you sit with execs and you basically given the short story on, hey, you know, what is AI ethics? What is this matter to business people? How do you like to explain it? Well, Dan, thank you. What I like to explain is that first off, this is no longer anything you or something you should keep in the Office of General Counsel. This is an operational problem, right? Now what we've noticed is that at the strategic level, and this is happening around the country, around the world, businesses, organizations, intergovernmental agencies, they are writing their charters with these lofty, strategic, even aspirational principles, right? And that's good. We all believe in fairness. We all believe in transparency. We all believe in reliability and accountability. But the question is, and this is where we have to throw it back at the executives and say, okay, how are you actually doing this? How are you operationalizing these things? Because to have them on the shelf isn't enough when it comes to brand risk, when it comes to financial risk, when it comes to other things, where you could be legal risk especially, right? There's things that you're going to be accountable for. So to speak to it isn't enough. How are you going to operationalize it? That's how we get there attention. We kind of snap them into attention with this is an operational problem. Yeah, well, and I guess just to nutshell that and solidify it, in terms of the operational problem is that I think there's one extreme of the AI ethics world, which is a very loud extreme, which I commonly poo poo kind of unabashedly, which is, hey, if you're using data, we might call you an "ist," right? You know the kinds of "ist" that you don't want to be called Chris. They're floating around out there, you know, if they label you with that, you know, all of a sudden, you know, you don't have a job and stuff, and then, you know, there's that. There's sort of the eggshells as a way of getting attention. AI equals, you know, danger of being an "ist," really, really rough and gruesome kind of line of thinking. But it's not that it's irrelevant at all. Like you said, there are plenty of legitimately valid instances here. When you talk about operational, what does that mean? You know, so there's a general meal you have, oh, by golly, you know, if they didn't think we were a fair company because we had this algorithm that did something with labeled somebody the wrong way, who really didn't want to be labeled that way. Like that's one general fear, but how do you make this more practical for day-to-day business? You'll operational as you said it. Well, I think the more you can operationalize it, the less likely you're going to be end up as one of those "ists," right? You have to really streamline these principles into low-level requirements and all the different constituent ethical issues underneath the larger principles, right? And the more you flesh that out and you operationalize them, you will less likely be offensive. You will less likely get something wrong if you can get this down to the tactical requirements for those developers, for those data scientists. And for something that you can explain to General Counsel and say, "No, this is what we're doing, and this is why we're doing it, and this is how we're doing it." Those questions, once you can answer those questions, there's going to be a lot less likelihood of you being labeled in such a negative fashion, right? Yeah, yeah. So is it some of that social pressure on some level that gets this stuff on people's radar? Well, absolutely. Yeah, yeah, yeah, man. And that's the world we live in right now, okay? It's the world we live in, and that's the way things are, but it doesn't mean it's not important, and you don't want to -- Oh, no, for sure. --You don't want to just dismiss its importance. It's just that -- Definitely not. That's the risk that's front and center of people's minds is termination of employment, et cetera. Yeah, yeah, or the PR and whatnot. And again, they're not trivial issues. It is childish to blow-v8 every document search and discovery program as a gateway to aggregate racism across the company, right? That's in my opinion just super insulting to everybody doing AI that sort of everything is actually that risky and that intentionally dangerous. But there's plenty of instances where it does matter. And of course you guys think about this. So you sort of -- I'd mentioned off microphone here -- seeing AI governance and ethics as a part of AI maturity. Now, we at Emerge study AI maturity a lot. We've executives from the ARMs and hardware companies talk about it. Amazon leaders talk about what AI maturity means, what are the components, the elements. Governance is a very particular niche of that. What are the pieces of AI ethics and governance that for you fit into maturity. We'll get into examples in a bit. But I'd love to say, you know, how do we begin to operationalize by thinking about the parts? So how do you like to explain that to people? So I'd like to say, like, one of the hallmarks that we see in AI mature companies versus immature. And I want to just say that the maturity and maturity is a spectrum. It's not binary, right? And some companies are better at different aspects than others. And when it comes to governance, I think that's where we've seen the most mature ones are the ones who have a specific, unique governance fabric for their AI systems. They know what they want to do and they know what they don't want to do. And so it can prevent unintended negative consequences when they have already spoken to general counsel. They know exactly what they need to do when it comes to governance. And I would say, like, when it comes to biases, biases, excuse me, regulations and privacy. It's like, okay, they know what they want to do with this specific use of AI. And they can take that and then extrapolate, okay, these are some risks that we might encounter. And we have, we have kind of turned over every rock when it comes to governance issues and anticipatory leaning into these issues, thinking about how they're going to do these things in the future. Yeah, yeah. Well, and so you bring up an important point here. And I think that on some level, there's an obvious validity, in my opinion, for the kinds of AI ethics work that I think is really important, which is, hey, you know, if there's a shot that we're going to go against our own values, or we're going to break the law by doing acts, let's say we're, you know, determining our loans based on some criteria that legally we shouldn't be doing. Or that we as a company just wouldn't want to stand behind in terms of the criteria by which we make decisions. And again, lending is a very, very heavy case, right? Document search for vendor invoices, little bit less is risk, right? But so, so I'm talking about an extreme example. And I don't think all examples are that heavy. But let's just talk about one of those. If there's a shot that we're going to be able to violate our own values or or or the law, then there should be a way on the way in for this project to be assessed, to be examined, to have certain questions asked to get under the hood on certain data and have some boxes that we check along the way so that this thing can get approved and we can all go to sleep at night and maybe have a way to monitor it. So we can go to sleep at night, relatively speaking, feeling like we're doing right by our customers, doing right by our values, doing right legally. And I think that's a wonderful thing. It's like, cool, let's have technology that's aligned with everything we want to be aligned with beautiful. You also brought up the looking under every rock. And I can see, and in fact, I mean, some of the AI ethics world kind of has to do this for itself, where it's like kind of drumming up, right? It's like, oh, what monsters could be under this rock? You're looking through those invoices from like a vendor that's sending you soap. What if you thought that vendor was Asian, you know, or like I'm giving you a stupid example, but you understand what I mean. So there's an overboard of looking under every rock. How do you think about, you know, building that that check box process that's going to handle the legitimate concerns, but not bog down every single project with absolutely undue quadruple checks under every corner when it's when it's not necessary. How do you find the balance or think about striking it? I think you strike a balance by this is another aspect of what makes a mature company is a proper team, a team structure, right? So who's going to be involved when it comes to thinking about it? about operationalizing these principles. It can't just be Office of General Counsel. It has to be obviously the CEO, it has to be data scientists, it has to be developers, it has to be lawyers, and if there is an ethicist or a risk officer on the team, all the better, right? And it also gets to the more the company knows, and this is another thing too, like the more they are thinking mission first and not AI first, the more they know who they are and what they want to do, the more likely they are going to be able to anticipate the risks that could be, that could surface. And then you can, you can, I think, investigate those things accordingly, understand what's on the horizon. And then again, do you kind of, I don't say a piecemeal, but slowly, slowly go to the uncharted territory, safely knowing, well, we only have a specific use. This is all we want to do with it, and we know why, and we have all the people, all the right thinkers on board right now. And so we feel like we're confident enough to move forward in this. Yeah, to turn over every rock, it's an onerous process. You don't have the money, you don't have the time, right? And then a CEO is going to ask, well, why we spend our money on all this legal research, what all I want to do is X, Y, or Z, right? And so there's a healthy bottom buy-in you need to get across the staff, right, or across the offices. - Yeah, yeah, and think about again, which of these facets of governance should we really fight for? Yes, executives might want to push back, but which of them should we draw a line in the sand? And maybe which of them are absolutely beyond the severity of the kind of nominal application that we're trying to build, and the European Commission and some other folks are thinking about kind of the OECDs actually doing some very interesting work on. The gradients of how impactful a use case could be and how much actual ethical investigation it might require. I imagine you guys might be using some rules of thumb and maybe some systems you're developing with Deloitte to think through that. I think that's a very important thing too. You mentioned something really interesting as you continue to go with maturity about why being mission driven first actually can make some of these decisions easier. Why did you say that? You know, what's an example of how kind of that sort of focus on the mission rather than AI unto itself can help streamline this decision making? - Well, I think we have to understand it from a perspective of how, you know, again, AI is, it's not a possibility it's here, right? So those mature companies who understand that who are, they are deferring and defaulting to the fact that they are in an AI world and this is the way things are. If you think about it as a coworker, right? And what kind of people you're gonna bring onto the team, that means you know specifically what you wanted to do and what you're gonna ask of it, right? So the more you have a specific mission involved with who you are, you're not gonna keep throwing tech at things, you're gonna know, you know what? We want higher that kind of person, we wouldn't use that kind of AI, right? So we're gonna scope it down to say, and you know this from 15 years ago, when people didn't understand things, they might have thrown a lot more IT at the problem or just throw more IT at it, right? Or throw more tech at the problem. That's not the answer. Like AI is not the answer right now. Mission is the answer. Like know who you are and what you want to do, what you don't want to do. And then apply that specifically or develop a specific use case or business line you wanna put AI against, right? And I think that helps, again, that drives, that is the mission. And then you can say, okay, this is what we wanna do with our AI. This is what it's gonna be used for. And now instead of thinking that some risk is gonna rear its bug we head from the corner, like no, we know it's not because this is all we're gonna use it for. - Yeah, exactly, exactly. It's not the open-ended Pandora's, literal Pandora's box of AI that we're opening that's gonna spin demons around the whole company. It's like, well, we are headed in this direction. We're supporting the strategic aim. We're enhancing our ability to recommend products in these particular ways. And here are the risk domains that are gonna be worth investigating based on the kind of data we're using. As opposed to, as you had said, kinda having to think about everything. So that's great advice. I mean, for AI in general, Chris is thinking about what we are as a company or where we wanna go as opposed to AI as a buzzword. We're seeing a little bit less of the latter in the last two or three years, but there's still plenty of it. So I think your advice is still more than salient. You've brought up a couple great points and I don't wanna interrupt your flow here. You just brought up team in terms of one of these maturity components. And you had talked about some of the other elements beforehand. Was there anything else you wanted to enumerate in terms of the pieces of the puzzle for governance and ethics when it comes to AI? - So governance and ethics, I think it's just, again, having a healthy understanding of what AI can do, what it's meant to do for you, for your company, right? It has to be, again, taken out of the kind of esoteric idea. What is it that you want to do with it? And then how is that risk associated or what risk is associated with it? And why? And then have the right people in the room to discuss, have the right people in the room to raise ideas. And I would argue, get the young developer in front of the executive and have the executive sit there and learn and listen and understand, okay, this person is enumerating the risks. I'm not gonna jump to conclusions and I'm not gonna let all the respect to lawyers. I'm not gonna let a lawyer who doesn't understand these things inform me of my risk. I'm gonna talk to the people who actually can appraise it more precisely. - Well, I guess that has to do with team. We think about Chris when it comes to project teams. When we're kind of setting the bounded reality of an AI project, being able to have our data science folks in the room, hopefully some strong and house talent there, our subject matter experts in whatever area were focused on and business leadership who understands budgets, who understands strategic goals and who ultimately is kind of deciding on what kind of resources we can allocate to what mandates and what kind of timelines we need. And make sure that the ambitious goals of leadership are being maybe checked by the technical realities that the tech folks could bring up. And that the subject matter experts can explain which projects are not viable because they know how that process works. And we can't automate this and here's the six reasons why and be able to share those things. When it comes to unearthing the risk side and the ethics side are their component parts. So you just mentioned kind of having the developer next to leadership kind of maybe talking a bit about the tech, who are the people on this team who can really help with that thumbs up, thumbs down on the ethical risks of a project. Who's gotta be there? - I think so the developers need to be there, the data scientists need to be there. And again, if we're, I'm thinking of it kind of abstractly because I don't know if the schedules are kind of be able to sustain a chief risk officer at any of these meetings, right? - Yeah, exactly. - So that might be an ideal. But my point is like people who are well informed about risk, people who are well informed about model and model behavior and then data and where the data is coming from, understanding that if there's reason for an ethicist to be there, great. But you have to think about it. Maybe our way to put this is there are certain kinds of risks that are going to be human centric. There are some that are going to be process centric and then some that would be techno centric. And human centric are going to be those that only human being would want to deal with or should deal with. There are such high stakes that we have to give this over to human being or a team of people to decide. Then there's process centric risk, which would be more along the lines of, okay, human beings need to be kept in the loop on this. But we can let the process unfold throughout the ecosystem. Then there is techno centric risk. Those low level risks aren't going to be two high stakes that we can give over to machines, right? So when it comes to the team involved, you have to have someone who's real speak truth of power and understand what is involved with each respect of model you want to develop. And if they can't and you got to get some new, but again, a collaborative approach to understand the inherent and latent risks of what you're trying to develop. Got it. So the elements, and like you said, maybe the chief risk officer can't be in the room for every AI project idea that's getting bandied about. But you seem to be talking about business leadership who has goals, objectives, maybe expectations. The technical folks, developers, data scientists who can talk about what realistically we're dealing with here. And then maybe somebody who's risk related who can maybe think through some of the legal considerations and make sure that we can categorize and you've given us three strata there. And I'm sure there's a lot of stratifications, but those seem like useful ones. We can stratify based on these use cases, these workflows. Is this something that actually we're not going to touch with tech? This is pure, pure people. It's too high touch. Like determining the exact loan price and terms of some gigantic corporate building loan or something. By the water with some really weird unique characteristics, it's like, no, maybe an auto loan for some really high volume area, we might be able to technically handle but maybe some things not. So kind of categorizing the level of involvement. And like you said, speaking truth to power in your terms, being able to be very frank with leadership about where we need to push back and where the risks are actually real. Right. I agree. Yeah. Any final kind of closing advice things you've learned, you know, you've gotten a little bit of hands-on time now thinking about this stuff and being able to work with some large organizations, including the public sector. Folks that are listening into this episode as we wrap up, they might be thinking about, all right, as we start to roll out more AI project or maybe consider our first big deployments, we might want to have kind of a process that we develop for this. Any last kind of bit of advice for leaders who might be kind of helping steer this process? Yeah, I would say two things. One is, one is a good one's kind of a bad, I guess. The first one is mission first, right? Understand who you are as a company. Understand what you want to do, what you don't want to do. And what you want to put AI against. Secondly, and this is the hard part that I think people are discovering is taking these strategic level goals or these strategic level principles, which is qualifying language, right? When it comes to the law, the law simply qualifies things, certain behavior. Well, how do you translate the qualified language into quantifiable metrics? Because that's sometimes the only way it's going to get noticed, the only way it's going to get measured. And the only way it will be understood at the bottom line is if you can actually quantify what this risk means for traceability. for fairness, et cetera, et cetera. So that's gonna be, I think, a friction point is getting the translation of those principles into tactical level measurable requirements. That's gonna be challenging, because right now their qualifying language and to turn them into quantifiable metrics is difficult. - Everybody is wrestling with that. I see this up close and personal in OECD headquarters as they think about wrangling this from a governance perspective. You guys are gonna see it boots on the ground with real companies. There is no one playbook here, but it is, I think, important for people to know this is what you're up against. This is a new space and getting to that level of granularity is gonna be tough, but quite necessary. So useful advice for the folks who are tuned in. Chris, I'm glad we were able to catch up and thanks so much for joining us on the show. - Thank you, Dan. I appreciate it. (upbeat music) - So that's all for this episode of the AI Consulting Podcast. Thank you for listening all the way through to the end of this episode and a big thank you to Christopher for joining us as a guest this week. We're grateful to have you as a listener, but we'd also love to have you as a subscriber as someone who is a reader of our newsletter to make sure you not only get all of our latest AI Consulting Podcast updates, but our other interviews and updates as well as our latest use cases, articles, and infographics. If you wanna stay ahead of the AI curve, when it comes to trends, use cases, and best practices, go to emeorg.com up at the top right as a button for a subscribe, it is completely free to be on our newsletter, which goes out twice a week, every Tuesday and every Thursday. So again, if you wanna stay ahead of the curve in terms of cutting edge use cases, or if you want more wisdom and insight from great AI consultants in terms of how they're growing their business and how they're servicing their clients and customers around the world, then again, emeorg.com up at the top right as a button for a subscribe, make sure to stay tuned there. Otherwise, keep it locked right here for next week. I look forward to catching you in our next episode of the AI Consulting podcast. (upbeat music)