AI Leaders Podcast #80: Human in the Lead: Redesigning Agency and Trust in AI

The biggest risk now is pretending AI is still just software. One systems are reasoning recommending and adapting. They're become decision-ath. Hello everybody and welcome to some time with us here on this podcast. My name is Andy Truscott. I'm the global health technology leaping sensor. And I'm giving my great pleasure to be joined here today by Simon Azarian. He's the executive vice president and chief digital and technology officer of the city of Hope. And a long term friend of mine. Simon, thank you for joining me here today. Thank you Andy. It's a pleasure to be here and really look forward to our discussion. So let me start by not talking about AI. Healthcare is facing kind of three converging failures at once. A workforce understanding, a collapse in decision velocity, and a growing credibility gap between what technology promises and what actually scales. And Gen AI was supposed to change that game. Instead, most organizations are stored at pilot proofs, controlled experiments that never quite tip into impact. And that's still how it kind of, I don't know, as I look at it, it's not a technology problem. It's a leadership and operating model problem. And here at Accenture, we've been using the phrase human and the leads very deliberately, not as a comfort blanket as a provocation because Gen AI does remove humans from leadership. It forces leadership to evolve. And the deeper idea behind reinvention is this, you cannot bolt the future onto yesterday's operating system. So today isn't really about use cases or pilot. It's about the architecture that sits underneath them. And Simon, I want to frame you here, not as someone presenting the case, but as a co-theorist, a co-conspirator, if you like. Someone actively working through what leadership, agency, and accountability look like when humans and machines are both accessing the system. So let me put the provocation on the table. What is scaling AI is what the objective at all? What is the real unlock is scaling agency with humans firmly still in the lead. When we talk about agency, we're not talking about empowerment, slogans or innovation, theater. Agency is structural. It's the right to decide the obligation to act and the accountability for outcomes. So most healthcare enterprises were designed to distribute responsibility and centralized permission. And that model already struggled with humans. It completely breaks when machines enter the workforce. Genai doesn't fail because models aren't good enough. It fails because organizations never clarify who is allowed to act when intelligence shows up. And human in the lead doesn't mean humans approve everything. It means humans design the system of decision rights that machines operate within. So Simon, how are you thinking about this at City of Hope? Where are you deliberately redefining decision rights and accountability as humans and machines start to work side by side? Yeah, thank you. Great question. I mean, for me, scaling agency is sort of in a hybrid human machine workforce really means shifting power closer to the work and doing it safely, especially in the healthcare context. So it's as you said it yourself, you know, it's not just about giving people their tools. It's about, you know, re-architecting decision rights. So all of these individual teams compact with confidence, speed, contacts, you know, while the machines handle the cognitive heavy method. Right. So there are three things that matter most to me. You know, context over control, I think is one of them. You know, that pay-ashtray people real-time insight data predictions, trade-offs, et cetera. So they understand why decision makes sense, right? And not just what to do. That's really important because agency grows when humans retain judgment values and accountability. You know, again, I'm putting in the context of healthcare. You know, secondly, God reals, not gatekeepers, right? You know, instead of sort of centralize approval chains, we need to embed, we need embedded policies. You know, you I should have yourself as well, technical constraints, automated risk checks, et cetera, that we go along. Let's more people make decisions without the increased risk that we would have, you know, because safety skills with them, right? And then, before I go into what we do at City of Poverty, and third factor that I keep in mind is learning when the flow of work, right? You know, in sort of in a human, I workplace every decision, you know, sort of becomes a feedback, you know, yeah, it just doesn't assist. It teaches and over time, the workforce does just and get faster, it gets smarter as we go through. So, skating agency, you know, as we talked about, is an enablement training or tool, it's workforce mutation, right? We got it, they got, you know, humans have to evolve from task, you know, executors to decision owners, right? And, machine has really become amplifiers of this judgment and replacement point. Now, that's how you unlock, you know, intelligence scale without losing trust and quality and humanity. But specifically what we're doing at, you know, at City of Hope, you know, first and foremost is Embedic AI into workflows with clear human owners, right? You know, we're just isn't piloting, you know, cool technology, you know, it's systematically integrating AI into clinical and operational workflows in a way that shifts who makes decisions, right? And, you know, they're, you know, our Jennifer AI platform, called LLM automatically synthesizes decades of clinical data and surfaces the case, key insights for physicians, right? But physicians still own the clinical judgment and the patient interaction. Right. Then secondly, you know, it's really operationalizing explainability, you know, to support responsible decisions, right? Predicted models deployed across not care settings are really providing contextual and exponentials alongside risk scores. So that's a really deliberate design choice that conditions are left with a black box number, right? They see what factors drove the prediction and this really elevates their understanding and helps them in temperate and that response, rather than, you know, they're just following an algorithm. We don't want that. And then, you know, I can go on, I mean, be redistributed cognitive effort to expand agency by removing administrative burden as much as we can. You know, when you look at government and strategic leadership, we have senior digital and AI leadership structure with myself and our chief AI and analytics officer that we're showing that we're building organizational accountability for how technology affects decisions that comes in ethics, right? Start an app-hawk experimentation. We are actually doing this strategically. In short, you know, you know, City of Hope is a scanning agency for our workforce isn't about getting rules or replacing people with AI, right? It's about regaining those decision rights. I agree with you entirely. You know that, but it's interesting. What I see consistently is that Gen AI exposes ambiguity that we kind of used to get away with. So humans can navigate onto accountability through social work around, et cetera, but machines cannot. So, scaling agency kind of isn't philosophical. It's an architectural thing. And just a quick one here, but where has clarifying agency for you been hardest culturally, but is just refer to or structurally? Oh, wow. Yeah, I think culturally. I think that, you know, when I think about it, you know, because, you know, agency sort of the new scale unit, right? You know, it's your ability to see a problem, decide what matters, you know, act without any friction, right? And you know, deliver outcomes not just outputs. And I think that it's a new way, cultural way of new way of doing work, new way of doing your work. And you have to culturally adopt it and move forward. Because structurally, we are able to put it into place, but making sure it's adopted, you know, it's important. And frankly, AI is massively amplifying this. That is our single individual is really able to understand their decisions, right? Access to data, you know, an augmented execution. So I would say culturally, it's just a challenge. But the good thing is that once you find the champions within the organization, those champions, really help bring, you know, the organs, the others along in that this is good. This is brewing. So it's just focusing on those, you know, it's an organizational physics. That's changed. That you are. You're absolutely right. And this kind of brings us into governance space, because most response play-eye models were designed for a world where AI is frankly occasional and somewhat slow, external oversight, committees, period interviews, and that model assumes that humans sit outside the system, pulling levered after the fact. But once AI operates a workflow speed, that model collapses. Re-invention demands something harder. Governance has embedded, there's continuous, and there's increasingly self-aware. Human in the lead does not mean humans might from managed machines. It means humans set intent, boundaries, and escalation paths that the system itself can reason over. Now, how are you thinking about governance evolving from something external and episodic into something that's frankly conversational and internal to the system? Yeah, I may have a view side of this. In a responsible AI governance, it can't be bolted on from the outside anymore. It has to live inside the system. And from my experience, there are few shifts that actually make this governance work. The first one would be, you know, you've got to move from oversight to embedded, you know, a chance ability to be talked about instead of here, sitting outside a loop. You know, governance has to be designed into the model of the lifecycle. Absolutely right. What I'm concerned is that human in the loop has become a false sense of safety. You know, in many cases, humans outside will become the slowest, least informed actor in the system. So where do you see that tension showing up? Well, the thing is that accountability is now become traceable. Who trained, you know, what did he take shape? What policies were active when a decision was made? So, you know, it's no longer a review meeting. It's an architectural decision, right? As you know, built this governance with it, right? And that, you know, pushes you to, you know, really treat AI as a decision, participant, not just as a tool, you know? So the biggest risk now is pretending AI is still just software in my home loop. You know, one systems are reasoning, recommending and adapting. They're become decision-ad, right? So that there is no explanation of who's responsible with AI is wrong, that trust collapses really quickly. Let's drop out the models. Let's talk about the data. Now, because this is another place where reinvention thinking really matters. Data quality is table stakes. It's necessary, frankly, in cost industries, completely insufficient. And you can have perfect pipelines, feeding systems that still make bad decisions, because data was never designed in purpose. Reinvention forces a shift from data as exhaustive to data as intent. And data is aligned to specific decisions, clear decision rights, explicit accountability. And that's what we mean by data intentionality. Now, how are you thinking about data at a city of hope, not just as an asset, but as a decision-shaving instrument for humans and machines working together? - Data with purpose is about meaning, right? So, you know, when you're moving from simply collecting data to creating data with real intent and impacts, but you have to flip the question, right? I think that instead of asking what data do we have, you have to ask, what decision am I trying to improve and for whom, right? And purpose comes first, data second, right? And that shift changes everything. You know, first and foremost, it means, you know, curating, not hoarding, not all data deserves equal weight, right? I impact data is trusted, it's relevant and timely. And for that, you know, you got to have to validate it, establish clear ownership, and no definitions, et cetera. Secondly, you know, it means structuring data for collaboration, right? It has to be understandable by people and usable by the machine, right? That's, you know, when, you know, not to get too geeky, but you know, metadata, context, lineage, all of those semantic layers become about you, they matter, right? And, you know, without this context, AI can scale confusion faster than humans ever could, right? And then thirdly, you know, the purpose driven data is aligned to outcomes, right? You want impact, whether the goal is better patient care, observancy or strategic insight. Your data is shaped around that outcome, right? And you don't just, you know, measure everything. You measure the right things and you measure the, consistently across the board. So finally, you know, it requires governance as we talked about that really enables not slows. That's a key fact, right? Gargoyles that can protect all of these without, you know, while I'm allowing the teams to work fast and be able to continue experiment. - Yeah, that's absolutely right. You know, this is human in the lead at a systems level. Now leaders is not just what data we collect, but why, for whom, and with what authority it's attached. You know, in healthcare, you know, we both work in healthcare for more years than I care to really count some days. That healthcare is brilliant at experimentation and deeply uncomfortable with commitment. And pilots feel responsible, but platforms feel irreversible. And reinvention doesn't happen through pilots, so it happens through platforms that encode standards, trust, repeatability. And the uncomfortable truth is that pilots optimize learning, platforms optimize accountability. And GNI scales only when leaders are willing to make that shift. You live that tension. And what were the moments that you've had where pilot thinking has become a constraint rather than a safety mechanism? And what changed once you leaned into platform thinking? - It's, the thing is that if we really stop thinking about pilots as a moment in time, right? Pilots become platforms, meaning that you are now iteratively improving on that pilot as it takes on a life of a platform over a period of time. It is never appointment time. Is your ability as an organization to continuously improve what started as a pilot and a transition into a platform continually? Right? And that brings the trust, right? And that really, you know, it's like when you get the data, right? You know, it stops being exhaust, right? It's just, you continually get the motion and the iterations into play and you're able to really make a difference and that platform is a continually growing thing. It's a living, breathing thing that, and when you bring the practitioners closer, they're actually seeing the platform's defaults. And they're excited about it. They want to adopt it because they feel a sense of ownership. When you think about it, when you're rolled it as a point of time, then all of a sudden, the workflows are not growing, they become rigid, they're not keeping up the times and that's where you get it to drop. This speaks to workforce because this is where most narratives seem to be getting softened, okay? And I want to say, I don't think it's about a navelment of our workforce. It's about, you know, for one of a better word, a mutation of our workforce. We're seeing the emergence of a new work species, digital agents with defined scopes and accountability, hybrid teams where machines are contributors and not tools. And an early forms of credentialing and assurance for non-human access. I don't mean one pet dog, I mean the agents. And this is not speculative, it is already happening. So how do you prepare leaders, how do you prepare leaders for a workforce where accountability is shared across humans and machines and where human in the lead means designing the ecosystem, not supervising every task? - Well, I go back to a couple of things that I shared before. Number one is that when other practitioners are involved in the creation, it kind of changes. And that those champions are helping you create the now really contribute to the adoption of this. So because folks have examples to see that this person who is doing the same thing I am, is now doing it differently. And guess what, I have to transform as well. And it also gives them the sense that AI is not here to replace AI, he's just here to enhance your abilities, do some of the cognitive thinking, et cetera. So it's really a combination of these chat. You know, is that our modeling the way that the rest of the organization will continue to see that it's working at the topic and put their trust in, you know, the emerging technologies of madness. Right? And I think that they're being in our replaced and now realize that I can do more, right? And they become excited about, you know, additional contribution and additional ownership over all of this work. We have to change, we have to change as well as our people have to change and, you know, I like to think that Mike over here, I think you do as well, that we are constantly learning and they I've not learned something as a day wasted. Um, but, you know, let's be kind of come to the close of thinking about this stuff. I'm going to hit you with some lightning questions and I'm going to, so I think about us. So, so what's one job description you would rewrite immediately for the general AIO? I'll change that. So one job description, I wouldn't rewrite. I think that a Gen AI and agent to AI, agent to AI, I think that are really changing the way we do things across the board. And I'll say this someone, I was jokingly, but, you know, technology or IT is now becoming a NHR organization. Because we got all this agent taking the AI, you know, we actually have to care and feed and, you know, protect these agents for the work that they do for us. So that's one thing I would do is make sure that we understand that, you know, we have to take care of these agents as well as we do our partners that work with them. So, HR is actually the exact job description that I was thinking on here because the way that we perform as our people, we should be performing and glancing our agents. And we should be weighing a measure of frankly our agents, they don't have the same termination regulations of our, as we do our people and they're wondering why I think, but with our agents, we should be managing. Are these doing what we want? Remember an agent is generally a subset of the human capabilities, so this particular slice of what they're doing. Is it right? Is it proper? How are we measuring that when they're not performing? How do we retire them out? And I said that the seat chief resources officer should be leaning into that as well or least fully informing that dialogue. So second lining one, and I don't have to think of an answer to this while you're talking as well. One blind spot, you still see most executives holding onto. There is a lot of misconceptions about what AI is, what AI can do. And in today's world, we get faced with a lot of things that come our way that have a label on AI. And when you start peeling the onion, you realize that a whole lot of majority is just the good old, you know, automation. That is now just, you know, pay eye. Then you get a certain additional ones that come your way that it's really a predictive analytics that using the past in order to predict the future. Right. And then you have those unicorns that come your way that are truly, you know, generative, agetic AI base that have the right orchestration right governance, all of the things they're in order to move the needle forward. I think as an executive, it is important for us to be able to understand and differentiate between all of these things that are coming out in 100 miles an hour. Where do we focus out of we can corporate AI first and forward thinking into our day-to-day life in order to be able to understand, you know, where is the areas of focus that bring us the biggest impact into our organizations and into our personal lives as well. I'm glad to hear you man. The interesting thing to me and I think I'm guilty of this a bit as well is thinking that this generative AI thing is just this new thing that suddenly appeared. And there's something new I need to pivot to and respond to. Well, I think the blind spot is it's been there for some time in encroaching and our blind spots have been we have been addressing it, which is why we have to address it now that it's suddenly much forward. And the exact side get to work with who I think have the most realistic views here is those who actually look at and say and a humble and say we missed this. There we need to get through it now because we missed it. Okay, a lot of question. One governance change you would make tomorrow if you could. I would do whatever I can to change the perception of governance from police to partner. I will the I want the organization to understand the reason we have put governance in place for AI for data. We can go we do is not to police to organization but to partner with your organization to make sure that we're safe. That make sure we're doing the right things in the world of health care that we are doing get the right way for our patients and we towards the right outcomes. There's still a perception that governance is policing and it is love. It's partnership. And I want to work very hard in order to change that perception. The you know, if there's one thing I hope people take away from our chat here is this the. Genie I doesn't replace leadership it demands better leadership and human in the lead is not about control reinvention is not about technology. Both about redesigning agency accountability and trust with the capital city for a world where intelligence is everywhere personal lives professional lives and beyond and the organizations that we need won't be the ones with the best models there will be the ones that will be able to re-arput how decisions actually happen. Simon, as ever thank you for helping that shape that future out here in the open. Thanks for your time, man. Thank you Andy always a pleasure to chat and hang out with you.