The podcast, hosted by Christine Elwood, features Yolghita Parula Carr, CEO of InverGrid, discussing AI governance. The conversation highlights a critical shift from governing AI models to governing the delegated authority of agents—software that acts on human behalf using LLMs. Agents are inherently probabilistic and can interact with other agents, tools, and APIs, leading to unpredictable cascading effects. Governance must be context-dependent: for creative tasks like generating images, minimal oversight is needed, but for high-stakes decisions (e.g., healthcare, infrastructure deletion), robust guardrails and human oversight are essential. Human-in-the-loop systems often fail due to review fatigue, lack of expertise, or insufficient authority to stop agents. Carr emphasizes that accountability for agent actions always rests with the human or corporate entity, not AI. She recommends defining risks at society, human, and corporate levels before deployment, and ensuring humans have the time, expertise, and kill-switch authority to intervene effectively. The discussion also covers technical controls like policy as code, knowledge graphs, and context engineering to make agents more deterministic. Ultimately, effective governance requires a tailored interplay of technical safeguards and empowered human oversight to mitigate risks and ensure responsible AI deployment.
AI realized the podcast about everything that is new, now and next for enterprise executives deploying AI. Posted by myself, Christine Elwood, and my collaborator, David DeCobovic. Our guests will share what's driving AI adoption, use cases, and business models for the data and AI economy. Welcome to AI Realize, the podcast for enterprise executives leading AI adoption. From tackling security, data, and operational challenges to navigating organizational transformation, AI deployment offers a unique opportunity to redesign our organizations from the inside of the world. I'm Christine Elwood, your host for today's episode. We're talking today with Yolghita, Parula Carr, the founder and CEO of Inva Grid. Yolghita is back on AI Realize. She joined us previously to talk about AI governance fundamentals. And since then, the conversations move from what it govern AI to how to govern agents operating a ton of state and at enterprise scale. Yolghita recently participated in the AI released executive roundtable on governing agents at scale, which was co-hosted with AIGovOPS Foundation. Today, we're going deeper into what she and her peers are seeing on the ground. Yolghita, welcome back to AI Realize. Thank you. Thank you, Christine Elwoods. Always lovely to be here. I'm talking to you. Yes, it's always nice to have you. And you have been building security and governance programs from scratch for nearly three decades. Starting as one of the first five people to build EY India's cybersecurity practice, then as Oracle's first technology auditor. And in those roles, you were deeply involved in the first GRC products before anyone even coined the term. Today is the CEO and founder of InverGrid, your patented platform for secure by design cloud infrastructure. You are talking consistently with executives in the space about how to manage and govern their infrastructure and their deployed systems properly and responsibly. You were recently named the Silicon Valley Business General's Power 100. And you also work with the private company boards through private directors association on AI governance and cybersecurity. We'll talk a little bit about that later in the show. But with that as background, I just want to highlight that at the roundtable, there was a big reframe in terms of the observation that we need to stop governing the model, the AI model, and start governing the delegated authority of the agents, if you will, the space between the model and the action. And from your experience building governance, all the way back at your Oracle days, but also now at InverGrid, what does governing delegated authority actually look like in practice? So, let's even before we jump into whether we govern the model or govern the agent, let's understand the lay of the land, right? We have, where did we start? We started off with machine learning, neural networks, AI, then it became Jenny AI, which were trained on our huge purpose of human historical records and data and became large language model with transformers and attention and all of that. We started using those Jenny AI models, which are primary language oriented and to perform actions on our behalf, to take decisions on our behalf. So those are our agents. So on our behalf. So they get our agency. They act on our behalf. They do what we tell it to do, but they are primarily understanding our instructions using the LLMs. They're using the language models and those language models are highly probabilistic, right? So agents are inheriting the probabilistic nature of the LLMs. In that process, we are trying to make the agents work and output more and more deterministic, concrete and responsible and safe because we are using it to make decisions on our behalf, using it to take specific actions, which can have repercussions and consequences for us as a human and as part of the company. One thing that I just want to be before you move on, make sure that we highlight here is that agents are also talking to agents and are making changes to themselves and sometimes to other agents based on the context they're operating in. So that is not operating on our instructions. That's operating beyond our instructions. Oudley agreed, exactly. That's the simple way to think about how you define an agent is, and you can have an orchestration agent and a multitude of agents working with other agents, with other MCP servers, with API's, with tools that you give it access to and everything that you are connecting agents to, which could be other agents as well, which can result into a variety of cascading effects through the whole time. So you're absolutely right, but fundamentally those agents, and the difference between a piece of agent, which is also software and instructions and between our other traditional software is that it uses LLMs in the background. LLMs to understand human intent. Whether that intent was written in advance as a form of instructions for the agent, but hey, you are a customer service agent. So this is how you will behave. You are a marketing agent. So this is what you will do. So whether it is those instructions and guardrails, we give the agent in advance or the oversight and observability and evaluations we put in over time across throughout its agent's lifespan and journey, it is still fundamentally using, we cannot forget that. It is using LLMs to understand whether it is those intent that instructions or anything. So which is what makes that agency that we give it very fundamentally different from having a software operate on our behalf and on our instructions. So with that, how do you recommend governing that delegated authority? That comes down to again how good your instructions are and the nature of it, right? So for example, in order to make our agents reliable and for us because they are acting on our behalf, it all depends on the context in which the agent operates. If it is to generate, hey, can we print pictures, infographics for something that I am writing or HIG is suggest a blog article or a modified this article that I have written. These are all very creative plays, it can give you a variety of options, you don't need a whole lot of oversight. But if you are giving the agent the authority to whether it is to change your infrastructure or in healthcare settings to suggest to diagnosis for a patient or life and death situations like that. So I will give you a recent example. There is a case of a nurse who caught the agent, a agent recommended to give fluids to a dialysis patient. Oops, exactly, oops, right? So can you have a fully autonomous agent in life and death situations? Probably not. Because we are finally agents are because they work on LLMs, they are they work, they are probabilistic in nature. You can now to make them more reliable, to make them more deterministic. You have a variety of different ways. Obviously one, the model itself is models themselves are improving dramatically over time, to make them better and more reliable, etc. You can use small language models, context specific language models instead of the frontier models, the larger language models. You can do context engineering, prompt engineering, fine tuning, drag, database, specific APIs, specific agency only, small, multiple sub agents taking care of each, every aspect, checking its own work, double checking. So you can do a whole lot of that engineering, you would agent engineering in the background, in order to make it as reliable as you get. But where, what is your comfort level, what is your risk tolerance for the agent to be not reliable, depends on the context in which the agent is used. In a life and death, life sciences, healthcare situation, you will probably not want it to be autonomous at all. You will want a doctor and a nurse, you have to take those final decisions for you on your life. So the idea of policy as code was discussed at some length. And the idea of using a knowledge graph, or excuse me, not a knowledge graph, of using a graph as a source of authority for the agent, the agent can only do whatever is in the graph database. It was another suggestion. There was some other specific cases of here are things you can do between the model and the agent inside the agent and as oversight to the end.
agent. Do you have experience in those that you those three areas that you can share approaches that people can use to govern at those three layers? Yeah, those are the examples I had offered up between the model and the agent. And these are all your guardrails, right. The point I'm trying to make is you can put all of those guardrails in place. But yet not have a fully 100% reliable system with agents. Yeah. And in your case, the example you gave of the health care, for example, a human in the loop is essential and human in the loop was also discussed at some length and how to make human in the loop effective because obviously there's a lot of ways in which human in the loop oversight can fail. And so there was some discussion about that as well. And that might be worth spending a few minutes on to so there's the issue the human in the loop issue of the human not having actual authority to be able to stop what the agent is doing or stop the agent period. There's the problem of review fatigue of it being difficult for a human to evaluate the outputs. There's also basically in the inexperience to as we get further further away from doing the work ourselves. We have less and less intuitive understanding of whether something is correct or not correct. When we see the outputs from the models. So do you have any thoughts on how to use the human in the loop to be more effective and overcome some of these inherent limitations. Let's step back a little bit. You cannot replace human expertise, human wisdom. And you cannot replace the fact that the human will feel a eyes doesn't feel consequences of their action. The accountability of your actions still remains with the human or the corporate, which is also legal entity. Right. As if you consider what entity is finally responsible for the actions that the agent takes the entity that it is it lies somewhere on the spectrum depending on the situation, the human itself whose agent it is the corporate entity whose agent it is right it lies in in that spectrum accountability cannot be transferred to a to AI. It is on whose behalf AI is acting is finally responsible for the action. So where what kind of human in the loop you need is something that you have to decide. And what expertise level of human you need. Whether it is in the loop on the loop inside the agent itself in terms of instructions in terms of what you tell the agent how you make it better and more deterministic all the guardrails that I talked about earlier. Whether it is in the model layer, whether it is fine tuning them are pre training fine tuning what kind of model you choose. To the instructions you give to the agent to the data sets you give to the agent to the tools you give to the agent the API is the context engineering the prompt engineering the harness engineering. Everything that you do which will make it more and more deterministic and again the context in which it operates is it for creating images block articles or is it life and deck. The extent of work you do and the reliability if you say the reliability meter you need and the way they should mind the effort you put in to make that agent itself as much reliable as possible right so it's the. So what particular technical action you take depends completely on the context in which your you will use your agent what layer of human in the loop you will add is it pre approved auto healing i'll give an example when it comes to deleting a database there was a. A recent case again an agent was able to delete the entire database and the backup database production database on the backup database in 90 seconds flat or nine seconds flat or something i don't have the number right but before the human could do anything that was an excessive agency problem. The full autonomous agent where it was just completely able to delete a production database so where do you put that kill switch right where do you put that okay these are the actions you cannot do period okay cannot delete a database you cannot. delete it's back of the database what is it that you absolutely cannot do now I just I give you healthcare example autonomous agent for just corporate production infrastructure example now let's take another example whether it comes to weapons using AI in that right where is let's take that the human society level decisions that we have to make all of that so that's why the context is very important. And it's an it's going to be a very rock solid interplay of what are you going to use that agent for what is the reliably me reliability meter you need which will decide on all the technical controls that you will put in place and what additional human. oversight you are going to need and what point in time before during after or specific actions after specific actions what is it so it is very contextual so wherever you put the human. oversight wherever you choose to put it because obviously it's always needed but exactly where as you said is one of the decisions you have to take how do you make that oversight real instead of performative. define performative for me. think of it as the how do you overcome the human part of the human in the loop the I'm reviewing lots of things and it's easier for me to say yeah this looks okay it was okay then it is to stop and figure out. Is something wrong here especially there are so many layers of this issue of being performative one is just not being rigorous another is actually not having the expertise necessary to actually judge it. A third is not having sufficient time some of these people were given 1.2 seconds in one of the reports to review and approve an output not just to short for any human to be able to do justice to it there are a lot of elements i'm not going to I don't I can't list them all but what I can say is they're all tied back. to humanity to humans being human so if you want to make sure humans can perform their job correctly and consistently and correctly and how do you make that oversight real and effective. Okay to make the oversight real and effective again let's look at what risks AI possesses every carpet and everybody should be talking about what is the risk of this AI fails. And then you decide what level of oversight you need what level of expertise you need and what level of what time you give that expert right you can give them 1.2 seconds. Clearly. clear and by the way this came out to you about this issue of you got to know what the catastrophe looks like. Because you're going to limit the blast rate is she got to know what the bad outcome looks like that those are two more obvious things the real issue I think that we need to press on is how do you make it possible how do you create a situation in which the humans can be effective in their role. And how do you make sure they have the right authority to be able to stop it that's two things that's of course the delegated authority but it's also the technical ability to stop the what's going on. Particularly in a situation where you're using a system of agents and not just a single agent. The more these systems are built we don't know the consequences necessarily of stopping something when it's in flow. So we have more hesitation as a human to take drastic action like stopping something when we don't know what the fallout is going to be these are just humans being people right that they're trying to do the right thing but they don't necessarily do it. And they don't necessarily know whether what they're doing is the right thing and they are afraid or hesitant to take action. So how do you create a context. I hate to use the same term on both sides of this equation there's context for the agent but there's context for the human how do you create an environment in which the human is able to be effective in their role in the human in the loop situation there when they're in that role. Let's start I think we need to step back a little bit right and we understand let's understand the risks of implementing or deploying the AI and how do you govern. The AI whether models to agents everything right the correctly the risks is if the AI agents you're building have risks at any of these three layers at the top layer is the risk to society right the risk for Then it's risk to humans right we whether it's our jobs or whatever and then it's the corporate risks the corporate risk can be your risk to your reputation it could be financial loss it could be operational risk so depending on
on what you are tackling, your answer will be different. The level of risk that you are engaging when you use the agent, what is the goal of that agent? Correct. And what is the risk to achieving that goal and what is the oversight you need? They all have to work together and that needs to happen before you start your AI program. Your, when you start your AI program, you need to understand what this AI is going to be. Do for me, correct? And what its impact is going to be at the society level, at the human level, and at the corporate level. And what is positive impact and what is risk? And based on that, you can, you have to decide what authority the human needs, right? Should this human be given a kill switch authority? How much time it should be given and what is the expertise level? Now, let me give you that another, the same example, very simple example. I can ask AI to give me, I mean, that this was a strategy, this was a very funny example. It's an HBR study, a Harvard Business Review study. They, recent study, they basically asked AI to give me, give strategic advice. Okay, and all they got in return was friends law. You have to have the expertise required. That's right. A nurse taking the, who caught that, this cannot work because if I give fluids to a dialysis patient, it will be life-retening to this patient. Or whether it is the person using that advice from AI on strategy for his company, if he is not well-wrestled and doesn't have the wisdom, then you're not going to your life. You're putting your company and yourself are the human society at risk. So depending on what you're using that AI for, that expertise, it doesn't take away the expertise, it doesn't, because primarily it doesn't take away the consequences. The accountability still rests with us. However, watch autonomous you make that agent. And if the account, so long as you understand that the accountability, and there are court presidents now, which state that the accountability rests with the human in the corporate, right? The company air Canada, the ticket, it had to give it for the dollar, right? The accountability rests with the company. If it rests with the company, then you have to provide the necessary expertise, authority, time, everything for the human. You have to know when the human is in charge, when the system can make decisions, and all of that will depend on what you're using that AI agent for. There's a set of assumptions in there, Yadita, that whoever the human is that's in the loop has the expertise, the time, the attention, and whatever to make whatever decisions are necessary. When we deploy systems, especially for a long period of time, if we deploy a set of agents, for example, for an extended period of time, where the human that is ticked at nurse, it's not always that nurse, it's other nurses, not always a nurse, it might be a different role in the hospital, right? So we do not necessarily have equal humans that are making, that are in the role of the oversight. And so I think there needs to be some things built into the system that accounts for the fact that humans are fallible, are get tired, are not equally knowledgeable, are not equally trained or experienced, and so forth. And their expertise is accumulative effect of having done the work themselves. And so when we have agents doing the work, the longer we have agents doing the work, the fewer humans we have, who know what that is to do that work, right? So their level of expertise about what good looks like in doing the work is changing all the time. Now, in the issue of identity, where this was one of the areas that was identified as an underbuilt area of governance, is the identity of the agents, that agents are still inheriting a lot of their human access, sorry, a lot of their access from the humans that we're doing the work previously. So they inherit human access tokens with none of the scoping or rotation discipline that we impose for humans on those same tokens. So when you're talking about agents operating with their own identities, which is a fundamentally different architecture than what we've used in the past in automating systems, walk us through how an enterprise should set up a identity governance for an AI agent from the beginning. - Actually, even today, software services, use that own identities. So a non-human identity, is not a new thing, right? So agent, whether it's agent or another piece of software, because agent is also finally pieces of software, yeah. So whether it's agent or other software, everything operates today with non-human identities, the non-human identity problem and giving it excessive permissions than what that software needs is of forever problem. The point, the difference here is, in our case, the agents are making decisions and acting on our behalf and those agents are probabilistic in nature. Software on the other hand, which is not interacting the agentic software interacting with LLMs on the other hand, are not probabilistic. They have specific options and they will respond in the software behaves in the same way the first time and the thousand time and the million time. It is reliable, that's a difference, right? An agent on the other hand, can give you a different output at different times. That is the difference. It's the identity, the difference is, it is the reliability because of the language module. - It's also the identity, when you give identity credentials to a piece of software that's deterministic, you know exactly how it's gonna behave forever. When you give it to an agent, it's much more like giving it to a person. Where a person can do different things depending on what the context is they're operating it. It is a different thing. It's not that it's brand new to give identity access. It's that the identity access we're giving them is more human identity access, not to terminus. And that's the real underlying reason for the question. - You made it. - The tools that we're using for the agents can access tools, they can access memory, they can access infrastructure, they can use different components in our systems. The blast radius exists at the agent level and not at the model level. So the containment has to be part of the architecture from the start and not retrofitted later. How do you account for the architecting the blast radius? - Architecting the blast, witty. - Just think about it from a multi cloud point of view. When you're building your containment architecture and a multi cloud environment, it's different than if you're doing it in a very simplistic environment. - Correct. This is where what we have been talking about for a time now and this is actually the premise of what we are building, right? Art security by design. We'll reduce the blast radius. It is secure by design governance, by design trust by design. You, this is also where you need stepping back and when you're architecting, when you're deciding what your agents will do, what actions it will perform, what it is goal, what access you will provide it, right? What is, what tools you will give it, what data it will access, all of that architecting that, that has to be secure by design. So you need to start thinking about that way in the beginning when you decide on what agent you're building and then follow through with, so continuously embedding those controls through the entire process, the development process, the deployment process, right? Making sure that what, and that is what is secure by design and I have a lot of articles written on that, but that is exactly how. So I understand the principles of secure by design, but in the actual deployment in a multi-cloud environment, what does that look like, what does that containment look like? >> There are different layers to it, right? The traditional layers don't go away just because it is agentic, but we have to add now agent, that that surface has increased, correct, with agentic. So the traditional layers of identity, network, software, and basically infrastructure, you have the software itself, the infrastructure it needs. So in case of agents, you, it's going to be starting with the models, right? The, the agent software, where that resides, it could be in a Kubernetes cluster, it could be in a serverless farm, it could be in a traditionally compute, or it could be in specialized, the managed services provided by cloud, like bedrock or word XAI or any of that. Okay, so in the model, the agents where they decide, which is your software code, the data it has access to, and the tools and MCB servers, et cetera, that it has access to. Now this entire infrastructure, may sit in cloud, may sit externally in a different cloud environment, what governs it is, again, same layers, the identity layer, the network security layer that you provided.
And when identity is identity, the permissions granted, the network layer means the connections it can make to each other. So that doesn't go away. So you overlay that to the increased attack surface. The increased attack surface are all of these components that I talked about from model to agents, software to data to tools, MCP servers, all of these components and through all those layers, whether it's identity or network layers. Are the software layer itself? Let's shift gears and talk about your work in the Directors Association, the Private Directors Association. Governance obviously needs an organizational model and not just technology. It's federated for speed, but with the unacceptable risks that are centralized and every risk needs to be matched to an owner. What do you advise boards on AI agent governance? What's the operating model that you recommend? And this is where it's really funny, right? The government's word is thrown around and misused so badly. But let's understand what governance actually is and what it means and how in the AI world, how you translate that for AI itself. And that is what boards are interested in. What are boards interested in? Boards are interested in making sure that they are the agents of the shareholders. Here's the classic agency. They are the investors and shareholders. They have to maximize shareholder value. That is the fiduciary responsibility of the board of directors. So that's a classic agency. Other agencies we know in the human world are power of attorneys. One human gives to another human. I look at even agency for AI agents as that I've given you the power of agents, the power of attorney, almost to act on my behalf to do certain actions within certain parameters. Let's come back to boards. They are maximizing shareholder values. For them, it starts with strategy. And that's the first. Is our corporate strategy going to help us achieve the corporate goals and corporate objectives? For which the money the shareholders are given as money. They've given us this money whether it's revenue goals, product goals, doesn't matter. So starting with those goals and objectives of the company and the strategy to achieve those goals and objectives. Then the next layer is the oversight layer. How am I going to make sure that the oversight layer is to ensure that those goals are met. The strategy is executed well. So I have the right people and I execute the strategy to achieve those goals. Which means an understanding of even what risks will come into play so that the strategy may not get executed properly. That risk can be financial, operational, strategic as well, legal, reputational, primarily into play. So now how does all of that translate into the strategy to achieve those goals, the risks to those achieving those strategy and oversight to make sure that we overcome the risks, mitigate the risks, transfer the risks, but yet achieve or implement the strategy to achieve the goals. Simple for point framework that I think about as performance. That's how they think. Boards think. How do I translate that for AI? Let's start at the very beginning. What is the goal? For why implementing AI? What do I want to achieve? Which area is it marketing, operations, sales, finance, engineering, infrastructure? What are my agents going to do? What are, what is, how am I going to use AI? So that's what's your goal? Then your AI strategy, which follows that goal. How am I going to achieve all of this to achieve those goals? Then the next layer beyond that again, so goals, strategy, risks to those goals. What are my risks? It could be people risk. It could be process risk. It could be technology risk. It could be data risk. I don't have good data. My supply chain risk, which model will I use? From that to what goes into my agent itself, what my agent reacts. So all of that falls in the, all the risk technology risks. Human risk, do I have the right talent to make it possible? Are my people going to be anxious when I blood and anxiety or go out there jobs? The cultural aspect, all of that. So those risks. And the reputation risk if something goes wrong. So based on those risk, I implement controls. I'm, and then I have, and at the board level, I have oversight to make sure that my goals are achieved with the right strategy. The strategy is implemented. The risks are mitigated. Proper controls are in place. And that's what your oversight will achieve. So that's my advice. My four point. To any board to think about it that way, start with the right people and who understand from technology to how to think strategically, starting with the, and get the tech people and the business people together. But the business decides what the goal is that AI. Tech decides how to do it. So jumping from the board back to the execution layer, the operating layer, we've been building governance programs from scratch, really your entire career. If you could hand an enterprise leader, a three item starter list for governing agents in production. What's on it? And in what order? Governing agents in production. All right, let's see. Three pointer list. Where would I have them start for operational? First things, agents operated machine speed. Okay. Developers, humans, or other humans, non techies use that are also using agents to do their work. That is also happening at machine speed. In that context, if you start look how to how to govern those agents that are operating at machine speed and the humans now that are going to have to operate at machine speed because they're using the outputs given by these agents. So how do we? So think about it from, again, yes, those three layers. One is your governance model. If it is stuck today in the slow process and again, what I mean by that is today's way of deploying agents is actually the operational model today is where is still slow is still stuck in the old age. Right. How do we deploy? Deploy is actually going to becoming more and more a bigger problem than developing the agent. Developing the agent is easier because you're using coding agents to develop agents. A thing is fast now. The deployment and operating at that scale is where the problem arises. So your question is superb, right. Because that is where the problem is. So one way is we need to modernize the process for doing that. The deployment and operation. Right. That's number one. Smart process. Now what does that smart process mean? Yeah. The process that process itself needs to be hyper automated. That process is not automated. So agents for agents in a way, automation, hyper automation for doing that, which is what actually we are doing. Integrated is doing. Yeah. Second is embedding the controls. Whether it is secure by design, right. That needs to happen frictionlessly. It's not happening frictionlessly today. And you, you have already talked about the examples I like to give whether it's the ring doorbell or the door that closes behind you. So yeah. So I don't want to reiterate those examples again. But that embedding the controls from day zero to day Z is what we like to call it. So the controls themselves are automatically embedded. So hyper automation or agents for agents embedding those controls automatically at that pace. That's the only way you're going to govern those agents which are operating at machines. We humans who can provide that oversight over this agent declared as well. I really only, I really only have two points for you instead of curry. Those are the two points. You are going to need hyper automation or agents to do covering your agents with good oversight on it from us. And you have to embed the controls right from the beginning. And that's also with oversight, right. Because that oversight decides what controls you embed when great. Okay. Thank you. That was very helpful. So as we get close to our end of our time here together, let's talk about what resources you would recommend for our listeners who want to learn more about you, about invagrid, and about managing agents in an at scale in an enterprise environment.
I would urge people to actually go to our website. We have a lot of blogs and inside articles already. We, and obviously please subscribe through the contact us from and we keep sharing more or follow our LinkedIn page and we keep sharing more blogs. And this is the hot topic. We're gonna see a lot coming from us. Other resources that I like to go to and fall back on the number name are, it's actually MIT's, this navigator. I wanna give a shout out to them and that team. They are also tracking all AI related attacks or AI-based attacks or AI used that that's using AIU. AI. So just to keep tab on what is happening, I love that as a resource. What are the resources do I use? Oh, gosh, that I can point out too. Yeah, how about I realized? And thank you. Yes, I think that's, I think that's right. It is a really good resource for enterprise executives who are adopting AI, deploying and managing AI. So in this AI era, you've been a leader, if thought leader as well as a corporate leader for a long time, it's a different time period to be a leader. In this AI era, what leadership skill do you find most valuable in your work today? If you haven't used AI, use it. Okay, and when you do, you will realize it's strengths as well as it's weaknesses. You will know exactly how to go on people, who are going to be using it? So get yourself knowledgeable in this area. It is highly important. But your expertise and your wisdom doesn't go away because, and that's an important thing for you to remember. Because what AI throws out at you, the confirmation bias, and there's studies and studies on that. How AI comes up with confirmation bias, it will say what you want to hear, 99% of the time. Unless you ask, "No, give me a counterpoint, give me a counter opinion, then it will give you a counter opinion." There's this case about someone thinking that he does discovered a new mathematical equation because AI said, "Yes, this is absolutely new, only to realize that it was all wrong." That confirmation, so you will also realize the weaknesses of AI and what that your expertise, your wisdom, it doesn't go away with AI. It actually increases the need for your expertise, who can put through any trend slap, any confirmation bias that AI will throw out here. But that doesn't go in. That itself is an immense amount. Of course, models will get better, we will get better at doing harness engineering with AI, and it will keep getting better. But finally, the other last thing to remember is the consequences of the AI's actions are still with us at a corporate level and at the human and global level. So responsible use of AI is going to be something that is going to define us as humanity and what legacy we leave behind for our children. I think the Pope would agree with you. If our listeners remember just one thing from today's conversation, Yogiita, what should it be? And why? I think that's the last two things that I just meant. Perfect. Perfect. Yogiita, Parula Korre, Founder and CEO of Invergrid. Thank you for sharing your experience with us today. I was lovely having this conversation with you. [BLANK_AUDIO]
Podcast Summary
Key Points:
AI governance is shifting from governing the AI model to governing the delegated authority of agents, which act on behalf of humans using probabilistic LLMs.
Agents can interact with other agents, APIs, and tools, creating cascading effects that require careful oversight and guardrails.
The level of oversight and technical controls (e.g., kill switches, context engineering, human-in-the-loop) must match the risk context—low-risk creative tasks vs. high-risk life-or-death decisions.
Human oversight often fails due to review fatigue, lack of expertise, insufficient time, or lack of authority to stop agent actions.
Accountability for agent actions remains with humans or corporate entities, not AI, as demonstrated by legal precedents.
Effective governance requires understanding risks at society, human, and corporate levels before deployment, and ensuring humans have the necessary expertise, authority, and time to intervene.
Summary:
The podcast, hosted by Christine Elwood, features Yolghita Parula Carr, CEO of InverGrid, discussing AI governance. The conversation highlights a critical shift from governing AI models to governing the delegated authority of agents—software that acts on human behalf using LLMs. Agents are inherently probabilistic and can interact with other agents, tools, and APIs, leading to unpredictable cascading effects.
, healthcare, infrastructure deletion), robust guardrails and human oversight are essential. Human-in-the-loop systems often fail due to review fatigue, lack of expertise, or insufficient authority to stop agents. Carr emphasizes that accountability for agent actions always rests with the human or corporate entity, not AI.
She recommends defining risks at society, human, and corporate levels before deployment, and ensuring humans have the time, expertise, and kill-switch authority to intervene effectively. The discussion also covers technical controls like policy as code, knowledge graphs, and context engineering to make agents more deterministic. Ultimately, effective governance requires a tailored interplay of technical safeguards and empowered human oversight to mitigate risks and ensure responsible AI deployment.
FAQs
The AI Realize podcast focuses on AI adoption for enterprise executives, covering topics like security, data, operational challenges, and organizational transformation.
She highlights a shift from governing the AI model to governing the delegated authority of agents, focusing on the space between the model and the action.
AI agents use large language models (LLMs) to understand human intent, making them probabilistic rather than deterministic, which introduces unique risks.
The context in which the agent operates is key, including its risk level, such as creative tasks versus life-and-death decisions like healthcare.
Challenges include review fatigue, lack of expertise, insufficient time to evaluate outputs, and hesitation to stop actions due to unknown consequences.
They must assess risks at society, human, and corporate levels, then provide the human with proper authority, expertise, time, and a kill switch for critical actions.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.