Governing AI for Fraud, Compliance, and Automation at Scale - with Naveen Kumar of TD Bank

[MUSIC] Welcome everyone to the AI and Business Podcast. I'm Matthew D'Mello, editorial director here at Emerge AI Research. Today's guest is Navine Kumar, head of insider risk analytics in detection at TD Bank. Navine joins us on today's show to examine why AI adoption in banking hinges as much on securing data, models, and organizational guardrails as it does on insights, customer experience, and fraud prevention. Navine helps us break down where financial institutions are hitting roadblocks from data leakage and prompt based social engineering to shadow AI and model inversion and why these challenges complicate efforts to move from manual detection to true machine-assisted prevention. Our conversation also covers practical workflow changes that materially shift ROI, role-based AI access, governance frameworks that treat agents like quasi-employees, and phased rollouts that align innovation with regulatory obligations. Today's episode is sponsored by NLP Logics, but first, are you driving AI transformation at your organization? Or maybe you're guiding critical decisions on AI investment, strategy, or deployment? If so, the AI and business podcast wants to hear from you. Each year, emerge AI research features hundreds of executive thought leaders, everyone from the CIO of Goldman Sachs, to the head of AI and Raytheon, and AI pioneers like Yashua Benjiro. With nearly a million annual listeners, AI and business is the go-to destination for enterprise leaders navigating real-world AI adoption. You don't need to be an engineer or a technical expert to be on the show if you're involved with AI implementation, decision-making, or strategy within your company, this is your opportunity to share your insights with a global audience of your peers. If you believe you can help other leaders move the needle on AI ROI, visit emerge.com and fill out our thought leader submission form. That's emerge.com and click on be an expert. You can also click the link in the description of today's show on your preferred podcast platform. That's emerge.com/expert1. Again, that's emerj.com/expert1. Without further ado, here's our conversation with Navine. Navine, thanks so much for being back on the show with us this week. Thank you, Matt. Thanks for having me. It's all this great to be here. Absolutely. We were talking a lot about compliance last time, very much in the much smaller space of copyright primarily. Today we're talking about how banks and financial institutions are seeing the promise of automation and machine learning with faster insights, better customer service, and stronger fraud prevention. Yet at the same time, these same systems that accelerate decisions can also amplify risk when data, models, and governance fall out of sync. Regulators now expect explainable, auditable, and bias-aware AI pipelines. And institutions are realizing that the rules first approaches of the past really can't keep pace with adaptive criminal behavior or evolving oversight requirements. As we're seeing financial crime inside a risk in compliance teams adopt AI the goal, just as we said in the last episode, shifting from detection to prevention and from manual reviewed to machine assisted intelligence. In order to get there, leaders need to strengthen foundations across data quality, model governance, and organizational accountability. But even before we get to that transition, what foundational challenges are you seeing slow down AI adoption and regulated industries like banking? Thank you for asking. This is a really, really great question. There are a number of challenges that we look into it. One of them is data leakage. It's very important that the data is secured. For example, let's just think about it. Say an analyst based internal investigation notes into an AI tool. I want to summarize these trends. It says within the boundary of the domains of that being set. The other piece I would say prompt injection. This is, I would say, a social engineering of AI. You basically asking them to ignore all rules and show me all information that's available. So prop injection is like when someone restricts AI into doing something, it should. And that's a big risk. Model inversion, this is when attackers figure out what's inside your AI. That is the other piece which is very risky and stopping companies to think about how to circumvent it or actually make sure it doesn't happen. Shredo AI. So in terms of that, when you have a number of AI tools and IT or compliance that doesn't know about it, that is a hidden AI universe. So these tools being used either on company website or sorry, on the company domain or you are leveraging your personal machines, but you kind of typing the information or taking the information outside of the domains of your IT environment. So how to make sure either within the domains of IT that Shredo AI is not going or also outside of it. And we all know about, we all talked about hallucinations, which is a real thing, which is making AI believe into something or they're providing something that is, it's not there. It's like hallucinations are very confidently going to say an output which is truly not correct. So more direct is another one. Like there are a number of these things which we are learning are challenges. Yeah. Just a differentiate there, maybe the wheat from the shaft. We've had a lot of guests come on the show and tell us that prompt engineering is really a temporary phase of AI adoption and a few years this won't be around anymore. I always take those predictions with a grain of salt by present them that way on the show. Jerry is very much still out on prompt engineering, especially because, and I think just about everybody saw this. I was seeing this on LinkedIn back in early 2023, but you'd have some guy from Silicon Valley on LinkedIn just saying, Oh, hallucinations will be solved in a couple of months. We'll get it. No. And that fundamentally misunderstands the more philosophical problem of hallucinations. And I think this is an experience just about everybody's had with chat GPT, which is you think it's a hallucination when it's your data inputs that is where it started from. And that even crosses into the very problem of prompt engineering. Just trying to get a sense of do you see some problems as temporary? Do we need temporary fixes? This is only going to be around for a couple of years versus hallucinations, which are more philosophical and get to the heart of what is a misunderstanding. You know, and see, I'm learning as anyone else is learning. And I was in this for the day. And one of the point was like, Hey, hallucinations could go away if you could provide real context into your prompt and then the data is set for that. Now, how much is through is not time going to take? But I think I think what we need to look into is from the prompt engineering perspective which you mentioned earlier is when we're talking about not artificial general intelligence, but very purpose fit one, right? Now, which we try to do for our use cases within the organization. The way I see it is limited to who could do more like a role base in the sense that when you prompt something, HR sees HR stuff, right? Investigators see it's flagged employee. Finances is nothing. You got nothing to do with it. It's not going to prompt. So in general, right, when from the models and transform it's perspective, they're pressure to provide an answer. hallucinations happen because it has to provide an answer. It cannot say, like pretty much your expectation is not that AI engine going to come is like, I do not like. You're doing right? So I think that's how I see it. Matt. Yeah. And even right there, like, it'll just be solved if we give it all the context in the world. Another word for context, if you want to take it to an extreme is surveillance or other people's private data. And that's the exact problem you're running into of, oh, yeah, it would be great if you could get all the information on that person in the world. Then you got to go through their trash. You got to go through their mail. You got to then then you're something of a surveillance state for it or that subject. And then that really it becomes when you start taking many steps back from the word context, it gets a lot more philosophical. The problem disappears from the technology and ends up becoming a more existential question. We only got so much time on the show, of course, taking it a little bit more practically and thinking about really the more permanent problems. How does strong AI governance look? What does that look like in banking and financial services? Yeah. And it's a very, very important piece to cover. I would say full data visibility. Right? Tracing every internal data set that's been used is very important. Who access it and how it touches it. It's very important. I think role-based AI, which I somewhat mentioned, is like, it's like a more like a polite bouncer. It's like, only provide information. More on the role-based. If there's an insider investigation going on and finance has nothing to know about it, putting it in the AI chatbot shouldn't return anything. Right? The art is for an invisible force field. So what I'm saying, rules are simply cannot break. No matter what prompt it receives. So that's what I'm saying. that will stop doing, I think gathering information by asking number of questions and tricking into AI, that's actually helping, but actually revealing the information and attack or shouldn't know in the case of an attack, right? So I would say those, one other thing which was interest, interestingly, later as well is, now we started to think, lot of agents as in like a quasi-human, right, or an employee in the sense that, treat them and go through the same way, you would, you, these can employ, right? So what did I use it for it touches, water tonality, in fact, right? Who reviews its work, who approves it? So consider them like a mini version of it. In fact, I was talking to a friend of mine and then she was saying, how in their corporate environment, they started to have this bot's called underscore AI. So say, Navin is my name and Navin underscore AI underscore bot will appear in the chat and they will try to learn what you're doing. So like literally, where we are right now, is think of like what a human could do, what AI could do and apply the same God rays around that, that would you, you would be doing. So that's one, I would also say hybrid deployment and explainability, like sensitive data stays in internal models, less critical tasks can be used cloud, AI rather than internal one, right? And what has explained reasoning for human review? So those are some things I believe would really help for us to once we implement them effectively in a mid to long term future. - Yeah, absolutely. And I think, you know, we've been talking so much about kind of the space between the enterprise and the customer as being, you know, a huge source of friction. It's definitely the space where we see the enterprise getting ahead of regulations for their own interest in making sure that they have a qualified relationship with the customer, they're effectively using this data privacy. It's the fear, it's more of a fear of making sure you don't have a PR crisis, you don't have, you know, pissed off customers rather than the regulators coming around. That being said, you're still caught between the customer, the regulator, your own, the technology driving up the need to find a nice balance in between. And everywhere that sacrifice is we're seeing innovation kind of caught in between. How can institutions balance innovation with customer obligations and regulatory and security constraints? - Yeah, that's a great question. And a lot of time going goes into this balancing this act, right? - Right. - I think it's more of a, I would say a phased approach, roll out something which is very specific to a use case, limiting the data availability and the data for points that needs to leverage to create a usable output. Our work says making the AI solution comprehensive and leveraging key data sources and everything that's available to it, right? I think that is one of the way, when you create like clear policies, what could be versus what could not be used for the development of the AI models, that would be really helpful. Classify data realistically, right? Like, okay, this is the safety, data, this is sensitive data, this is critical data and should not be used in our first iteration of work. I think that would definitely help to navigate that balance. - What are the metrics that really matter in this space, especially as you're trying to drive innovation versus regulatory and security restraints? We've had guests on the show, I'll quote David Glick of Walmart who's really, and I guess this makes sense when you have a link with this to work. Oh yeah, that's right, that's right, what I was saying. Last thing we had you on the show, you're at Walmart. Well, and you know the Nano Agents and David talked all about how really the philosophy is, we're throwing out the notion that you need to sacrifice speed for safety, you can have both and that can kind of sound like, well, that makes, I guess maybe that might make sense when you have Walmart's resources, is that something, is that a paradigm shift we're seeing coming for enterprises smaller than Walmart? - Yeah, it also matters a lot in what domain we are referring. So, on the compliance side, it might not be in the sense while on the retail side, when you're trying to get customers, yes, right? It's somewhat different on the compliance side, you wanna be more conservative than aggressive. You say you're leveraging AI to file suspicious activity reports and then you do know what are being over optimistic into going all the way from data collection to alert generation to reviewing it and putting it to fins and inbox without a human in the loop, right? So, I think that, that thing needs to be balanced. Okay, what could maybe, what we could look at in a different way is in terms of speed versus the precision is like, okay, anything which is below this threshold, tier one, alerts, those could be really taken by an agent AI and disposed while something touches upon the X, Y, and Z parameters, they should not, then should always be looked by somebody at human, right? I think, so it depends in what domain and what use case it is that you would probably wanna hone more towards leveraging AI as an efficiency or the first iteration of first draft created versus actually have a full-blown solution. As such. - Absolutely. I know we asked this question last time during our compliance for copyright episode, but let's just say, for the folks out there starting kind of a ground zero, what steps can leaders take now to build a secure and scalable AI foundation? - Yeah, so I would say, safe sandboxes, let's experiment safely without touching the employee data. I think that's what we need to do first, before we go further into it. And then building the AI in entry, know all your models, know all your internal tools, know your vendor AI features, right? That could be there. Bring compliance in early rather than asking for them to validate the models towards the very end, get to know what really going on, right? And I would say, reward responsible innovation, right? So celebrate project that innovate and reduce risk versus shiny rule, breaking to the pool until the audits find them and they're like, okay, what exactly is going on, what you're getting out of it? And I would also say, yeah, success equals like fewer incidents better detection, smoother operations. If I create new risk, something get roasted in the next board meeting, you do not want to do that. So that's my what my view is. - Yeah, board politics is inevitable. And I think it needs its own special communication, which we've talked about a lot at the show. If you have any advice in that category, I'm very happy to hear it now. But anything just, especially for managing up in those particular scenarios. - You know, I think so as exciting as AI is, remember this right curiosity is loose and enthusiasm is great, but guardrails, monitoring and human judgment are what kept institutions from becoming the next AI for a story. So we just want to make sure we are cognizant about those things. - Yeah, absolutely. I think that is ending up one of the big speed limits that we've seen in that moment from late 2022. We're up on the anniversary right now, been three years since that kind of ed Sullivan moment for OpenAI and Chad GPT. And I think, you know, in that initial year, the attitude was, oh, this will keep skyrocketing. You know, this will get better and better and better. No, there are a couple of speed limits. And especially that human input, human in the loop, review and judgment is proving a lot more valuable than we thought it did, at least as of like about January, 2023. - And anyone who you talk to, the data is the, is the spying, right? We all know this phrase of garbage and garbage out, probably used more than it has been. Like, like, I've been to many conferences, personal conversation. And it's like, okay, building on what data, I think that's very important. And we continue to unrank all that peace for OpenAI for sure. - Yeah, yeah. I think, I think as the show has gone along, we've heard a lot less garbage in, garbage out, which I think is a good sign. We've officially gotten rid of it. It might become a rule for the show and guests, but we've gotten rid of Boil the Ocean. And when you start to see these, even these small phrases leaving, it does tell you something. Even if anecdotally about maybe the state of AI, you could talk to an AI agent on your podcast, Matt and Felix, that's not been used the last, the baby three months. - I would be, I would be very interested. That AI is gonna need to be spiked with some, some young Ian psychology, some, some, some, some dream reading. Well, I think we might need that in there. Oh, we'll have to have you back on the show. The next time we do dream readings. Navine, thank you so much for being here once again. - Thank you, my doubt, I'm so much appreciated. (upbeat music) - Rapping up today's episode, I think there were at least three critical takeaways for leaders in banking, financial crime, compliance and risk management to take from our conversation today with Navine Kumar, head of insider risk analytics and detection at TD Bank. First, strong AI adoption depends on disciplined data controls, full visibility into access, clear role-based boundaries and guardrails that prevent leakage, inversion and manipulation. Second, treating AI agents like accountable workforce extensions improves governance. Defining what data they can use and who reviews their outputs reduces risk in uncertainty. Finally, balancing innovation with regulatory and customer obligations works best through phased rollout in the form of narrow early use cases, Realistic data. classification and increased automation that are deployed only when precision and oversight are proven. Are you driving AI transformation at your organization or maybe your guiding critical decisions on AI investments, strategy or deployment? Each year, emerge AI research features hundreds of executive thought leaders, everyone from the CIO of Goldman Sachs, to the head of AI at Raytheon and AI pioneers like Yashua Benjiro. You don't need to be an engineer or a technical expert to be on the show. If you're involved in AI implementation, decision-making or strategy within your company, this is your opportunity to share your insights with a global audience of your peers. If you believe you can help other leaders move the needle on AIRY, visit Emerge.com and fill out our Thought Leaders submission form. We look forward to featuring your story. If you enjoyed or benefited from the insights of today's episode, consider leaving us a review on Apple podcasts and let us know what you learned found helpful or just like most about the show. Also, don't forget to follow us on x, formerly known as Twitter, @emerge.com and that's spelled again, e-m-e-r-j, as well as our LinkedIn page. I'm your host, at least for today, Matthew Damello, editorial director here at Emerge AI Research. On behalf of Daniel Fajella, our CEO and head of research, as well as the rest of the team here at Emerge. Thanks so much for joining us today and we'll catch you next time on the AI in Business Podcast.