A Framework for Choosing Winning AI Use Cases [Agent Readiness Part 3]

Welcome back to the AI Daily Brief today. We have part three of our agent readiness series featuring new far gas bar And we are digging into use cases the AI Daily Brief is a daily podcast and video about the most important news and discussions in AI All right quick announcements before we dive in first of all Thank you to today's sponsors blitzy KPMG rovo and robots and pencils to get an ad-free version of the show go to patreon.com Slash ai daily brief or you can subscribe on Apple podcasts for information about sponsoring the show We are rapidly running out of inventory for q1 now So if you are interested and want to hear more about what's available Stand this a note at sponsors at ai daily brief dot AI Lastly a reminder as always about our AI ROI benchmarking study Thanks to all of you who have contributed now hundreds of use cases if you would be willing to take one or two minutes to do the Same it is at ROI survey dot AI In the first episode we talked about the cultural dimensions of agent readiness with new far introducing her change framework in part two We talked about data and technical readiness Where we have found at super intelligent that data issues are by far the biggest blocker for agent adoption across all the different challenges that enterprises face Today we are talking about what makes for a good use case and where to invest your resources in time With this you're gonna get a little bit of an inner working in the way that we do use case recommendations And it should provide once again a nice actionable framework for how you can think about which use cases might benefit your organization most Alright you far welcome back part three of three This is technically we're calling this use cases But I think this is like to make it practical like go do interesting things kind of section of the conversation I'll let you kick it off and frame it for us, but all right. Yeah, that's a good take I like you said three out of three final part of the agent readiness series And we did save the best for last and that is the use case readiness And we don't just refer to the are there enough use cases in production But rather we are talking about whether there are enough opportunities in the company and often these are not opportunities that the company is able to Articulate but rather ones that were able to identify for them It can be question on whether the business process in the company could be augmented or replaced by agents And whether they have the right mindset in order to do proper use case discovery and execution So that's what agent readiness for my use case perspective is and in terms of the process And because we are all such big fans of frameworks I wanted to cover and this topic following the following steps So we will start with identify and then select manage and track our use cases And I want to give you enough practical tools not only to identify the next agent use case But also to manage it almost like an investment portfolio such that you can continuously improve your use case readiness so let's start with the identify phase and First I want to talk about the sources of ideas for agents in many cases What I'm hearing is that like a CEO or a company that will go into a room and they will say we need to build an agent Let's build an agent and this is probably one of the worst way to source a use case idea Because you'll probably build the wrong agent good source for you to build these agent use cases Will be probably to get it from either buttons up or mid level up because they often are the ones that know best What is feasible and also the other one that needs to be bought in in order to execute the question still remains What are these good use cases and what I wanted to do here is to provide it with some hints for what good use cases are And sometimes what good use cases aren't and I want you to look first for use cases that involve Involves a very complex and a highly changing Decision-making for example to resolve a customer issue will require a different action each time There is a new customer issue being raised on the flip side And that's my very aggressive opinion that in any situation where you can describe a fixed process or a Decision tree with limited amount of branches. You shouldn't build an agent You should just go for the simpler technology because the agent will probably not be worth The next thing that I want you to consider are places where humans are in the loop But they are being the bottlenecks This is often where you will find the goldmine of our life for your use case Because you need a professional judgment and you just don't have enough professionals think for example a legal contracts review Another place where you should look into our places where you need to have 24/7 human response It can be for employee support for customer support and so on and also I want you to think about cases where you want to achieve a high level of Personalization think for example where you want to issue a highly personalized email Not just one where all the text is the same and you say hi no far This is not the one I'm talking about cases where you will issue a personalized outreach at the right time With the right offer and with the right text to get me hooked into your product Next thing that I want you to consider is that I want you to only focus on use cases where you do have some tolerance for errors for example with one of the companies that I work with payroll said we want an agent to replace some of the processes for calculating the employees salaries and after I stopped catching my breath from how Intimidated I was with this notion. I said guys you don't do an agent here You don't even include AI this process you do a simple automation because with salary you have to get it 100% or 200% right so agent will not be the right way to go about this process and until now We didn't talk about kind of the elephant in the room but because around agents there is so much discussion and Sometimes fear around job loss a good place to start will be to focus on the use cases that employees wants to offload Those will be the repetitive or the tedious type of works. You should focus there and then you will create a better momentum for more sensitive use cases and also in cases where the only way to understand how a job is being done is to say hey Sarah can you explain how the job is being done because Sarah is the only one who knows This is not a good place for an agent We need to have a process that is well-defined and well documented for an agent to be able to interject and Lastly we want to have a use case that is extremely measurable Not just to measure the ROI but because these agents are goal-driven entities And if you cannot measure whether you are closer or not to your goal, you cannot implement an agent So if you use all of these hints one thing that you can do at your company in order to initiate the creation of many many such ideas is to have an ideation sprint and These will help you harvest more agent ideas in the company and what you do there Typically you will teach the employees using slides like that or others what agents are and aren't and what they can and cannot do Then you harvest many many many ideas across the entire company and then with this very large inventory You do a very crude Realization of your use case to keep it simple and here I want you to be very aggressive and nip in the bad any use case that you can execute in a different manner Automation or otherwise. All right, so that's the first step With so many ideas the next step will be to select and plan an agent roadmap The selection should always be driven by the holy tree the feasibility the investment and the value And this means that you will need to score each of our use cases accordingly And if you've heard the previous episode where we talked about the intentional opportunism you by now know that I'm very very much into having a low-hanging fruits and focusing on them first and These should be use cases with high feasibility and low investment or ones that are highly critical and start with them and then you will create the momentum of Learning and doing in order to benefit future higher stakes use cases So let's make it even more concrete and talk about The list of use cases that almost all companies should consider and these are ones that we often find ourselves Uncover or recommend companies when we audit them for agent readiness And I think none of them will surprise you but the first one will be the FAQ or the policy bots These are either internal or external And I'm not talking like old-fashioned bots that only know to answer from a predefined set of questions But rather agents that are able to answer complex questions with many nuances in the information sources The other thing that everyone needs and are asking for will be the company knowledge retrieval I think this comes across all interviews everyone needs good access to their data and Even though many companies are already utilizing the Microsoft and the Glyn and other solutions often these are not enough And they need to create either an additional layer or looking for additional Ways to get access to their very specific data that is fragmented across systems and so on the next one will be operational workflow automation so those will be dredge work at the team level like Automated status reporting and many other things that people spend unnecessary Brainpower and time to do and agents can take from their plate and lastly I'm calling them like market watchers These are everything related to keeping a close eye on your competition on your regulation or everything that you need to know in order to do your business well, and you never have enough time So these are the top most prevalent use cases but we do find ourselves recommending many other use cases as part of the readiness audits and I don't believe that anything on this list will surprise you and of course it starts with the top two most common use cases And those will be vertical use cases around customer support and software engineering They come across very often, but we also see in many cases Condemned generation for marketing or other purposes as well as many sales related use cases They come across very very often in the audits and with many companies that we've been working with and this is also something that I mentioned in previous session agents can often help with things related to Contract and regulations and other things and with the process of cleaning your data Which is where there is a lot of unlock with the data access issues and many things that we mentioned before and lastly There are many industry or even company specific often ones that will create opportunities for growth in the company That come across as highly relevant in some of these audits So this is a very rich selection of use cases and what I want to encourage everyone to do is basically to Manage the inventory and the choices of which agents and which use cases you want to pursue like you would manage an investment portfolio as I said at the beginning and I want first to have you balance between Two main elements and those will be the efficiency or the cost focused use cases And also those that are more focused on the growth and when I talk efficiency use cases I'm talking about all the do the work with the fewer resources type of use cases Well growth use cases in my book or everything that basically has an impact on the top line And I want you to balance the two often We're seeing companies highly biased towards the first one of efficiency and not thinking enough about the growth opportunities and Often the biggest value is on the right-hand side of the growth So make sure that you pay a closer attention to those as well and then to continue unbalancing your agent portfolio I want you to look at this proposal and I hear a paraphrasing on the 1970s Boston Consulting work by Gross Chair Matrix It's a kind of a classical and I want you to look at the identified use cases in the lens of the complexity versus the value and of course where there is high complexity and low value and Often we see such use cases just don't go there at all There are of course the low-hanging fruits or low hangers These are an awesome place to start and over time though those should become the thing that people self-serve So these should be catered by agent building platforms or other capabilities rather than having a company focus on and Eventually you should aim to have a handful of what I refer to as moonshots These are kind of the high-risk high reward use cases often by the way They correspond to either a radical shift in how you do the work or they are a gross use case so that's often where the moonshot is and They should be led and executed by a professional and centralized AI team rather than just the best effort in the business units and They require a significant investment and a specialized knowledge. So these are not for the faint of heart and lastly most of your area of focus should be in in this like high or high ish value and decent complexity Because this is where there is enough value, but still you will be able to get agents out the door and By the way for companies that are getting low readiness scores from us in the audit We will never offer ideas for stuff that are more on the moonshot We will always focus them either on the low hangers or the focus areas and over time will encourage them to go After bolder bigger things So there are a few other dimensions that you should probably consider one is to Bounce between vertical versus horizontal agents don't just do one or the other And also build versus buy so keep that in mind as you balance your portfolio and make sure that you have Diverse and well-balanced portfolio of all the complexity Combinations as well as a vertical horizontal and build versus. Okay, so this portfolio has to be centrally managed and vertically updated in order to support the constant learning and the growth This episode is brought to you by Blitzy the Enterprise autonomous software development platform with infinite code context Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise scale code bases with millions of lines of code Enterprise engineering leaders start every development sprint with the Blitzy platform bringing in their development requirements The Blitzy platform provides a plan then generates and pre-compiles code for each task Blitzy delivers 80% plus of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint Public companies are achieving a 5x engineering velocity increase when incorporating Blitzy as their pre IDE development tool Pairing it with their coding pilot of choice to bring an AI native SDLC into their org Visit blitzy.com and press get a demo to learn how Blitzy transforms your SDLC from AI assisted to AI native What if AI wasn't just a buzzword, but a business imperative on you can with AI We take you inside the boardrooms and strategy sessions at the world's most forward-thinking enterprises Hosted by me with any one of more and powered by KPMG This seven-part series delivers real-world insights from leaders who are scaling AI with purpose From aligning culture and leadership to building trust data readiness and deploying AI agents Whether you're a c-suite executive strategist or innovator This podcast is your front row seat to the future of enterprise AI So go check it out at www.kpmg.us/aipodcasts or search you can with AI on spotify apple podcast or wherever you get your podcasts Meet rovo your AI powered teammate rovo unleashes the potential of your team with AI powered search chat and agents or build your own agent with studio Robo is powered by your organization's knowledge and lives on Atlassian's trusted and secure platform So it's always working in the context of your work Connect robo to your favorite sass app so no knowledge gets left behind Robo runs on the teamwork graph at lassians intelligence layer that unifies data across all of your apps and delivers personalized AI insights from day one Robo is already built into Jira confluence in Jira service management standard premium and enterprise subscriptions Know the feeling when AI turns from tool to teammate if you rovo you know Discover rovo your new AI teammate powered by Atlassian get started at rov as in victory o dot com Today's episode is brought to you by robots and pencils When competitive advantage lasts mere moments speed to value wins the AI race While big consultancies bury progress under layers of process robots and pencils builds impact at AI speed They partner with clients to enhance human potential through AI modernizing apps strengthening data pipelines and accelerating cloud transformation With aws certified teams across us canada europe and latin america clients get local expertise and global scale And with a laser focus on real outcomes their solutions help organizers work smarter and serve customers better They're your nimble high service alternative to big integrators turn your AI vision into value Fast stay ahead with a partner built for progress partner with robots and pencils at robotsandpencils.com/aidailybrief If you're talking about gross let's focus on this one on the tracking And the topic of measurement is one that is very important for us I know that I completely disregarded the build phase not for lack of importance But since it's like an entire episode so I'm skipping over the build and moving on to the track side of things And I want you to continuously monitor Everything that you put in production and that means everything And you shouldn't assume that just because you had a very good result in the pilot It will be so in production We often see that once something is deployed the hands of actual users not just alpha protesters Things go haywire and yield completely different value than expected And we've seen that more companies that are being underwhelmed by what they put in production Rather than being pleasantly surprised the performance and quality and cost And especially if they didn't measure enough in a pilot and won't measure enough in production So that's why it's so critical for us to make it very concrete I provided you here with a formula of how you measure the agent ROI So it's going to be like a return versus investment So return should be measured as the benefits from the agent And you should make sure to include the usage and the impact and you should continuously measure It's not easy and whenever there is a judgment call I encourage you to be conservative rather than anything else So you can get a realistic view of what your agent actually accomplishes And you should discount the uncertainty of the benefits So in many cases I'm seeing companies not accounting for the cost and the impact of the errors If your agent causes you to have to have a refund of something then that's a cost of managing your agent Then it needs to be discounted from your return basically and you should subtract it from the investment An investment will include the resources and it should be all the resources What it cost you to build or to buy what it costs you to use and what it costs you to maintain And often we disregard these resources that will get the agent to be sustainable in production So it will be the owner of the user's time and of course the tool and model use And here especially when you do a projection takes sufficiently large buffers Because in most cases we completely underestimate the amount of investment we will need to do And of course you need to take into consideration the cost of your resources And you can assume cost variability because we are seeing that models cost going down However with agents because they become more sophisticated often they will require more tokens and thereby balancing things out So just reevaluate periodically, but there are many hidden costs and you should try to account for them So that's the most technical slide and with that I want to summarize this session and give you a concrete to-do list So first I want you to identify only agent relevant and worthy use cases not vibes And then I want you to select the ones that have enough ROI and enough visibility And manage them as if you are managing your investment portfolio such that you have a diverse Well-balanced portfolio and then rigorously track the impact and adapt over time So you don't just get it right in theory, but you get it right in practice Awesome. So first of all I have to mention this For the first two parts of this series when we were talking about the culture and leadership changes that were required to really do this well And the data and technical readiness We can help in ways like this, you know by providing information best practices comparative things that we're seeing But our ability to help and this is I'm speaking in terms of super intelligent now is limited But with this one, this is exactly what super intelligent does is help create systems for figuring out what use cases You should pursue and it follows a lot of this thinking especially this sort of bottoms up idea of actually discovering What's going to be useful from the ground level from these sort of actual work level perspectives of people So if you are interested and need help with this that is my shill for super intelligent for this episode This is what we do pretty precisely a couple of the things that I wanted to double click on The first one is really really small, but I think I talk about it a lot because it's surprising to people in that portfolio balance The Low hanging but sort of like low nudging into high value It would appear low value, but I actually think it's higher value than people think Is really that company knowledge retrieval internal information sharing type of use case We see this so frequently as a gateway drug for people and what's interesting about it And I think what pushes it from a low value to high value or like low medium value to more high value Is that the value is actually double the first part of the value is of course Actually getting people the information they need what you've set out to do with that agent or that AI But the other part is it is like a light bulb moment for a lot of folks when they use those tools That sort of starts their own individual thinking about how AI can be useful for them, right? When they have a problem that gets solved Much more quickly than it otherwise would have as opposed to for example Just like my email is a little bit better than it was before because I use chat gpt or something like that I think it really is we very often see that being a surprisingly high value starting use case Yeah, it's the magic of context basically Yeah, okay So then the another thing that I want to mention is on the ROI front, you know, I think it's it's easy to talk about ROI It's much harder to figure out how to do it This is something that we're spending a lot of time thinking about better systems for helping people with I think even having some framework like you provided at least gets people thinking more comprehensively about it It is not as simple As performance analytics from traditional SaaS software. Unfortunately, it is not the same as just, you know What percentage of people use the tool you have to add on some layer of exploration and insight to really understand How it all came together, however, it is absolutely the case that broad expectations Of when ROI is going to show up are being pulled forward fairly dramatically right now so KPMG recently came out with their annual CEO study and Among the CEOs they serve it's something like 1300 CEOs all 500 million dollar companies are bigger Something like 65 percent of them thought that it would take three to five years to realize ROI From their AI efforts. So this is just last year only about 20 percent said one to three years This year they just released these results It's now 69 percent. I think 67 or 69 percent say that it's going to be one to three years And 19 percent said six months to one year in terms of how fast they think they're going to actually see ROI right like it that the investment has paid for itself Moreover, I just recently saw actually just before we were recording that morgan stanley has Said publicly that they believe that the cost that they have put into AI has now been made back in terms of value So that they are actually ROI positive ROI is something that has Been lurking as a thing that would be sort of for some time in the future I think that even with all of this happening. It's still complex to understand what ROI is But you're absolutely going to see way more companies showing up and saying No, we actually have figured out systems that we're comfortable with and confident in and are seeing that happen now This is not a some future far-off thing. I think Yeah, I agree. I think 2026 is going to be in many cases They show me the money year for the companies that have been doing the Usage metrics and the let's play with the tools and encourage usage type of situations However, in many cases, ROI measurement is very Involving because they need to do like a b testing and especially on the efficiency use cases often It's very very hard to quantify exactly. That's why I said like take buffers take very aggressive Let's call it discounts on the AI impact. So that there will not be a continuous discussion of Is it really the ROI or is it not the ROI? It's just because people are doing the work better Can you attribute it to the ROI? So I'm having multiple such discussions with some companies and I believe it's going to be even more complex going into next year because the tools are going to be From one hand more powerful, but are going to be also table stakes So it will become like do you measure the ROI of having an excel sheet? Or is it like something that will be a proven incremental value? But even with all that said and done, I believe that you have to measure because when you don't measure It's just a matter of vibes and it's not the way to make business decisions Not when it's year 203 or four of adoption of such a technology I was with a group of cio's I guess a couple months ago now in vegas for a big event that I was keynoting and There was a breakout session after that I hosted to discuss things And one of the things that they all felt there's you know, there's a broad agreement and discussion about Was that traditional kind of ROI frameworks did not work for this and it was interesting basically They wanted to figure out how to measure ROI But their intuitive sense was that these things were obviously having very powerful impacts that you know could be improved But they were not willing to throw them out with the bathwater because all old metrics didn't fit And so they were all looking for new systems that could better fit these tools So I think that this is going to be a big journey this year But boy, I'm very encouraged to have leadership go into that Not saying does it fit sort of like old heuristics? But how do we design new heuristics that actually match what we're doing so we can tell if we're improving how well it's working How well system a versus system b does and ask that sort of questions rather than just kind of bundle it into Thumbs up thumbs down should we or shouldn't we I think there's such an assumption that you know These these things are happening to your point that so much of this is table stakes That they have to figure out how to go beyond that sort of layer one analysis Speaking of which I think the last point that I want to hone in on is this efficiency versus growth thing Obviously, this is a huge sort of bully pulpit kind of thing for me that I talk about a lot as well And my point has never been that you shouldn't do the efficiency thing It's just that that really is going to be table stakes It is you're not going to get gold stars for having 50% more marketing content output when everyone has 50% more marketing content output That's just the way that it's going to be You know, I had a conversation with a professional services firm Who had just gotten out of a meeting with their biggest client and the biggest client told them to their face that they expected them next year In 2026 to do the exact same amount of work for 50% of the price and it was now their job to go figure out how to do it Right, you're not being rewarded for being clever about using ai. It's just what you have to do now And that's why it's you can't just do the efficiency thing because Then you're just going to be at the exact same relative position to everyone else You have to kind of think broadly about where new competitive opportunities lie And that's really the sort of blue ocean space that is opportunity I think that the hard lesson is you kind of just got to do it all. There's not an either or here It's just about doing everything. That's why I like your sort of portfolio approach idea that makes do everything Which is my invocation. Maybe a little bit more manageable people Do everything in a smart and balanced way and by the way in most cases the big bucks Are hidden in the ghost think for example on the long tail use cases where ai has a huge unlock there Of multiple app sales and stuff that you never ever had the bandwidth to do and all of a sudden You can start deploying agents and you have so many millions hidden Just and think that you never prioritize because for humans it never made business sense to go after Yep, super helpful. Again, I'm excited to see how this hits and where people want us to go deeper But thank you so much for this three-part series for those of you who are listening Please use spotify comments youtube email, whatever to let us know what you think about this and what you want to hear next And new far. Thanks again and see you in slack (upbeat music)