From Core To Edge: Akamai On Where AI Inference Must Live Next

[Music] How much faith should we really be putting into AI right now? And what happens when that hefty cloud bill lands on the boardroom table? Well, my guest today, he's somebody that has the answer to that. He sits right at the intersection of cloud infrastructure, AI inference, and the very real world pressures to make this stuff work at scale. So from the shifting economics of AI to the growing role of the edge, my guest today spends his days thinking about where decisions should happen, how fast they need to be made, and what happens when trust breaks down. This is a conversation about what this year might look like as the hype that surrounds AI continues to fade, and the trade-offs that we're seeing become unavoidable and must be challenged and tackled. Big question, what does smarter, more disciplined AI adoption mean for businesses, and indeed consumers alike? Well, it's on that note that I will officially introduce you to my guest now. [Music] So a massive warm welcome to the show. Can you tell everyone this thing a little about who you are, and what you do? Yeah, absolutely. My name's John Bradshaw. My role is really to help articulate what we do as an organization, Akamai, how we're there to support our customers. And if I can use some of the experiences I've had over the years and in different roles to help translate some of the more complex or esoteric bits of technology into something that's meaningful and useful, then that's what I do. But the actual title is Field CTO for cloud for the Amir region here at Akamai. And there's so much I want to talk with you about today, because I think when boards look at AI spending this year, many are questioning whether the returns justify the compute and GPU costs. So from your perspective, what you're seeing with your conversations from your customers and beyond that as well, where are enterprises typically misjudging AI or AI today? Tech does look a couple of good acronyms out there. And also what separates the projects that deliver those real business outcomes from those that are quietly paused or stuck in pilot purgatory. You're absolutely right. And I think there's a big difference this year. And 2026 in my mind is absolutely the year of ROI. We're going to see organizations move away from personal productivity improvements. We've seen some of the big hyperscalers, some of the big enterprise collaboration businesses revise their numbers around what they think they're going to be able to do this year. And a lot of that's to do with the difficulty in measurement. So if you're able to save each of your employees five minutes a day in what they're doing, that's great from a quality of life point of view. But there's no FD in the world who can measure that and say, "Yeah, I managed to return money million dollars worth of productivity to the business. It's not going to cut it." So this year is going to be focused on how do I grow my business and how do I materially save money within the business? So for those two sides of the coin, you've got tools and programs that will, let's say, drive hyper-personalization, create a meaningful relationship with your end customer and therefore help increase attachment rates, improve, check out experience and turn over more all the way through to reducing costs. So how do I avoid another higher, how do I redeploy personnel or materials equipment to the right location? So I'd say you're selling ice creams around the country. How do I use AI and weather data to predict where actually this this group of people would be much more interested in buying ice cream even though it's raining? I say that living in Scotland to be have some amazing ice cream shops that are open all year out and even on the coldest day we want to be buying ice cream all the way through to how do I make sure I've got parts for jet turbine engines that I think might go on the blink shortly and how do I do predictive maintenance on that while I need AI? I need the people, I need the materials in order to do that. So it will be that shift from personal productivity into gross and cost saving at scale for business. Another big focus for many companies now is the rising cloud cost but one of the things that attracted me to you is you've spoken about cloud costs becoming a lever rather than a limitation. So how should leaders rethink cloud and AI architecture decisions? So things like performance, cost control and innovation can all coexist rather than compete against one another. So I've been fortunate to do a number of different roles both customer side and this side of the table where I'm trying to articulate value and how to work. And what I've noticed is those teams that can partner with the rest of the business to drive outcomes get more flexibility in our better place to drive change. So if you can go back to your business and say, look, if I save you a thousand bucks this year, can I spend off of that doing this cool thing which I think might pay off in a year's time? Well, pretty much everyone that you meet is going to go, yeah, absolutely. I will take that. That's sort of shared benefit or shared outcome approach is brilliant. Now if you can do that, you can start to demonstrate to the business your value immediately from the IT side of the world and start to put things in place to support growth objectives or help your team train or build additional skills within within the department. That lets you use cost as a lever. So it's okay if your cloud then is going up. If that's commenced with your revenue, it's similarly okay if it's declining and that's commenced with all of your other costs declined that your revenue is staying stable. So it's about using it as a lever in order to define the business outcomes. If you do that, if you get your teams to buy into the vision or the value you provide, then you've moved out of that service provider model into an actual partner with your business. And that's a very different place for internal IT teams to be. And here we are recording this towards the beginning of 2026. I've already been through to a through I've already been to a few tech conferences in the US predictably. All the big topics are around agente.ai and looking with AI agents. So one of the reasons I wanted to bring that up is you predicted that by the end of this year, people will routinely delicate every day to ask two AI agents without second guessing the outcome. So what do you think needs to change technically and probably most importantly culturally before that level of trust becomes normal behavior? It might feel a little bit off now, but I agree with you towards the end of the year and next year, this is going to feel normal. But what needs to change to get to that place? So it's a funny one. I think of agente. So I think of AI as a whole as it's kind of a really eager intern. Now I've been an intern. I've been a grad. And when you're in those positions, you desperate to help. And you'll always say yes and you'll want to do stuff. And AI has been very guilty of being overly positive and always trying to confirm what you want to say. I think we're going to see a move now to it becoming a more stable and responsible partner to you. So out of that, rad into the two or three years business experience kind of person that starts to demonstrate that about you. But that's a lot of that's to do with things like the ability to influence system products and to demonstrate the way that you can reflect your own controls within that. And we'll see this move away from it just being really happy to ease you to more challenging and responsible. But that goes hand in hand with the regulatory frameworks that we all operate under, maturing at the same patch. So three or four years ago, the idea of using AI to do anything seemed a bit as an o it's a bit not whole. I'm not sure what we're keen on this. And we're starting to see more and more people get comfortable with it generally. And as we progress through the year, it will it'll get to that point where it's considerably more ripe than it's ever raw. So we've moved out of that confirmation bias hallucination mode still happens on occasion as we've seen that out with some public stuff recently to it generally being absolutely right. And if you're saying to a gem to look, could you book me a doctor's appointment for Friday, please? That's hard as a process to mess up unless it suddenly decides you're in Madrid and Tristan, but she would have five in there. That will be a little bit harder. But we'll also start to see it reflect into some more business processes. So how do I get my agentic system to react to a customer demand? That is much more intelligent than those older chat bots, which were essentially press one for this, press two for the other, into a more dynamic system, which is able to make decisions independently. So you'll see, let's say things move away from having to phone into the call center to cancel an order to being pretty comfortable with a web chat with your agentic bot able to go. Yeah, okay, it's your bar. This was your order number. This is how much it was due to come on Friday. I'm going to cancel that one, but not cancel the order for Saturday. So it will be able to cope with more complex demands on it. But this is going to be a comfort level thing that happens over time. It's not going to happen by the end of this quarter, but I definitely think by the end of the year, it'll almost be second nature. And when I was doing a little research on you before you joined me today, I was also reading that edge AI plays a central role in your outlook for the immediate future too. So on that side of things, why does moving inference closer to use this? Why does that improve reliability and trust so dramatically? And for people listening, what practical differences will they actually notice in their daily digital experiences as a result of this? Do you think? Yes, and there's a few elements in that. One is simply around speed of response. So if you land on a web store site, you need the recommendation engine to be as close to instantaneous as possible. We all know that for every 10, 100 milliseconds of delay on a site, your drop off rate massively improves. So if you're having to backhaul a recommendation from LA all the way over to London, that's not going to be a great experience for anyone. But the second part of that is on the trust side. So consumers are becoming more and more conscious of privacy as our businesses. They don't want their data to traverse geopolitical boundaries or national boundaries. They want it to stay within the area that it's protected that they designed their control sets for. So it's doing inference within a few miles, tens of miles of the end user is a better experience, but it's also much more compliant as a consequence. That then triggers into these agenteic workflows. So if you're trying to manage process with lots of different inputs from all over the place and let's take a supply chain one where you're trying to just in time pull apart from a different store and then send it on a robot somewhere and all of those sorts of workflows. You can't be moving that data and the decision-making process everywhere. It would be the same as phoning Hong Kong to see what part number needs to go into this thing and your manufacturing plant is in burning it. Why would you bother doing that? That just makes it mad. I also think over the last 12 months we've seen a lot of outages and security and security incidents and as a result confidence in things like smart homes and connected devices have all been so much shaken. So how do you see things like hybrid cloud and edge architectures maybe helping to restore trust and what mistakes should manufacture as a void repeating as well? I'm sure you you see and hear lots of stories around this but tell me more about them. Well I think it's a challenge where people have decided right we want to make something really clever. Yeah absolutely. That makes perfect sense and the best place to get lots of access to compute and and all this other technology is in these big data centers in different parts of the world and that's great when everything's working. But as soon as you have a disruption on that line it could be your local broadband. It could be DNS going down. It could be any one of these things. Then you've got a very brittle workflow and there was some stories of people's smart beds not working because they couldn't connect to their data center in the cloud and therefore couldn't adjust the size and bed. Why do you have to go all the way to a major data center to adjust the level of this bed? That doesn't make any sense. So what will start this the easier? More of a continuum of compute. So they'll be on device capability to cope for out out not quite out of band but if it's disconnected for a period of time all the way through to what can't the device answer because it hasn't quite got enough compute locally to it. So a small specialized language model isn't going to quite cut it. So we might need a little bit of extra oomph to use a technical term in the edge to try and answer that question and if the edge can't answer it right well then I'll expand out to an even bigger capability. And that approach coupled with not putting all your eggs in one basket so not just picking a single DNS provider or a single GPU provider cloud service whatever it happens to be. And I'm running in this heterogeneous type environment where you can move workflows where they're going to be most secure, most performant, most cost appropriate. It is going to help balance these things out. And what I've certainly seen around the AI space is people aren't looking at their existing cloud provider to do their AI service. And some of that is a asset recognition of cloud locking. Having been a problem for the last 15, 20 years but no one really addressing it all the way through to this market is moving so quickly. Do I really want to place a bet with provider A when BCD, E and F have this really new capability that looks amazing that I want to try. And that can be anything from some of these coding agents which are really nifty through to co-working tools or responsive chat services. That ability to go, no, I don't want to be there today. I'm going to go over here and back by Sturcer. That makes a big difference. And the ability to start a b testing things as you're working through that becomes critical in taking best advantage of those services. And if we look even closer to home how we access entertainment now, I think many people over the years have cut the cord to traditional cable and satellite packages in favour of streaming services. But they now find themselves with an increasingly list of subscriptions and saying to themselves, why am I paying a company to just watch ads in a lot of these services. And streaming services themselves, they're facing rising costs, fragmented audiences, and the dreaded subscription fatigue. So as this market matures, I'm curious, how do you see AI and infrastructure choices shaping which platforms might thrive and which might struggle to keep pace. Because it feels like there's a lot of competition in this area now. Yeah, I agree with you. I think it's interesting. Certainly after COVID, we had the explosion of providers with specialisms in different kinds of content category. And I know when I travel for work, I seem to come home and the kids have spent up for yet another streaming provider. And honestly, I feel like I spend more now than I've ever spent on entertainment. I think the distinction is going to be in that user experience. And I don't know if you've you've experienced this, but I know I've spent 20 minutes going through a catalog and going. And yeah, that's that looks interesting. I'm not quite in the mood for that. Maybe this, maybe that. What I haven't seen yet is any catalogs that reflect that in a deeper sense. So yes, you'll get, oh, here's an action movie or here's a set of movies we think you might like. But what they tend to lack is context. So what I might want to watch on Friday night after we managed to put the kids to bed is not the same thing as what I want to watch at lunchtime on a Tuesday. It's just not. So that lack of both location awareness, temporal awareness, those contexts are missing from those catalogs. And the organizations that can can actually start to build a profile of their users and reflect that are going to be much more successful. Because I don't know about you. I cannot spend 20 minutes going through a catalog and going. And by which time I really need to get, if you have some things, I know they're going to be up at six in the morning or whatever it happens to do. Yeah, I'm completely with you there. And even if this, I'm just a film that you want to watch, find out which platform that particular film is on. There's one service I use called, I think it's just watch or just watch it. And that will tell you where everything is. You tell it, what platform you've got and it will tell you where everything is. It makes it a little bit easy. But you do still find yourself searching 20 minutes for content, which is not ideal. And of course, Outside of home entertainment, another big topic this year is AI inference. We've already mentioned it several times in conversation today. So why do you think this year is the moment when centralized inference will start to become maybe a bottleneck and how does pushing decision-making closer to the edge? How does that change what applications are possible? Any big changes you see here? Yes, I do. And I think a lot of it comes from the growing understanding that there is a difference between usigning up for a service that sits on your laptop and costs you 20 pounds a month or whatever, through to an enterprise scale version. Now we've all seen the different models from the the different providers and they are they're exceptional. The challenge is that they pretty much contain the sum of all human knowledge and experience to date. And there's a consequence to process that to refine the model to build the next five, got three or six, or whatever the model ends up being. Requires thousands, hundreds of thousands of GPUs, crunching through all of these data sets in real time or near real time. And the the outlay for these providers is in the millions if not millions of dollars. The problem with that is that if you're running a website that sells trainers, you're having to pay for that. Now do you need your inference engine to be able to write you an essay on the pennepleetian war? Or do you need it to tell you which trainers go with which pair of jeans? Yeah. And the consequence is your tokens cost orders of magnitude more because you can't make that that refinement. These edge services that organizations like ourselves provide allow you to take a thugs set of those models that are parameterized or shrunk to just address the challenge that they have. And you can operate them at a tenth hundreds of thousand of the costs. Now if you're wanting to scale out or sorry scale up a a chatbot or an inference service, that order of magnitude becomes a really big issue for you. And it can be the difference between going bust and being wildly successful as a as a business. So this move to the edge allows you to bring a lot of that intelligence or rather the intelligence you need without having to have all of it there if that makes it. Yeah and I think as these technologies begin to not only evolve but converge, we're also seeing growing interest in orchestrating multiple AI models and services together in a bit to improve things like accuracy and resilience etc. So how do you see this shift towards model orchestration changing the way enterprises might even design trusted AI workflows throughout the year? Any big surprises here or any changes you say? Yeah so we I spent a long time speaking with people around say AI gateways and orchestration as an approach because yes the big two big three providers of models have really interesting capabilities but they're not identical and some are better suited for this task and some are better suited for the other. So coupling that with your ability to provide guardrails and begin to test things as part of that orchestration comes really interesting. So do I like model A from provider A or is model B from provider A better or C or so on. Now you can't do that if you've got to manually cut over code all of the time but you might also want to go I'm going to get the answer from model A but I just want to run it past provider C's model just to validate it because yes you get hallucination in these things but also it maybe its context isn't quite as up to date or it's got flooded with context and and therefore has got a bit of hilter in the way it's trying to answer your question. So that validation step becomes really important. There was some issues earlier last year where consultancies were generating reports for even government clients and some of the data wasn't as accurate as it might have been and your ability to validate that with a second or a third model is going to help you produce better results for your customers but also protect your reputation as you as you look to scale out that technology. We've covered so much in a short amount of time today and just listening to you it's easy to say just how passionate you are about the topic. I see. So for anybody listening though maybe they want to continue the conversation with you or keep up to spend with some more of your musings throughout the year. Where would you like to point everyone listening and of course anyone wanted to find out more about Accomi2? Actually well Accomi.com is it's a great place to start around all of our capabilities. I'm on LinkedIn as John Bradshaw but equally I write there I'm I also write on other publications and other services but almost all of that is linked from either my LinkedIn profile or accomi.com. Oh so well I will add links to everything and I do urge people especially to follow you on LinkedIn now. I mean in a 30 minute conversation today we've talked about the AII Imperative Trusted Workflows Smart Home Backlash next phase of streaming and not to mention Accomi's AI Influence Predictions and there is much more to come so please I urge them to check that out but more than anything just thank you for showing your time with me today really appreciate. Thank you. It's been a pleasure. I think there's a lot to sit with after listening to this one. From the rising scrutiny on AR return on investment to the idea that trust in AI agents will very quickly feel routine rather than risky. I also enjoyed talking about why inference is moving closer to users, how outages and security failures are reshaping expectations and why orchestration, resilience and transparency, how these things matter way more than shiny demos at a tech conference. And I think the most important thing here is none of what we talked about today points to less AI. What it does point to is better choices. Choices where intelligence lives and how it is delivered. And if this conversation may do rethink how AI shows up in your own work, in your home or in your business that alone is a signal worth paying attention to. But over to you what decisions will you trust AI with? Which ones will you still want to keep close to home? I'm sure each and every one of you have different answers and I'd love to hear from you. Please techtalksnetwork.com send me a message over there, connect with me on socials and we'll continue this conversation. But as always thanks for listening and I will speak with you again tomorrow. Bye for now.