ROI, Tokenomics & the AI Bubble | AI Unplugged Ep. 15

AI is getting expensive. - Why don't they just gonna drop the price in three months? - Big multi-billion dollar, circular funding deals we hear about. All of them had a massively negative ROI. - And that's where the real return kicks in. - Even 30 days ago, customers were not having that conversation with us. - Might actually slow you down. - Mibers is build. They've decided to buy. - We're already seeing the natural evolution of people being more productive. - AI has propelled now space travel, rocket development, the next generation of internet. - AI in space. - We wouldn't turn it on, and thus we had controls around this. (air whooshing) - Welcome to AIMplugged episode 15. I am joined today with Jeffrey Valentine. Andrewson Clear, and the one I only, Justin Copic. - Why did you call Jeffrey Jeffbury? - What, is this formal name? - This is my mother-in-law speaking at this point. - Is your given name, right? - We don't use that name. - Which of you will name again? - You just told that Jeff hates it. - Oh my God, let's move on. - David, next we're gonna talk about AI. - Jeff's name, is it? - Yeah, anyway. - Hey. - Jeff. Today, we're here to talk about a whole slew of things, ranging from new models like Fable and Mithos. All the way through tokenomics, and why AI is getting expensive. As well as the impact it's gonna have on the economy, we think based off of a number of IPOs launching pretty soon. So, about to get into. - Let's do it. - Like going on. - Expensive, that's for sure. I remember last week we were talking about how a lot of the jobs that were optimized for agentic tools are now kind of facing some regret and backlash because the cost of those services is more than the employees that they were meant to replace. And there was just a study that came out from report about the financial times, looking at the implied return of all this hyper-scaler AI investment, these big multi-billion dollar circular funding deals we hear about. And of Microsoft, with Azure, Alphabet, with Gemini, Meta, Oracle, all of them had a massively negative ROI. The only one that showed positive was Amazon. - Right. - With everything they were doing. Here's a question, Nova. - Why? What's behind that? - Why would that be case? - I got a guess. - Yeah, I'm a theory. - I'm not sure. I think we should ask somebody, but my guess is that Amazon is spending relative to enabling other platforms and other models more than they're spending on their own model. - Yeah, so less R&D. - Less like, Al-Alem development is expensive and doing, keeping up with the Joneses is like a lot of money. But if you're mostly enabling the, maybe in a traffic motion on AWS or the OpenAI motion in AWS, you don't need to invest as much in Nova. - There's no, no, it's huge partnerships around both of those in those months. - There's so much. - That's making it more shovels. - Yeah, so how does that relate to Oracle? Because Oracle was the biggest loss at a projected 35%. - That was a RY number though. And that was, they might be spending a lot and not showing a lot of revenue from it. Maybe it's the same spend, but not a problem now. - Is the economies of scale of the infrastructure? Like is it that AWS is so far advanced? - It may be, maybe it's a far advanced. - Right, and they haven't seen any return yet. - Right, so it's projected return. - Oh, it's predicted, okay. - I mean, from Amazon lens, I know they have released less and less of their Nova product line, as of late, not saying that's good or bad, but they have, right? And I think that speaks to what you're saying, Jeff, it's like, look, they're spending more time on selling shovels, enabling other partners to build their models and host it. Which to me, is a good thing. As Amazon and its purest, is them as an infrastructure provider and an ableer, they're not trying to be an ISV, they're not trying to be the model vendor themselves, they're trying to enable others and they make the most money on it. - Isn't this kind of the same strategy as Apple with Siri, they weren't spending a ton of money with R&D building out Apple AI or whatever they're calling it? - Intelligence. - Is Apple intelligence, thank you? It's that they're using separate third party tools, in this case, I think they chose Gemini to be able to build it into their products. - Yeah, I think it's similar, right? It's a partner motion. They're partnering instead of, the virus is built, they've decided to buy. - Yeah. - Is that a building? - Do you guys really think that we're seeing the productivity gains at the customer level from AI? - Who do you mean the customer level? - Well, customers using AI. So we could look at our business and the fact that, if you look at our work type, the work output of innovative solutions as a system integrator for customers, the majority of our work type is code, it's coding. So one would argue, in a business like ours, we would see a significant uptick in productivity and we'd measure this from a variety of lenses and maybe conservatively, we could all probably back of 15 to 20% of productivity gain. - I think it easily measure that amount and then there's the immeasurable. - Okay, so that's us. - Yeah. - What I was saying, or asking and throwing on the table for us maybe just to spend a second on would be just kind of our own take on where we see customers seeing a productivity gain. 'Cause I don't think that that has materialized yet. And I also believe that a lot of the agentic workflow that's being built, either by customers or for customers, that the returns on a lot of that automation is in the repeating of the tasks that it will be doing on a monthly quarterly, let's just say annual basis. And that's where the real return kicks in and so that there's this lagging indicator where a customer today am I being like, "Ah, we're not really seeing the efficiency gains or the productivity gains." But the real story is going into the end of this year and the next year as some of this automation really starts to allow itself to propagate and build on itself, that's where the real productivity gains will start to be shown. So like if we fast forward to AI unplugged. - Yeah, unplugged. - Episode 32. That might not be as big of a conversation because we will be seeing it all over the place. I'm throwing that out there as maybe a. - Well, it's just not your observation and just of what Innovative is doing. Just yesterday, AWS, the official AWS account, tweeted saying, "There's more AI generated code. "It does not make your team faster. "It might actually slow you down." So that's obviously the opposite of productivity gains. They're seeing that it's moving them in the other direction and that's just kind of a wild quote for them to have based upon their business and what theirself. - It's interesting point. So Justin, remember back in the before the cloud was a ubiquitous thing, people were talking about the productivity gains of the cloud or the cost savings. - Or the cost savings. - Say, remember that. - Everybody wanted to talk about, where's the cost savings of the cloud? - But the reality was the biggest value people got from the cloud was doing business differently because of the cloud. You didn't have to just wait to spin up a server. You could write an application differently and it wasn't just about displacing Dell. It was about changing your software development process and it took time for that to sink in. So I'm with you on that. But like you just said, what if a year from now to years around we're doing this podcast and we're looking back and saying, I can't believe we questioned the ROI. We're questioning the productivity gains. It's like today looking back at our days before email and saying, what was it like to send in to office? Who knows? So yesterday, what was it like before the internet? I don't know, there was no baseline anymore. I think we're gonna, we're already seeing the natural evolution of people being more productive in ways you might measure and in ways you can't. But we know right now, as late as an hour ago, we're sitting in a meeting and we're talking about this fact that AI is getting very expensive for our customers and it's always been expensive for us because I think we've consumed it in many ways that our customers don't. And we've been a little bit, I think, ahead of the curve on how do you manage and optimize, cost-optimize that AI investment where I don't think a lot of customers are thinking right now, but they're starting to question. And I think that that is probably the most, like if you had to like put a pin in it today, June 10th, 2026, that's the conversation. I think a lot of customers wanna have, that even 30 days ago, 60 days ago, certainly not 90 days ago, customers were not having that conversation with us. - I feel like it's, you look at all the companies that put up leaderboards, how many tokens you consumed and it turned overnight, almost, and took a joke. - Is this the theory of tokenomics? - Tokenomics, yes. At the time, that was called token maxing. Where they're trying to, like look maxing, but they're trying to max out their token usage to win a leaderboard and reality is a lot of people just putting agents on loops. They were doing nothing useful just to consume it. - The old days of mouse movement. - Yeah, it's just like that, right? And the practical outcome was they were just spending burning cash, right? Or making things that were unused. - So the game of fight in the wrong way, but there might still be something there, is the market shifting toward being critical of AI token spend, we'll call it consumption spend. And are there other models that are gonna make a result for that? - Well, that became a joke, and it would cause a lot of reflection, I think in the last month of people saying, like are we doing this for the right reasons? And we're spending my cash, and I could have used that heck out. I didn't actually get the claw chat about things, didn't help me do something faster if I actually anticipated, it made some other things faster, but the key goal I wanted to achieve didn't happen. - If everybody just did the same amount of work in a little less time and they got a better lifestyle, right? - Yeah, it doesn't actually translate to business profit. - Correct. - So there has to be something else that does. >> Right. >> And then people are looking for it. >> Andrew, this reminds me of the original Cloud days. Again, I think there's a parallel here to what happened in the Cloud. We were talking about Cloud being scary for security. >> Right. >> Just like AI might be scary for security. People were talking about, oh my God, it's got access to my company record. It's going to train itself in my data. Maybe not, maybe we've solved for that. But cost and compliance might be the next frontiers. And this whole AI management thing is headwinds right now to AI adoption. >> Yeah, it's paradoxical. I mean, we hear even from our friends at AWS that they're internally tracking how many tokens people use and their overall general AI usage and the theory is the more the better that those that are power users are going to be most productive and innovative last week. >> But yes, now the script has flipped where it's like, hey, maybe this is costly. It is unproductive. And this brings up the idea of the scary term of AI bubble. We have all these mega companies like OpenAI and Thropic moving into IPO, SpaceX with GROC as part of it, GROC AI or XAI that are all coming really within months of each other trying to time this moment. And I think there's this fear like in the investing world of is this the top, right? Are we seeing a bubble that mirrors the dot com side of things so history is kind of seeming to look at the Jeff when we say dot com side of it. >> You know when Google started, it's very, I'm sure during those dark times. >> Yes, exactly. Not only were great companies started, but it means that the bubble was a stock market valuation issue. It wasn't a business productivity thing. The dot com revolution continued to propel businesses forward with internet adoption well after the pop. In fact, it launched some of the world's best companies. >> Amazon. >> And wouldn't the same thing happen here? >> Yeah. >> Yes, the stock market might reset to a better even evaluation multiple. I don't think that means that we're in the store for like nobody's gonna get value out of AI in the future. That's the most ridiculous thing ever. >> Well, and AI, my opinion, just going on record, this is my opinion, AI has propelled now space travel, rocket development, the next generation of internet, which is coming, which will be a hundred times faster than what it is today, 'cause you'll need it for AI processing. And AI data centers, which will not be physically located on planet earth. That's the moon. >> AI in space. >> Well, there's somebody just announced, I think it's the, >> All because of AI though. >> Yeah. >> It wasn't because of our ambition to go to the outer galaxies. It was because we realized that there's certainly more cost effective ways to process information, to send information, and to manufacture and produce the processing power necessary for the next waves of AI development. To me, that's the bigger headline. No one is really talking about, I think you're getting a little bit of it with the looming IPO of SpaceX, which I think is the first of more to come. Talk about exciting times, man. These are things our grandparents, great grandparents couldn't have dreamed of. I just, I am, every day, I'm just, I'm absolutely floored by more and more that's coming. That just isn't widely talked about, but when it hits and when it makes a splash, like it's just so remarkable. >> Yeah. >> Yeah, exciting. >> I got totally, I give you an example. We have it all employees in this week for our in a week, right? One of them drove from Ohio. Happened to be in a Tesla, didn't touch the steering wheel from the time he left to the time he got into our parking lot. That's a little scary. >> That's a, that was not a thing three years ago. >> Right. >> So that's all computing power necessary. It's already here for that. >> Right. >> Can you imagine putting AI in the next robotic dog or in the next home assistant? The ability for us to transform literally our consumer experience day today, I think is the thing that people are going to experience differently. >> Do you think? >> Or the chat box. >> Going back to the ROI piece for a minute, do you think people just measuring the wrong thing? >> Yeah. >> Like, yeah, I understand. >> I understand. >> Like, like, oh, I'm not getting value, well, I'm making it up. They're, they worked 40 hours before. They're going to work 40 hours later. The perception is that you still work the full amount and you got work done. So as a matter of how do you value the acceleration or the completion of those deliverables? >> It's a great point. >> Right? >> What about the intangibles? So yes, there's the gas in the room sort of expands to fill the size of the room example. This is what you're saying. There might be a productivity cap based on what humans do otherwise. >> Or do businesses know how to measure it? >> Or they have no measure board. But there's one other thought here. What about what we did, we just did this for a consumer electronics company. We helped them build AI features into their consumer product. And now it can help sell more consumer products and SaaS subscriptions to that. AI enabled that transformation. >> And they didn't think of AI, I think prior to our conversation, they didn't think of AI as anything more than just a chat bot. And let's face it, isn't that the majority of the initial conversations we're having with the customer? They're like, hey, we want AI in our business. We'll ask them what they mean. And they're really talking about a glorified chat, assistant or some chat functionality. >> It's the gateway drug. >> Dude, but that's so three years ago. >> Right. >> Like, but that's where the mainstream still is thinking about AI. They're thinking, when Elon Musk talks about building an AI data center on the moon and we need all this processing power, they're like, well, for what? For like higher quality graphics generation and Grob or-- >> Or memes. >> Exactly. >> For that chat is not what we're talking about. >> Right. >> Maybe what's happening. We see LLMs consolidating to a few mega players that can afford to build a new model, a foundation model. Sure, maybe chop-ups are going to do the same thing. Maybe we're starting, because we just saw Amazon last month release their version of a desktop product that we should'll call Quick. And we first got that question, how is innovative as Darcy product different than Quick? Because we happen to have a chat interface? >> Yeah. >> No. I think the reality is you're going to have a few chat interfaces. Maybe it's cloud or maybe it's Quick or-- >> Jammini, chat CBT. >> But maybe tools like Darcy can power all of them. >> Sure, maybe that layer of enabling the-- whatever the front end happens to be is where the real value comes from. >> I just don't know. I think every company values people differently. Right? And if you don't know how you value your own staff, you probably don't know how to measure ROI impact of giving them an AI tool. Right? It's like a lot of what we do when we consult is we talk about the business process we're trying to automate. It doesn't matter who it is. Right? And then backing into why it's useful to automate. I think a lot of companies just adopt for a sake of adopting. They don't actually put the critical thinking into the term it's useful. >> Right. >> Which if you mentioned a consolidation with a lot of these AI providers with the foundation models, there's big announcement of a long rumored release from Anthropic of the Mythos class. >> Yeah. >> Travel, what can you tell us about that? It sounds cool, but maybe not. >> So Mythos is a bit this back up. So Anthropic announced what was called Project Last Wing, which was a small set of security companies and banks, like the Finnserve sector. And they said, we got a special model called Mythos. It can do some wild things with security, like vulnerability scanning and being able to access information, you probably wouldn't have anticipated it and finding weird things. And they gave it to these very small providers to say strengthen your security before this model goes live, because we don't want a bad actor to use this model to misbehave on your network. That's Project Last Wing. Well, yesterday they announced Anthropic Fable. Fable is a Mythos class model, basically it's Mythos, and then they neutered pieces of it. So that it's less negative and antagonistic and so forth, very specific use cases like security and gene sequencing and so forth. So that came out yesterday, super, it's a super great model. The honest of you is actually, it is really, really good, especially for software engineering. It's special for software, but it is ungodly expensive. It is twice the cost of Opus, which was already at the time the most expensive model in the market. It's like $50 per million tokens, which is nothing in many cases, right? So there's lots of reports of people trying it and blowing their AI budget for a month in five queries. You know, I was so happy with Opus, like 4.6. Yes. It was fine. I like 4.8 too. I don't know if 4.7, but yeah. But you were, I mean, like it was, it was okay. Is it going to be the case trap that the same quality of this new model is going to be available to us at a more reasonable price and just wait three months? Yeah, I think I'm a big fan of open source. I think open source models are like the cost-effective way to get that kind of experience. I'm sorry, even within their own pricing model, I think just going to drop the price in three months. You think it's always going to be that price? I think it's like a used iPhone. I can get a used iPhone that's only a few months old. There are 100% extending the pricing on almost all these models. Guarantee you. So it's never going down? I'll bring the receipts. I don't think it's going down. Unlike, you know, compute with AWS where they're always reducing the costs. That's what I'm saying. It's going to maintain and/or get more expensive. Yes, here's the trick, right? So look at Sonnet. It's brought an easy example because it's been out for so long. So Sonnet 4.0 is the same price as Sonnet 4.5, 4.6, and so forth. It's always been the same price. right and you say oh, they're Optimizing or something. No, what they've done is they say Sonnet 4.6 Produces more tokens even if it's the same price you produce more of it, which means your pricing goes up Right, they always do the same thing of opus and hiku and so forth It's always maintain the price but the price really is really elasticity if that's the price is increasing if that if what you're saying is that It's always going to be ish always $50 per million tokens for this new model. Yes, then people have to be really efficient with those tokens Yeah, I mean we disabled it for our cut for our staff right I turned it off for season came out So we wouldn't turn it on and thus we had controls around this it said oh my god we're spending we're wasting it on generating Poms. Yes, let's not allow employees to use poems like let's instead have it right software for it Yeah, I don't think it's a reactive Solve I think you have to solve it on front because they're like they're not gonna be like oh sorry You spent $10,000 and you're still gonna pay $10,000 for your mistake right so you restricted it because of cost controls Not necessarily because it's too powerful or security concerned. No, it's too expensive I wouldn't want our staff or uneducated as to how much it costs Right because you don't have a budget the company they don't have a budget right the company does But also when you ask an agent to go do something it doesn't tell you how many tokens is in the cost to go do it right It just does it yes, so you have no idea how much is gonna cost you until it's done. Yep And so for staff who don't know how much it costs don't know how much it's gonna take to do the work I don't want to give it to them until we have a good handle as to what it actually can do Well, I thought it was interesting timing because inthropic with cloud they they drop this You know super powerful model that is you know supposed to be a game changer at the same time made a blog post Talking about how maybe the industry should slow or temporarily pause this frontier AI development Hey, maybe right after we did ours. Yeah, yeah, so I don't know if the timing of that was suspect But we've never heard that from these foundation models before like slowing down innovation Oh, same on same out one asked for it like a year ago Is it because that they're afraid of the competition between Dario and Sam and Elon and I think they're afraid of Chinese models I don't think they're afraid of Within the government or within our national our national politics of do you think deep seek is actually driving enough Competition in the market that's gonna percent so that will be the price control that keeps prices from going through the roof right I think you're gonna see a lot of open source models Get near not exactly the same but near the intelligence good enough for a 60 or 70% cost delta And they're gonna try to race it to the bottom and when that is just like this is a classic move for You know Chinese models or Chinese like sort of go to market in general They drop the price dramatically until they run the competition out and then they race prices It's not interesting though that so that the cost of the model development is all up front It's like CapEx you're building a factory and then you have to recover it over time But once the factory is paid for you can charge whatever you want correct. There's really no incremental cost the GPU is gonna run it A little bit, but not as much as compiling it in the first place. Yeah, yes But it's still costs you something run it. Yeah, yeah, yeah, yeah, 100% yeah, I get you I've heard people make the analogy of the rising cost of AI Like in it to the early days of Uber. I don't know if your member would Uber came out But I could get across Chicago for like five ten bucks and yes, of course they put you know all the cabs that were plentiful Out of business and we thought it was revolutionary and now that same ride 50 bucks and there's no more cabs and so I think people are seeing that same thing of hey, they've been Operating at a loss buying the business really changing the market and the way we work having these You know massive employment disruptions and now that the markets kind of cornered between three or four big firms Prices are going up. I had a customer ask me today They asked how can your team use so many tokens For the cost because they had to know how much we spend on our our tokens per month compared to like their business Like 4x the cost right and my answer was we host our own models There's no middleman. We don't pay a margin tax to in-thropic or Open AI right and we optimize what models people get for the different uses use cases they get and our costs are Dribatically cheaper to scale out the business as a result of that I don't think a lot of companies are there yet I think in the next six months because you have a ton of people be like oh my god What did I just buy and how do I optimize it and if they want to optimize you're saying they could go run an open source model On an AWS cloud environment yeah, are less expensive than they could spend total tokens from cloud totally Well, you may be happy to know that when I am at work using Darcy I do feel cost-conscious for how I use it Sometimes you don't need to be doing things that are maybe I could have done otherwise. I'm like man What is this costing us? But I know it is a big initiative for us to use AI and what we do not only at work, but also, you know personally We share every week and our all staff a segment called AI in the wild where people talk about how they use it and You mentioned Jeff we are here in Rochester for in a week where we gather our entire company for intentional time for collaboration and recognition and team building and giving back and it's awesome and one of our co-workers built a tool For folks coming from out of town just talking about all that you can do in a nearby neighborhood of Fairport to Rochester And that was an awesome tool built on Darcy and I used it extensively for my wife who's in town to plan what we should do And I'm like I wonder what this costs Probably small, but at least it was on my mind. Yeah, probably a dollar How have you guys been using it lately and have you thought of the cost consciousness? Personally, my son has so I have a skill in Darcy for so my son likes some TV show and It's I'm not gonna spend TV show because it's too complicated to explain But he likes his TV show and it's got two characters in it and I made a skill that replicates the TV show who the characters are with their shapes and so forth and so right And when I go home on the weekends We do story books So I load the school a skill Ask what he did during his school week and he tells me and then I reenact His school week from the persona of the school. You just don't want to share because it's ip theft But I love this idea. Yeah No, it's a it's a It's very hard to explain story uh the school that the show the show with itself But he loves it he comes you comes up to me every single day. He goes oh we make a story today. That's really cool Right as I go see yeah Well a great conversation men changing topics is always fast moving We're great to chat and can we for next time all right. Thanks guys. Talk soon [Music]