Go back

Uber Just Exposed AI’s Biggest Cost Problem

25m 23s

Uber Just Exposed AI’s Biggest Cost Problem

The discussion centers on the rapid adoption of AI coding tools at companies like Uber, where usage exploded so quickly that the entire annual AI budget was exhausted within months. This highlights a major challenge: traditional annual budgeting is ill-suited for AI's exponential cost curves. However, the productivity gains are undeniable—75% of AI-generated code reviews were deemed helpful. The key is not to pull back but to optimize token costs, for example by using open models for routine tasks and frontier models only for complex work. Companies that master this optimization will have a structural advantage. Despite fears of mass unemployment, historical data shows that while some jobs are displaced (e.g., bookkeepers), many more are created (e.g., AI trainers, engineers). The real focus is on increasing output and revenue, not cutting headcount. A proposed "Return on Tokens" metric could help companies track the value of their AI investments and avoid wasteful spending. Overall, the message is that AI is driving companies to move faster, hire more, and focus on quality output rather than simply cutting costs.

Transcription

5171 Words, 27234 Characters

English
So Uber gave 5,000 engineers, so Uber gave 5,000 engineers access to cloud code in December. By February usage and nearly doubled. By April the CTO told the company they burned through the entire annual AI budget. So the adoption curve tells you everything about what happened. In December 2020 for 32% of Uber's engineers were using cloud code. By February 2026 that number was 63%. That seems a little slow to me. But anyway, that's not a gradual rollout. That's a product. So useful that engineers pulled the internet workflow faster than finance could could model the spend. Okay, that's a lot of employees though. Uber has about 34,000 employees and you're going to just roughly 15% of that. Okay, so all this to say, Neil, is this CTO if I go back over here. So look, AI related costs at Uber up 6X since 2024. This is what we're talking about. I'm working this in our organization, yours too. CTO Praveen Nepali Nagas quote was, I'm back to the drawing board. That's the CTO of $144 billion a company admitting that the tools work so well that his team can't afford to keep using them at this rate. So here's the thing. So he has to go back to his CFO and ask for a larger budget now. But to Neil's point, if this is working for you so well, you should be spending more. And here's the other thing. You might be thinking, oh, I can put this all on open source tokens. But if you're actively building things in shipping code, you can't put on an open source tokens. Because you might need to rework it a couple of times and the time you spent there's not worth it. You like chances are if you're shipping a lot of this stuff, a lot of it's going to have to be on frontier models. So and there's an important thing here. They rolled us out in December. And by February, they had good adoption by April, they went through all their tokens. I bet you there's a lot of terrible usage of cloud code within this organization. I don't mean in bad ways. Actually, in almost all organizations where they're doing a lot of things inefficiently and they're not optimizing for cost savings. That'll start coming out soon where companies start thinking about it. And then his budget from April will last the whole year and he won't need to go back to the CFO. We'll see. I mean, so it says the CFO problem here is now the bottleneck of AI adoption at enterprise level. The technology works productivity gains are real. Uber's own data says 75% of code review comments are marked helpful by engineers. Okay, there's your thing. The 75% are helpful. The constraint is that the traditional annual budgeting was designed for tools with predictable per seat costs and AI coding agents have usage curves that look like cloud compute bills from 2015. Exponential until someone notices every enterprise CTO is about to have the same meeting. The tools are too good to pull back. The costs are too unpredictable to ignore. And the companies that figure out token costs optimization first will have a structural advantage over every competitor still running annual budget cycles. I'll take you something Neil. We've been playing around with token optimization this week just on my own stuff. I'm trying all the models are Gemma 4, Quinn, which is from Ali Baba. I'm trying all these other ones. It's like they're decent, but there's still nothing close to what you get with cloud. Well, on our end, we're, so let's look at from a business standpoint, we tend to create a lot of tools for marking related purposes. All right, for ourselves, our clients, or, you know, just putting out their tools, get you more brand mentions, build more links. So we use cloud quite a bit. What we found is our outputs are similar. And I don't know exactly what my team is using, but I saw a report on this. We're down in 21% and cost, because we've been optimizing for efficiencies. So 21% savings over the last 30 days is huge. And I bet you as time goes forward, no, only will the models get cheaper, but people will come up with solutions for optimization, because this is a big problem. And I bet you our bill a year from now will be maybe like 25% of what it is today on a monthly basis. So I'll give you guys some practical things you can do. And I'll give you my thoughts on this. So you guys can all use open router. So open router will help you route to the right models and no save, it'll help you save a lot of money. They take 5% of the, the, the, basically the spend, because you're, you're routing everything through them, right? That's one thing you can do. You can buy a local infrastructure. So the reason why I bought these DJ X marks is to see kind of how local inference will work. And then we're going to consider buying more and more. The thing is, Neil, I think we're going to token optimize. I think I mentioned it last time, maybe 75 80% goes through it, maybe maybe for, for open models and then 15, 20% on frontier, which is like, when I say frontier, I mean, you know, you're using cloud opus, like the latest version, right? Or using mythos when it comes out. I think, Neil, the costs are going to come down, but what's going to happen is usage is going to skyrocket even more. And so what ends up happening is costs still keep going up because the usage is still unpredictable. And because I see my team using it now, the ones that are getting peeled by it, they're like, we need more, we need more because they seem how I'm using it. And they start using it that way. They're like, oh my god, right? And I'm like, oh my god, if they even start using it, even like 25% in the way I use it right now, my costs still go up exponentially. Yeah, but I don't look at costs going up exponentially because at the same time, even though the models are becoming cheaper and cheaper, you're going to have it to a certain point where some of the older models are good enough for a lot of tasks and you won't be using as many frontier models. So that'll, I think, will save a lot. B, and this is the big one. No business really cares about the cost if the work is actually producing a direct ROI. So if they're able to, if they grow their cost by 10X, but they're rev it for that department, but their profitability and their revenue grows in total, a business is happy. Look, if you're publicly traded and you spent too much on AI credit, but you can show, hey, we're predicting 20% growth in revenue and we actually hit 28% and I know our profit for the quarter was supposed to be a billion dollars, but we have the same amount of people. Yes, we use AI more and do the revenue increase. Our profitability was actually 1.2 billion dollars. The market will reward you and your stock goes up. No one really cares if you're spending more if you can produce more. I, I just think this and it is actually, I have a thing that ties in with this. I just think we're all going to be producing a lot more just looking at how I work. You might say you don't work that hard. I still think you work really hard right now, but I think the way we, I think we're just going to end up doing more. So let me, let me share this with you real quick because I've kind of said this before, Neil on this podcast, I think there's going to be net new, there's going to be more entrepreneurs in the world because you can do so much now. So this guy from Deal, remember, deal, the, the one that did the spy with Rippling. So yeah, the company that you can use to, I think it's like pay employees internationally or something. Yeah. Yeah. Yeah. I think they're a good company, but they did that whole spy thing. So he said P Mark, so P Mark is Mark Andreessen. Oh, look, hey, look, I'm cool. These guys, I'm homies. So look at this. So when a company becomes more productive, this is from Alex Boaz. I don't know how to pronounce it. So when a company becomes more productive, it doesn't sit still. It goes after more customers enters new markets, builds new products productivity gives you leverage and leverage makes you want to do more. Not less exactly what I just said. So no CEO in history has looked at a more productive team and said, great, let's streak. They hire more. That's what we're seeing in our data. So we also tend to forget that new tech creates new jobs and industries. A job title that didn't exist. Now the man 70 K people, 283% year on year for jobs that directly experience the productivity gains, the mansion surge as CEOs watch how productive these people are and how much they can be doing. So if you see that's over here, Neil, you want to read it? Uh, so AI trainers outpacing every job in 2025 at 283% growth. I can't, I can read 20. I'll call it the rest, Neil. So, um, thank you for reading that part. So 2025 starts. You have 18,000 AI trainers employed globally. Now that 70,000, wait, I never even thought about this. Like you're good and need more AI trainers. Obviously, right? AI engineers and PM roles are actually up 400%. See a 15,000 open AI engineering jobs, not the company just open AI jobs on an 1135 open AI product manager job. So it's going up. Right. So I think Neil, we're going to see more entrepreneurs and then I think people are going to people are going to realize that creation gives them a sense of meaning and purpose and they're going to want to do more of it. And I think we're just going to want more, more, more, Neil. Um, that being said, I don't disagree with you that for more basic tasks like doing one plus one, maybe it's, you know, that's going to be free or close to free. But I think we're always like maybe 10 to 20% of time, you're still always trying to build new stuff. And that still eats up a lot of budget because you still want the smartest models. Yeah. So I'm in Brazil right now as we're recording this. And yesterday, I was at a really large publicly traded companies office in Brazil. They're global. To give you perspective, they own three floors of office space. They're not the biggest floors. They're normal size like, you know, just good size buildings, but they have three floors or that's what they take up. I don't know how much they spend on their new office. If I had a guess 50ish million dollars, okay, which is really, that's a lot of money for like decorations and technology and all that kind of stuff, right? And I'm in Brazil. So if you take the currency exchange here, that's roughly like 250 million USD equivalent. That's a lot of money because remember, people get paid less in Brazil as well. So for a company to spend around 250 million in their currency and employees get paid, you know, proportionally, like, US employee getting 100 grand, they get 100 Ray, so that would be like 20,000 US dollars. And I was talking to the whole marketing team and we're talking about strategy for Brazil as well as global strategy. And the one thing that's consistent with them and almost every other company I talked to, no one's thinking about shrinking or laying off employees in marketing or anywhere else. Everyone's talking about, hey, everyone's using AI, it's causing companies to move faster. We need to move faster. How do we adopt these tools in the correct way to be more effective? How do we focus on the right KPI? So that way we actually see revenue growth and not just increased costs in AI and no change in the top line. But everyone has this mentality that we've been seeing recently because if you go back a year, Eric. I think you'll agree with this. Everyone's like, oh my God, doom and gloom. Everyone's going to start getting laid off. Tens of jobs are going to be displaced at least in the corporate world. We're not talking about flipping burgers or stuff like that. Even in the corporate world, people are afraid of that. But we're seeing the opposite. We're not seeing sales reps get canceled, marketing employees get canceled. We're seeing them stay roughly the same. Sometimes even increase. But everyone's like, how do you get 2x the output and productive output? Productive output doesn't mean, oh, they did 2x more work. It's more so they did 2x more stuff that actually moves the bottom line or the top line. And that's what we're seeing. And if you don't keep up with the competitors, you're going to lose. So you can't actually cut. You just got to figure out how to move faster with what you have. You're not using your microphone. I'm not. That's right. Okay, Neil's going to get cut. Okay. So I want to know what happened. What? So I had on my last trip, come back from crime. I keep it in my bag. So I have protein powder in my bag. Okay. Like I travel with protein powder. Yeah. Show the brand. That's Legion. Right? Yes. We should we should give a shout out to Legion. There you go. Great. Okay. So I didn't know that they had to check it. Like they check to see if there's powder. I had my mic on top. The rep pulled it out and then boom. It hit the ground and it got messed up. So I ordered a new one from Amazon. But I was pissed. But what can I do? You know, you can't be like, hey, TSA rep. You owe me. You know what's funny? Sometimes I get pulled aside from TSA. They take the mic out. I get stopped from the mic because they think it's a grenade. Yeah. I don't get the grenade. They would tell them because you know when they say take all electronic devices out, I would say maybe like 30 40% of the time, they stopped my bag. And they're like, you didn't take out your technology. And I think it's a microphone. You just said laptops. But electric toothbrush. They never make me take out. But for some reason, the mic it gets flat. Yeah. The mic gets flat quite often. Anyway, real quick. If you want to acquire customers faster and more efficiently this year with the latest strategies and tactics, then check out single grain.com. That is my ad agency again. WWW dot single grain.com. Check it out. And it seems like a fit. We'll get in touch and help you with a free marketing plan. So Neil, I think I have one point to bring up. I'll come back to me. So I want to bring up these charts over here. So these charts are from co-2. We've kind of covered them before. But I think it's worth bringing it up because the key thing here is will AI lead to mass unemployment. Oh, I remember I was what I was going to say. So the way I think we might measure this is going to be an experience for us. Neil. So the way we're going to measure AI adoption is we have clawed teams right now. We can see usage and cloud code. We can see usage in cloud co work. So we want to see one. We want to see usage. But we also want to see the pull requests coming from GitHub. Now those two things can be gained. If you're just shipping a bunch of pull requests and you're just spending a lot, you could it could be stupid spend right to Neil's point. So the thing that you saw earlier from Uber, where they're saying 75% of engineers said the code was helpful code. We might have something like that word, the engineers, or maybe the people, the managers are checking and saying, Hey, like you did all this you're going to engage, but what have you actually built. Can you screen share with us, right? So my point of saying this is that you can have these metrics that are maybe easy to record, but they might be easy to game. You need to have a pairing metric next to it to make sure that what you're doing is actually quality and it's not just a bunch of slop, which is what my homepage got called even though it converts. So yeah anyway, Eric's homepage was not sloped. That's just what again what someone else thinks. I'm neutral. His homepage was crap. I would tell you. Yes, it's a perfect from a design perspective. No, but get it out there. Find out if the model works. If the marketing message, you know, converts and then go and adjust it and fine tune it later on once you figured out how to really make the funnel. You know what I've learned from this, this Neil right now is is one that like you and I we grew up in the internet era, but we weren't exactly working at the time just yet. And what I've learned is when the demand is there, it doesn't matter. Like you're going to get pulled. The demand is going to pull you and like who cares, right? So, um, Okay, check this out. So this is a done from co-to. So literally they had a post about a month ago saying, well, AI lead to mass unemployment. Okay. And so I just want to call out here. The ROI that we want to look at instead of return on investment, we want to look at return on displacement, Rob, ROP. Okay. So when you look at agriculture in 1912 million agriculture workers, 41% of workforce, you go all the way to 1970. It's 2% of workforce, 3.5 million workers. But net net what happened was agriculture lost 8 million jobs from 1910 to 1970 and then new industries gained 46 million jobs. Okay. So 46 divided by by by by 12 over there. 5x right. So there you have it. I see the same thing with engineering. A lot of like if you go back to the Uber example, 46 divided by 8. Yeah. Go ahead. Still, it's amazing, right? But if you look at like the Uber example on how they're talking about 75% efficiencies, the or 75% approval, like they found it being helpful. The first thing that comes to my mind is outside of cost. When you're using it, yes, you're releasing more, but is that releasing more improving, uh, LTV customer, you know, repeat visits or repeat usage. You know conversion rates, new potential onboarding or revenue, like all those kind of things. And what you'll, what I believe is you'll end up seeing is you'll see some displacement because people like you're using all this a stuff to we actually need all these people. And B, I think you'll also see in addition to cost going down people being like you're using air for all this stuff. And this is great. And you did. You know, 50 things this month versus the normal 10, but other 50 things you did only six of them actually help driver return. Well normally when you do 10 four of them driver returns you did 50% more things on that. And but you did five time five X amount of output. So then they start fine tuning what is things are being used for. And you're going to start seeing not as much usage in certain areas more usage and others employees getting displaced because they realize they don't need them all. And then some of those employees go on to other sectors, other fields, other jobs. And it's the same thing and marketing do the amount of customers that are asking us to build them agents right now is ridiculous. And it just goes to show people are like, oh, you're going to get real people. Well, we're at the same time. We're getting new requests that are requiring us to hire people or reshift. We're replacing different people. So here's what I'll call it. I actually went. So guys, here's a free idea for you guys. So when what Neil and I just said, this is a problem. We're all going to be spending more on tokens. We would like to track our return on tokens, then our our rots return on tokens, then, okay. So if you can figure out and I just went I just went to my age and I asked, hey, can you build something that tracks return on tokens, then because like that is something we'd all spend on like so we can optimize our costs, right. That's a smart idea. Someone take that idea. Neil will invest. I will co invest with Neil. I'm speaking for Neil here. So anyway, I know it's really a model. You look at this. Okay. By the way, Neil, remember, I think it was 98% of people used to be farmers. And then you go all the way to 2% but this displacement took a long time, 1900 and 1970 or so. Okay. I'm not even done here. There's another one over here. Spreadsheets, for example. Okay. The return on displacement with spreadsheets. So people are like, oh my God. So bookkeepers like you see this in the blue over here. It's going up and up. So bookkeepers and then visit count comes out Lotus 123. Remember that one, Excel. And then it's the bookkeepers start going down. But what happens. Accountants and auditors starts going up and then financial analyst starts going up. So 400K bookkeepers disappeared at the advent of the spreadsheet era, but 1.3 million accountants and financial analyst were at us. So we're going to be doing different things. And again, we said this before, don't hire humans to do robot things and more human think tasks that we're used to are being taken over our robots. That's okay. Because nobody wanted to do those things in the beginning anyway. Yeah. I have one more. Neil go for it. Share the next one. Here's one more. So, no, by the way, look, New York Times in 1973 predicted 8 teams would cut bank teller jobs by 75%. Instead, teller employment grew by 81% from 1970 and 1988. That's because way more banks open. Here's the other thing. Radiologists. Oh my God. 10 years ago. I think one of these these AI like, you know, speakers was like, you know, radiologists. That job's going to go away. What happened? There's more demand than ever for radiologists. So I think, yes, I think short term, there's going to be job displacement for the people that don't pick it up. Those are the people that I'm sensationalists that that I'm talking about that like I am genuinely concerned for those people, right? Because there's going to be some short term displacement long term. That's why I keep saying Neil's been saying this to everything's going to be okay. Because you see, you can see that history of rhymes. It doesn't repeat, but it rhymes. Speaking of 8 teams, it was funny. I saw a lot of the original ATM ad campaigns from back in the day, but, you know, if you have in Google, I'm, you'll see them. They're kind of entertaining. Do you even use ATMs yourself? Like I never use them. Only when I go to Vegas when I lose money. And I go to age. You're so high. Yeah. But it's okay. I paid anyway. Because I'm at Vegas. I don't even have an ATM card because I'm worried about fraud. You don't use a ATM. You just use your regular card. What do you mean? So okay, this is when you're being degenerate like me guys. Don't be like me. Okay. So let's say you're gambling, for example, your ATM card has a limit on it. So you don't use a debit card because there's a limit on it, right? And then you can call them to try to break your limit, but it's annoying. So what you do sometimes, if you get desperate, is you just do a cash advance with the card? Are you at the pay even a big or fee? So I have a credit card. I have an amix. You're saying with American Express, I can go to ATM and put it in and get money out. So I know what's here. Let me speak on a debit card. The debit card piece first. I know you can get a cash advance. I think you could do the same thing with the credit card too. But I. if memory serves me right, I haven't done this for a while. This is only doing my degenerate times when I gamble a lot. So. And Eric's talking crap about himself. He's actually a very good gambler and overall has made money. He typically pays poker and usually comes out winning. No, but when I play stupid table games, I lose a lot. And this is in our 20s or so, like you're like you're actually partying and drinking and stuff and being stupid. So like, you know, we don't do any of that anymore. So Eric came visiting me in Vegas one time and we all did dinner with his friends in Aria. And I remember Eric was gambling because we were waiting on some of his friends and we were there early. And one of his friends comes and he's like, well, Eric just lost five grand at a table playing Blackjack, right? And even though it sounds bad and a lot of people were shocked, his poker winnings more than make up for all his table game losses. He's actually very good at poker. Poker by the way, poker is great for poker is great for business. All you learn a lot from it. You know, you know, remember your brother-in-law, Heaton. So I was just talking about how poker's been so good for me and I tweeted it. And he's like, while you're playing poker, I'm playing business. So Heaton tweeted that. I mean, right? I was like, oh, mother fucker, right? So that's part of my language. And so you know what I did? I was like, well, Tramoth doesn't think that. Tramoth thinks it's good for business and then Tramoth responds. Tramoth's like 100%. You know, poker has been great for business. It helped me with everything. It helped me with my return on investment. I was like, that's right. He didn't. That's where you take that. So anyway, that's just a fun story. So, um, uh, wait, to go back on it, um, on the poker thing, it's not even just great for like strategy or whatnot. If you ever want to get into poker, I'm not big on poker. It is amazing networking. Like just from that aspect, the people you meet at like home games, you'll find that you can end up doing business with a lot of people if you are playing in the right games. So I just want to put that out there for anyone who's thinking about gambling. I can speak on that. Neil, because I actually do go to these home games. So one of the home games has like a, there's like a few entrepreneurs. They're actually, everyone's an entrepreneur entrepreneur there. There's some best-selling authors there as well. You would know the names. Um, and so we play like once a month or so and then we were even talking about going to the world series of poker in July to play. But it's good because we end up talking about business. We talk about AI. We talk about, um, and we just, we have a lot of fun and we, we, we, I will say this too, Neil. Um, it's good for networking. That's number one. You're just gonna make sure you're hanging out with the right crew. Don't, don't hang out at those like underground poker games in LA. You probably don't want to do that. Um, but the other thing is you learned how to manage your bankroll. You learn how to manage your emotions a lot better too. And you learned when to bet hard and press your chips versus why not do so. Neil, I got to give Neil a lot of credit. Neil knows when to press hard. When he sees an advantage, he presses really hard. I'm not going to give examples here, but very hard. More like harder than you would ever imagine, right? I'll just leave it at that. Even when it comes to SEO. So all that to say is, um, poker is good for you for, um, you need all these things. You need to learn break from manager. You need to know who's who's the fish at the table. You need to know when to press hard. You need to know what your odds are at every stage, um, of the game. So and then to your point, Neil, it's great for networking too. Yeah. When you said you play with the best selling author, is this best selling author a Stanford graduate? Uh, no. Okay. So there's not one thinking not the one that post a lot of political stuff on, uh, I don't even know who you're talking about. You do. Oh, you can write it in chat. You can write it in chat. Oh, yeah. Where's the chat button? Okay. So while Neil sends me, you can. Okay. Whatever. Oh, yeah. No, I can. All right. Check this out. Yeah. It's not that person, but that's why I assume when he said, oh, no, no, no, no. Absolutely not. No, no, no, no, definitely no. So anyway, that's a good place to end it. We will see you all tomorrow. Don't forget to rate, we will subscribe.

Podcast Summary

Key Points:

  1. Uber gave 5,000 engineers access to AI coding tools in December; by February usage nearly doubled, and by April the CTO admitted they burned through the entire annual AI budget.
  2. AI-related costs at Uber increased 6x since 2024, forcing the CTO to go back to the CFO for a larger budget.
  3. The main bottleneck for enterprise AI adoption is traditional annual budgeting, which cannot handle unpredictable, exponential usage curves like cloud compute bills.
  4. Token cost optimization (e.g., routing through OpenRouter, using local inference, mixing frontier and open models) can save significant money—one company reported 21% cost reduction in 30 days.
  5. Despite fears of job displacement, companies are not shrinking; they are hiring more AI trainers (283% growth) and AI engineers (400% growth), and focusing on increasing output and revenue.
  6. Historical examples (agriculture, spreadsheets) show that while some jobs are displaced, new industries create many more jobs overall.
  7. A proposed new metric, "Return on Tokens" (ROT), would help companies track the value of AI spend and optimize costs.

Summary:

The discussion centers on the rapid adoption of AI coding tools at companies like Uber, where usage exploded so quickly that the entire annual AI budget was exhausted within months. This highlights a major challenge: traditional annual budgeting is ill-suited for AI's exponential cost curves. However, the productivity gains are undeniable—75% of AI-generated code reviews were deemed helpful.

The key is not to pull back but to optimize token costs, for example by using open models for routine tasks and frontier models only for complex work. Companies that master this optimization will have a structural advantage. , AI trainers, engineers).

The real focus is on increasing output and revenue, not cutting headcount. A proposed "Return on Tokens" metric could help companies track the value of their AI investments and avoid wasteful spending. Overall, the message is that AI is driving companies to move faster, hire more, and focus on quality output rather than simply cutting costs.

FAQs

Uber gave 5,000 engineers access to cloud code in December, and by April the CTO reported they burned through the entire annual AI budget due to rapid adoption.

AI-related costs at Uber increased 6X since 2024, leading the CTO to go back to the drawing board.

The main challenge is that traditional annual budgeting is designed for predictable per-seat costs, but AI coding agents have exponential usage curves, making costs unpredictable.

Use OpenRouter to route to the right models, which takes 5% of spend and helps save money.

ROP measures how AI displaces jobs in one sector but creates more jobs in new industries, like agriculture losing 8 million jobs but gaining 46 million in new sectors.

Track return on tokens (ROT) to optimize costs, and pair usage metrics with quality checks like code review helpfulness.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.