Why an AGI Delay Doesn't Mean an AI Bubble

Today on the AI Daily Brief, should we be pushing out our AGI timelines a decade? And if we do, what does it mean for this so-called AI bubble? The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, superintelligent, KPMG, and robots and pencils. To get an ad-free version of the show, go to patreon.com/aisdailybrief, or you can sign up on Apple Podcasts. Now one quick note, you may be hearing that the sound isn't quite as good as it normally is. I am traveling, and the location that I am traveling at has a full studio setup except that it wasn't actually working. And given that we don't really have the benefit of delaying things for days at a time, we're doing our best to use the tools available to us, which means laptop microphones and AI sound improvement. So it is still me doing the presentation. I'm not using an 11 labs version of myself or anything, but the sound quality will be a little bit less. Luckily, just for today, tomorrow we'll be back to normal. Also because of that plus an already kind of long main episode I decided to just do a main today, headlines will be back again tomorrow as well. Appreciate your patience and let's dive in. Today we are talking about the weekend where Silicon Valley seemed to lose faith in AGI or at least change its AGI timeline very dramatically. And what we're going to discuss today is why even if the AGI timeline that has been all a buzz and the new discussion is correct, it does not mean that AI is a bubble. And the bubble talk is certainly setting the context for everything happening right now. You can't throw a stone right now without hitting some article about how the AI infrastructure build out is a bubble. CNN Business, for example, just this weekend wrote why this analyst says the AI bubble is 17 times bigger than the dot com bust. Some are trying to have a nuanced take that yes, it's a bubble, but it's a good bubble. Muhammad Ali Iran calls it a rational bubble. It's very clear that markets are extremely fearful right now. The fear and greed index is a simplified way of understanding what emotion is driving the market at any given time. And we have switched over the last month from greed to deep into the fear category even heading down towards extreme fear. Now part of that obviously is the spate of deals between open AI and AMD and Oracle and Nvidia that have people terrified that it's all one big circular web and if one domino falls all the rest will now I've gotten extensively into arguments for and against the bubble. But the point for this show is that that is setting the tone in the backdrop. Now why it is setting the tone in the backdrop is that we increasingly recognize that AI is propping up the entire economy. Harvard economist Jason Furman shared charts recently with a simple analysis, investment in information processing equipment and software is 4% of GDP. But it was responsible for 92% of GDP growth in the first half of this year. GDP excluding these categories grew at 0.1% annual rate in H1. White House AI David Sacks writes GDP growth is currently an outstanding 3.9% and AI is 40% of that. It's easy for politicians to posture and grandstand by beating up on tech. The real question is whether they want 4% growth rates or 2% growth rates. And being that AI has big implications for the economy. And so in that context, the worrying signs that people are most looking for are about more than anything, a slowdown in demand, which frankly so far we don't have any evidence of. Take for example how we just covered that Google is now processing 1.3 quadrillion tokens per month. And to the extent that there aren't clear signals of a decline in demand, that instead people are looking for declines in the rate of progress that could translate to less demand in the future. In other words, it all comes back to whether there's going to be enough demand to create enough revenue to justify all of this investment. And so while the most worrisome sign would be a decrease in demand, the next most worrisome sign would be the technology not getting better fast enough to keep high rates of demand growth up. So that was the setup heading into this weekend where there were a few different things contributing to malaise in the AI space. The first came in an article from the information about why Microsoft let open AI go their own way on infrastructure. And they wanted to, Microsoft could have doubled down on their relationship with open AI and built out hundreds of billions of dollars worth of infrastructure, essentially playing the role that Oracle is now in. The reporting that came out this weekend wasn't anything particularly brand new, it just sort of spelled out what had been assumed. Basically, Open AI's near-bottomless appetite for compute was more than Microsoft was willing to supply. Sources said that all the way back in the summer of 2024, Microsoft CEO Satya Nadella and Open AI CEO Sam Altman agreed it would be impossible for Microsoft to be Open AI's sole provider. They decided to break the exclusivity of their deal. What was new in the article was the articulation of the amount of discreet tension in the relationship particularly from the Open AI side. According to the information sources, Altman told another AI researcher that, quote, "Microsoft's refusal to build new data centers fast enough was the single biggest roadblock to Open AI successfully developing AGI." Now it should be noted that the sourcing is, quote, "someone who heard and made the remarks," but still the point is that this article clearly articulated a difference between these two partners in what they thought the appropriate amount of tech buildout would be. Now again, nothing is particularly new here even if it's built out, but it does just reconfirm Microsoft's stated view that there is likely some AI overbuilding happening. The next thing that happened was a seemingly small gaffe that represented to some just how far ahead of reality the hype machine had gotten. Over the weekend, in a since-deleted tweet, Open AI's VP of Science Kevin Wheel wrote, "GPD-5 found solutions to 10 previously unsolved Eredos problems and made progress on 11 others. These have all been open for decades." Now the Eredos problems are a set of more than 1,000 unsolved math problems proposed by Paul Eredos and maintained after his death with modest cash prizes attached. 407 have been solved to date. This was followed, however, by Oxford University mathematician Thomas Bloom, who maintains the Eredos problems website, correcting Wheel, writing, "This is a dramatic misrepresentation. GPT-5 found references which solved these problems that I was personally unaware of." He clarified that quote, "The open status only means I personally am unaware of a paper which solves it." Now Bloom continued, "GPD-5 has been a very useful tool in searching the literature and this has been a valuable addition to the website. Its literature searching ability is already useful and impressive enough, no need to describe it as something it's not. Part of why the conversation picked up steam is that Google DeepMind's CEO Demis Hasabis waited into the fray, responding, "This is embarrassing." Three words which got 1.2 million views. Meta's chief AI scientist, Jan LeCun, added his own pun, "hoisted by their own GPT-tards." The folks from OpenAI deleted their post, including Sebastian Bubeck, who wrote, "I deleted the post I didn't mean to mislead anyone, obviously. I thought the phrasing was clear, sorry about that. Only solutions in the literature were found, that's it, and I find this very accelerating because I know how hard it is to search the literature." Now, honestly, the substance of this doesn't matter all that much. What matters is that this dropped into the lap of increasing concern around AI hype exceeding the reality, and so just seem to many as a tailor-made example of what they're concerned with. All of this, however, was just prelude to the discourse surrounding OpenAI co-founder Andre Karpathy's comments on the Duarkash podcast, where he pushed back on the AGI timeline, basically broadly saying that a lot of what AI companies are selling just isn't real. One of the quotes that was most shared and that spoke to the current anxiety around model development was when Karpathy said, "Overall, these models are not there. I feel like the industry is making too big of a jump and is trying to pretend like this is amazing and it's not. It's slobbed. They're coming to terms with it and maybe they're trying to fundraise or something like that. I'm not sure what's going on, but we're at this intermediate stage. These models are amazing, but they still need a lot of work." He basically took issue with the idea of 2025 or 2026 being the "year of agents" because he thinks that we are at the beginning of the decade of agents. He commented that while there are already some very early agents that are extremely impressive and that I use daily, Claude and Codex and so on, I feel there's so much work to be done. My reaction is we'll be working with these things for a decade. The comments were picked up enough that Karpathy felt like he had to add some additional thoughts on Twitter. He wrote, "My comments on AGI timelines look to be the most trending part of the early response." He actually referred back to a tweet of his "from January of this year" where he said "People on my timeline are saying 2025 is the year of agents. Personally, I think 2025 to 2035 is the decade of agents. I feel a huge amount of work across the board to make it actually work. But it should work. Today, operator can find you lunch on DoorDash or check a hotel, sometimes in Mabee. Tomorrow, you'll spin up organizations of operators for long-running tasks of your choice, e.g. running a whole company. You could be a kind of CEO monitoring 10 of them at once, maybe dropping into the trenches sometimes to unblock something, and things will get pretty interesting." The point is I think that Andre was a little surprised at how much people picked up on his comments given that he's been saying them all year. He continued in his more recent tweet, "Basically, my AI timelines are about 5 to 10 times more pessimistic with regard to what you'll find in your neighborhood at San Francisco AI House Party or on your Twitter timeline, but still quite optimistic with regard to a rising tide of AI deniers and skeptics. The apparent conflict is not, in my opinion, we simultaneously, one, saw a huge amount of progress in recent years with LLMs while two, there is still a lot of work remaining. And also, research to get done before we have an entity that you'd prefer to hire over a person for an arbitrary job in the world. I think that overall 10 years should otherwise be a very bullish timeline for AGI. It's only in contrast to present hype that it doesn't feel that way. And it's really hard to overstate if you are not on AI Twitter just how much this dominated the conversation all weekend. You had just about a million tweets like this one from TradeFox CEO PJ who says, "If this Carpathy interview doesn't pop the AI bubble, nothing will, 10 brutal quotes." And then goes through and gives those quotes. And there was enough of this that I think it's really worth digging into. Both yes as representative of the moment that we're in, indexed as we are towards fear, and also just on their own terms. First of all, it's important to understand why people respond so strongly to Carpathy. In short, they feel like he's not selling them something. Anyone from the labs or who makes money off AI has to have an agenda, right? Carpathy on the other hand has already got money, gobs of it. He doesn't have any particular project that he's trying to push on us. And frankly, I think this is a reasonable reason to be excited about what he has to say. It's a reasonable grain of salt to understand everyone's incentives when they're talking about AI. Labs need to get you excited to keep fundraising going. And critics build their media business around being critics and make lots of money off of the sponsorships for their media or their speaking gigs that they get for being that loud, critical voice. I will only say here that one, sometimes incentives aren't as clear cut as they seem. For example, is it in my incentive for AI to be insanely hyped so this pod does well? Or is it actually better for me if it does take a decade because that means steady businesses companies make the transition? It ends up not being all that clear, right? The second thing to note about this is that while it's understandable, it is an extremely cynical take to assume that the only people you can listen to are ones who don't have a financial stake in something. People tend to acquire financial stakes in something because they've taken a disproportionate amount of time to learn about it and have decided based on that learning to go all in. Regardless, for our purposes here today, the point is that people really respond to Carpathy because of that unique lack of incentives that he has. And I think that's understandable. This episode is brought to you by Super Intelligent. Now, for those of you who don't know who are new here, maybe Super Intelligent is actually my company. We started it because every single company we talk to, all the enterprises out there are trying to figure out what AI can do for them. But most of the advice is super generic, not specific to your company. So what we do is we map your AI and agent opportunities by deploying voice agents to interview your teams about how work works now and how your people would like it to work in the future. The result is an AI action map with high-potential ROI use cases and specific change management needs, basically everything you need to go actually deliver AI value. Go to bsuper.ai to learn more. What if AI wasn't just a buzzword, but a business imperative? On You Can with AI, we take you inside the boardrooms and strategy sessions of the world's most forward-thinking enterprises. Hosted by me, Nathaniel Wittemore, and powered by KPMG, the seven-part series delivers real world insights from leaders who are scaling AI with purpose, from aligning culture and leadership to building trust, data readiness, and deploying AI agents. Whether you're a C-suite executive, strategist, or innovator, this podcast is your front-row seat to the future of enterprise AI. So go check it out at www.kpmg.us/aipodcasts or search You Can with AI on Spotify, Apple Podcast, or wherever you get your podcasts. Today's episode is brought to you by Robots & Pencils. When competitive advantage lasts mere moments, speed to value wins the AI race. While big consultancies bury progress under layers of process, Robots & Pencils builds impacted AI speed. They partner with clients to enhance human potential through AI, modernizing apps, strengthening data pipelines, and accelerating cloud transformation. With AWS-certified teams across US, Canada, Europe, and Latin America, clients get local expertise and global scale. And with a laser focus on real outcomes, their solutions help organizers work smarter and serve customers better. They're your nimble, high-service alternative to big integrators. Turn your AI vision into value fast. Stay ahead with a partner built for progress. Partner with [email protected]/aidailybrief. Beyond that, he also just has a bit of a savant reputation. This is the guy who coined the term "vibe coding" back in February, and he is just a great listen. This signal on Twitter put it, "Karpathy speaks like someone who's running a mental compiler in real time, with minimal interpretive latency and almost zero runtime garbage. He's not verbose, he just threads complexity into compressed lossless statements. Most smart people can be dense but they lose clarity. Karpathy keeps clarity while cranking the bitrate like an LLM tuned with perfect temperature control." Okay, so we've got one, Karpathy not having incentives to lie to us, two, the general quality of his thoughts, and the respect for his thoughts, and that of course leads us to the next thing, which is that reversals always feel like a bigger potential narrative violation or shifting force. And even though as he points out, he's been talking about this being a decade long project throughout 2025, to people who weren't really paying attention, this might feel like a shift. This is the guy who coined "vibe coding" and now he says it's "slop" that feels bigger than just some other critic, even if that other critic was saying the same things. The next point of resonance is that there are lots of folks, even who work in tech, who believe in AI but think that the promises are overblown at least in the short term. And Neil Dash wrote a long post called "The Majority AI View" where he argues, "We're in an unusual situation where the most common opinion about AI within the tech industry is barely ever mentioned. Most people who have technical roles within the tech industry like engineers, product managers, and others who actually make the technologies we all use are fluent in the latest technologies like LLMs. They aren't the big loud billionaires that usually get treated as the spokespeople for all tech. But what they all share is an extraordinary degree of consistency in their feelings about AI, which can be pretty succinctly summed up. Technologies like LLMs have utility, but the absurd way they've been overhyped, the fact that they're being forced on everyone, and the insistence on ignoring the many valid critiques about them make it very difficult to focus on legitimate uses where they might add value. Now, to be clear, I hate the framing of this as the majority AI view. This being the majority AI view does not at all resonate with my lived experience of dealing with a lot of folks in this same industry. And anytime someone tries to paint an entire group of people with a broad brush, I tend to bristle. But it is certainly the case that a lot of people feel like this, whether it's a majority or not, doesn't matter. The point is that there is a large constituency of people in the know who feel like the hype and hyperbole crowds out exciting but rational progress. And what's more, it should be noted that some of this group have identified the potentially damaging effects of hyperbole that have nothing to do with AI bubbles. I had a discussion with a friend about a headline from a couple of years ago on this show where Ahmad Mostak, who was then the CEO of stability AI, had said that AI would replace all coders in five years. I turned that into a broader discussion with the headline, will AI replace all coders or something along those lines? And while the purpose for me was opening up the conversation, the person that I was talking to said that there's a reasonable chance that at various parts in their journey when they had been intimidated to get into coding, that's the kind of headline that would have contributed to them staying away and that they might never have made it into the conversation. So bringing it back to the Andre conversation, the point is simply that there are a lot of people who share this view, who feel like they haven't had a loud representative voice. That group, I think, intersects with another group who are responding to this, who are the technologists who are pushing AI and agents to their limits. In other words, a lot of the folks that were head nodding along with Andre are the people that are specifically asking AI to do novel things. In other words, the things that it's quite difficult for AI to do. These the folks who are living at the edge of the capability set and running up against the limits every day. So that big hyperbolic transformation rhetoric is the farthest from their lived experience because they literally can't get it to do what they want. But then man, there were so many of the people who latched onto this, not because of technology resonance, but because they're just simply annoyed with AI people. In some cases, they're market participants who are now positioning for a bubble. But some just don't like the conversation or the people in it for whatever reason. So here are my problems with this discourse as we've had it. First of all, I should note that I don't have any problem with carpetly sharing his beliefs here. Nor even with people I disagree with picking it up and using it on their bully pulpit. To the extent that there are bubble dynamics, if you are one who doesn't wish to see a bubble, these moments which let out air are the single best salve and probability reducer for a bigger potential problem down the line. Second, it should be noted that carpetly is far from bearish here. He articulated that in his post to Danielle Fong also added carpetly being bearish is the most mid-twit response to this. He gives a totally inspiring vision of the future, which has the advantage of actually having a pathway to build it. In addition to his timelines being more realistic, it's incredibly bullish actually. But I do have some issues with the conversation because I think that the critiques that are real aren't nearly as clear cut as they're being presented. First of all, Andre is speaking extremely specifically from his own experience. I just discussed how a lot of the technologists who this is resonating with are those who are running up against the limits of these tools every day. And that is absolutely the experience that Andre has. Frankly, he himself uses a lot of hyperbolic language. He talks about things being terrible and quote unquote slop that are clearly in my opinion loose language and which I don't think most people would consider as such. Now this isn't a bad thing, by the way. We want people building these technologies to have extremely high standards and aspirations. And this is the main point. There are two or even three totally different AI conversations happening right now. Over there in another location is market AI, which is just its own beast. But the two that I'm most interested in for our purposes today are the difference between builder AI and applied AI. Applied AI happens downstream from builder AI and is a radically longer and different process. Applied AI is about taking the possibility that was built during builder AI and turning it into value. It happens at a significant lag to technological process, especially when it runs up against human and corporate inertia years and years of calcified process buildup that AI has to slowly undo and change. And I mean no disrespect at all to Andre when I say that he does not have any stake in or any particular insight around applied AI. He's not in the trenches understanding how these tools are actually impacting the boring knowledge work that is the main part of work that AI will impact outside of the specific technology field in which he operates. You can tell frankly that he's operating in a different world based on his definitions. The definition of agents, those agents that he is saying suck. He makes clear his full human replacement, something you can hire instead of people. That might be a fine definition of a fully realized agent, but it's dismissive not only of discrete automations that take on one workflow, but also of agents that while not replacing entire jobs are replacing big sets of tasks. Maybe those sets of tasks don't add up to human replacement. But human replacement isn't the soul or even necessarily main barometer of AI impact. The point is that companies aren't investing in AI based on a careful analysis of where it will be in five years. They're investing based on what it can do now. Aaron Levy from Box retweeted Andre's clarification post and said, "This is actually extremely pragmatic and realistic based on what is likely to happen, especially in an enterprise context. We have rapidly improving model capabilities, but the diffusion of these capabilities into real life workflows will take time and require lots of integration, change management, and new solutions that must be built." In another post he wrote, "Having talked to hundreds of IT leaders over the last year alone, it's clear we have a capability overhang where the current AI models are already very good at solving many problems that haven't been adopted yet. The biggest hurdles generally are the imagination for what's now possible, the sheer speed at which the tech is changing, and the change management to make it happen." So while on the one hand, Carpathia is arguing that today's agents aren't good enough, broadly speaking, or at least that they have a ton of work to go, you've got Aaron Levy, who's got to be in one of the best positions in the world to understand where enterprise leaders actually are, pointing out that there's a capability overhang where we haven't even close to adopted the capability set that we currently have. I tweeted yesterday, "Hard to get real worked up over AGI timelines when less than 5% of work that could be improved by today's AI actually is. That less than 5% by the way is just my estimate, but I think it's extremely clear that we are at the very, very early innings of integrating this technology into the real world. And the reason that I think the AGI timelines lengthening doesn't mean that we're in a bubble is that I believe that we're going to see massive demand uptick just based on the AI that we have." As the AI investor put it, "My take is that the AGI timeline argument and therefore the bubble claim is nonsense. This was unexpectedly intensified by Andre's recent podcast. AI doesn't need AGI to be transformative. Today's LLM models and infrastructure, like Nvidia's Blackwell, are already capable of disrupting trillions of dollars' worth of knowledge work." Maybe even more crisply, I saw this post from Jacqueline Rice Nelson, the CEO of Tribe, who hosted a private dinner last week with OpenAI, where the main takeaway from all of the leaders from major financial institutions that participated was this. Even if there is a bubble, there's no way I'm stopping using chatGBT. Anyways, if nothing else, it was a very interesting weekend. Lots of good food for thought, and I'm sure lots of developments to come this week. For now, that's going to do it for today's AI Daily Brief. Until next time, peace. (upbeat music)