Go back

20VC: Why Robotics Will be 100x Bigger than Coding Agents | Inference Will Be a $10TRN Market: What No One Sees | Why Software Investors are F***** Today with Naveen Rao, Unconventional

1h 9m 22s

20VC: Why Robotics Will be 100x Bigger than Coding Agents | Inference Will Be a $10TRN Market: What No One Sees | Why Software Investors are F***** Today with Naveen Rao, Unconventional

In this 20VC interview, Harry Stebbings speaks with serial entrepreneur Naveen Rao, who sold MosaicML to Databricks for $1.3 billion and now leads Unconventional AI. Rao argues that energy is the foundational constraint on AI progress: building a single gigawatt of capacity costs roughly $50 billion, the U.S. grid's excess capacity will be exhausted within a few years, and China has outpaced American infrastructure investment for two decades. He contends society must adopt an "energy first" mindset, since physical efficiency gains from Moore's law have stalled and remaining gains lie in co-designing models, architectures, and hardware together. Rao believes specialized chips will dominate, hardware is the best place to invest over the next decade, and pure software investors face a difficult future. He also argues that data is not commoditizing, that curated datasets and training curricula are the true source of model capability, and that reinforcement learning matters because verifiable rewards let systems tackle tasks where data alone is insufficient. Looking ahead, he predicts transformers will eventually be surpassed, the inference market will reach $10 trillion within ten years, robotics will dwarf agentic coding, and AI will expand humanity's ability to pursue fundamental science.

Transcription

13250 Words, 71475 Characters

English
Speaker 1we're going to be out of energy pretty soon in the next few years. This is a blip. It's not some kind of bubble being burst. We don't treat the model as sacrosanct. And we're energy first in our mindset. Intelligence, to me, will have infinite demand. I do think we will be in a post-transformer world. I don't think the transformer is the be all end all. I actually think the hardware is the place to be over the next 10 years. I would say if you're a pure software investor, you're kind of fucked, honestly. The inference market is going to be a $10 trillion market within 10 years. I don't look at, oh my God, AI is going to kill all humans. That's the stupidest worldview I've ever heard.
Speaker 2This is 20VC with me, Harry Stebbings. Now, what started as a very planned discussion, it evolved into one of the most phenomenal discussions that we've had across a wide range of topics from AI to data to robotics and everything in between with an incredible guest in the form of Naveen Rao. Naveen is a serial entrepreneur. I mean, he sold his last business to Databricks for $1.3 billion. Mosaic ML. Now, he's back with Unconventional AI, which has raised hundreds of millions of dollars. Their last round, close to $5 billion in valuation. And I think Naveen is one of the most articulate and clear thinkers across such a wide range of very complicated topics. And he's got this unique skill of being able to really break down very complex ideas into very digestible lessons. He was fantastic, and this show was a joy to do. But before we dive into the show today, founders and investors. Naveen Rao: I'm going to start with the first one, which is the Astor Sleep on 8 Sleep. It is the intelligent sleep system trusted by Mr. Mark Zuckerberg, Dr. Andrew Huberman, Charles Leclerc, and the Aston Martin F1 team. The pod is a smart mattress cover that fits on any bed. It cools each side to 55 degrees, heats to 110 degrees. It even tracks your heart rate, HRV, and sleep stages without a wearable. Thank God. The new pod, 6, has a hub 50% smaller than the last one and starts at just $1,999. Naveen Rao: That's amazing. Naveen Rao: Visit 8sleep.com/20VC. That's E-I-G-H-T-S-L-E-E-P.com/20VC. While 8 Sleep helps you sleep better, MongoDB helps you build better. Naveen Rao: MongoDB has always been the database developer's love. Well, now it's the data platform AI agents need. Agents need accurate context, fast. MongoDB stores, searches, and reasons over your data in real time. json native database vector search and voyage ai embeddings all in one place one system instead of 10 no data pipelines to maintain oh this sounds too good to be true build and scale from your first user to billions of vectors run on any cloud on-prem or your laptop even that's why 75 of the fortune 100 run their most critical apps on mongodb moving trillions of dollars every single day and it's why ai native companies like 11 labs run 40 million agents on mongodb so if you're building an ai mongodb for startups helps you move faster with atlas and voyage ai credits and dedicated support don't build agents that answer once and forget build agents that remember and learn from your real-time data go to mongodb.com agents that's m-o-n-g-o-d-b dot com slash agents while mongodb scales the product but framer scales the story so if your team wants a website that looks and feels handcrafted but is really still fast to ship framer is built for exactly that framer is the ai website builder that helps creators teams and businesses ship production-ready sites faster than ever while getting every detail right prompt inspect edit and publish in one place at a whole new pace agents and humans work in tandem agents bring speed and scale you bring taste judgment and control the work lands on the canvas and stays editable build custom code components manage cms content optimize seo and audit for issues all in one place enterprise grade hosting security and 99.99 uptime slas trusted by leading brands like perplexity and miro learn how you can get more out of your site from a framer specialist or get started building for free today at framer.com 20vc for 30 off a framer pro annual plan that's framer.com 20vc rules and restrictions may apply you have now arrived at your destination naveen i am so excited for this dude i told you i stalked the out of you matt bornstein matt ocho many others thank you so much for joining me today man absolutely yeah i was looking forward to it yeah i think it's going to be a lot of fun because i i went through a ton of your shows i wrote down what i thought were the most controversial quotes there were a few zingers by the way but i actually wanted to start on like the foundational layer of this infrastructure which is energy you've spoken before about how it will be the bottleneck and the importance of it how do you evaluate the state of energy supply today and what are we not seeing that we should see energy supply
Speaker 1continues to be difficult to build right i don't think people appreciate it i mean i've seen numbers from general industries about 50 billion dollars to build one gigawatt so people are throwing around the word gigawatt like it's like it's easy and it's like guys this is not easy just to put that in perspective building a fab like a semiconductor fab is on the order of 20 or 30 billion dollars so this is one gigawatt is more than building a fab like if you if you look at it in terms of dollars and complexity so it's hard i think what we've seen over the last several years has been soaking up the excess demand that's been around so we over build generally and we don't run infrastructure at all we're going to run infrastructure at 100 output so what we've been doing what we've been getting energy for data centers has primarily been from this like can we run things at higher utilization you know and get that next couple of gigawatts out of it so if you look at the us power grid we're on average something like 900 gigawatts of output but i think it's capable of peaking up to like 1.2 1.3 terawatts so 1300 gigawatts so there is some excess capacity and that's primarily what we've been doing in the last couple of years and that's what we've been doing in the last couple of years and that's what we've been doing in the last couple of years and that's what we're going after that's what you can get access to quickly but building infrastructure is hard and we need to build more we're behind how far behind are we i think over the last 20 years china surpassed us in terms of total amount of capacity that happened i think in 2023 but they were building 6x faster than us for 20 years they were coming back from far behind but they really went fast and hard at this and we criticized that a lot because we said oh they're using dirty methods to do this coal burning this they're using dirty methods to do this coal burning this but they're also trying to grow as fast as but they're also trying to grow as fast as possible and i think now they're sort of going back and possible and i think now they're sort of going back and cleaning up some of this so meanwhile we cleaning up some of this so meanwhile we didn't really invest in energy infrastructure and that's didn't really invest in energy infrastructure and that's going to hurt us because it's going to take going to hurt us because it's going to take time
Speaker 2to really get there dude what can we learn from time to really get there dude what can we learn from china being so much further ahead of us china being so much further ahead of us that we should be like way more loose on policy and that we should be like way more loose on policy and regulation what are the takeaways well i think
Speaker 1regulation what are the takeaways well i think some of this is hard because it hits the very core some of this is hard because it hits the very core of our culture and and their culture i think of our culture and and their culture i think china you know can centrally plan we don't china you know can centrally plan we don't and there are advantages to both of these things and there are advantages to both of these things right and in these things where you have these right and in these things where you have these giant infrastructural rollouts yeah there's an giant infrastructural rollouts yeah there's an advantage no doubt about it however i think when advantage no doubt about it however i think when it comes to innovation tip of the sphere sort of it comes to innovation tip of the sphere sort of things like the us is far and away better ai was things like the us is far and away better ai was born here not there you know like uh all of these born here not there you know like uh all of these latest innovations have all come from here now latest innovations have all come from here now they're doing some really great work in china they're doing some really great work in china also i don't want to diminish that because it's not also i don't want to diminish that because it's not just distillation they're really doing some just distillation they're really doing some true innovation however i think that a tip of the spear true innovation however i think that a tip of the spear stuff where you really open up you know new stuff where you really open up you know new areas comes from here not from there so you know i think areas comes from here not from there so you know i think we have to leverage our advantages what it
Speaker 2we have to leverage our advantages what it comes down to when you look at what's scarce today what comes down to when you look at what's scarce today what do you think is not scarce today that will be do you think is not scarce today that will
Speaker 1be in three to five years i think that excess capacity in three to five years i think that excess capacity of of much more conservative growth rates of ai of much more conservative growth rates of ai we're going to be out of energy pretty soon in the next we're going to be out of energy pretty soon in the next few years and it's not easy just to say okay few years and it's not easy just to say okay i'm going to find some more capacity you will have i'm going to find some more capacity you will have used up all of that excess so then we're in place used up all of that excess so then we're in place where we kind of converge to the rate of building where we kind of converge to the rate of building and you know i've heard people say oh we're going and you know i've heard people say oh we're going to build 40 gigawatts in a year there's no way we to build 40 gigawatts in a year there's no way we have never done that i just don't see it as being have never done that i just don't see it as being possible at least not right now maybe in a few possible at least not right now maybe in a few years we'll come up with some new inventions and years we'll come up with some new inventions and we will build our way out but as it stands today we will build our way out but as it stands today we're not going to build 40 gigawatts in a year we're not going to build 40 gigawatts in a year maybe we build 10. if we build 10 that's amazing maybe we build 10 if we build 10 that's amazing that's half a trillion dollars just to put it in that's half a trillion dollars
Speaker 2just to put it in perspective so when we look at the debt cycle that perspective so when we look at the debt cycle that is already being talked about as being incredibly is already being talked about as being incredibly toppy and we're seeing that now starting to toppy and we're seeing that now starting to cascade or unpack in cool weave and a lot of cascade or unpack in cool weave and a lot of concerns around that is it actually just at the concerns around that is it actually just at the start or does the music turn off at the party at
Speaker 1start or does the music turn off at the party at 7 p.m the biggest deterrent of that will be 7 p.m the biggest deterrent of that will be is there demand for ai services and therefore for is there demand for ai services and therefore for gpus and therefore for this excess all right this gpus and therefore for this excess all right this new energy build out obviously i i'm in this new energy build out obviously i i'm in this industry i'm i'm pretty ai pilled i think there's industry i i'm i'm pretty ai pilled i think there's plenty of demand i don't think we even plenty of demand i don't think we even scratched the surface you know there's a lot of problems we scratched the surface you know there's a lot of problems we have to solve in ai still but as we have to solve in ai still but as we solve those and we will over the next couple of years it's solve those and we will over the next couple of years it's just going to open up new new types of just going to open up new new types of demand that we don't even really know about today so my sense demand that we don't even really know about today so my sense is that this is a blip it's not some is that this is a blip it's not some kind of bubble being burst it's sort of like if you said kind of bubble being burst it's sort of like if you said hey is there demand for people curing hey is there demand for people curing cancer of course there is right like if you cure cancer cancer of course there is right like if you cure cancer people are going to want that like if you people are going to want that like if you build an intelligent machine capable of doing anything
Speaker 2build an intelligent machine capable of doing anything yeah people are going to want that i yeah people are going to want that i always remember vinil coaster saying to me i want to always remember vinil coaster saying to me i want to take as much technology risk and as little take as much technology risk and as little adoption risk as possible and it's exactly adoption risk as possible and it's exactly that i have a cure for cancer very difficult to do but that i have a cure for cancer very difficult to do but zero adoption risk but then does that mean zero adoption risk but then does that mean that we're mispricing the excess supply today if we're we're literally staring down the barrel of 18 months running out. -
Speaker 1I mean, you could argue that. suppose that's up to the energy markets, right? That's how they should be pricing and moving up. I do see evidence through anecdotes, at least. I don't have specific numbers to quote around this because every one of these behind the meter deals is bespoke and people are paying way more per kilowatt hour than they would off the standard grid. So I think we're already seeing a two-tiered pricing occurring where the value of electricity for data centers is sort of higher than it is for general use, right? So there's a lot of behind the meter stuff going on right now. Companies are building new sort of technologies like Bloom Energy. There's a Bloom Energy deal for building a couple of gigawatts in Arizona. If that works, amazing. That might be the technological advances we need to keep going, but it hasn't really been proven out yet. And those economics, by the way, are very different than what you see on the grid. How so? I think if you just boil it down to how much people are willing to pay per kilowatt hour for energy for data centers, it's just hard to tell. You can tolerate a higher price because the margin structure is just so much better. If I have a gigawatt of compute and I want to sell that, I can sell it for, let's call it 5x the price. I'm making numbers up, but it's on the order of 5x the price of what I pay for that electricity. So I got a lot of room, wiggle room there. So I could pay 30%, 40% more for that
Speaker 2electricity. When we kind of go slightly further down from energy, which is kind of the core foundational power, to the compute that you use, what do we know now about compute that you wish you'd known two years ago?
Speaker 1I think one thing I wish I'd known is the fact that we don't really see physical efficiency gains anymore. So you can separate out the way we build a computer and let's call it three levels. So the first level is the physical. So if I can send a bit from this point to that point and do it cheaper, you know, let's say I represent zero as being here and one as being here. My phone has a certain weight. So there's energy required to move between these two states. If I make a lighter phone, it changes the amount of energy required. So that's just a pure physical thing. Moore's law, as we generally refer to it, or Dennard scaling, you may have heard, where you make devices smaller, is really a physical thing. So you're making the physical layer more efficient and denser. So those innovations have largely stopped. And if we look at the last few generations of GPUs or other chips, you don't see efficiency gains much anymore. You see efficiency gains from the next layer up, which is kind of the computing architecture. So I would say most of the efficiency gains we've seen over the last several years or several generations have come from smaller data types, going from 32-bit to 16-bit to 8-bit to 4-bit. So you get pretty good efficiency gains because you're moving fewer bits, you're computing with fewer bits. Then the next level, which is not really exploited that well, and we started this a little bit mosaic, is about how do I start enforcing locality and reduction of data movement at the model level? And five, six years ago, we didn't think about it like this because the model was not the primary driver of compute. Or even, yeah, maybe six years ago. It was still, compute was a general purpose thing. I run websites and databases and all that kind of stuff. However, now if I look at the computer solely as a physical device designed to deliver intelligence, it changes the mindset a bit. Now I can co-design all the way from the model, to the architecture, to the physical. And that's what I wish I'd known how much sort of alpha there is left in the top layer. It's not just the physical. We've focused on the physical for a long
Speaker 2time. So how does that design mindset change when moving across the stack with that transition from pre-model focus to post-model focus?
Speaker 1Yeah, I mean, it means we don't treat the model as sacrosanct. And we're energy first in our mindset, meaning that everything we do has to give us an energy advantage. It sounds like a simple thing, but it's not how anyone has designed these things in the past. It's always been about speed. And this actually has its roots that go all the way back to ENIAC in the 1940s, because computers were designed to be faster. That was always the case. And in the ENIAC's case, by the way, actually, do you know why ENIAC was built? No. It's an acronym. Electrical, numerical, integrator, and calculator, I think is what it stands for. So this was a computer that was built by the US government in 1945. It was one of the first large-scale digital computers built. And its purpose, like every other machine that's ever been built, has a purpose. Its purpose was to do artillery trajectory calculation and do it faster and more accurately. Accurately than the alternative. The alternative at that time were humans who literally did the maths. So you have to be faster. That was the key. Next generation comes up. It's got to be faster than that. You know, GPUs came out. They had to be faster at rendering than a CPU. Each thing successively has tried to be faster than the previous thing. That has led to efficiency for several reasons. I mean, making things faster in the same power envelope is more efficient. But also Moore's law was chugging along. And, you know, we went from vacuum tubes to transistors. Transistors kept getting smaller and smaller and smaller. Now we're in a place where that physical side of things is not really changing a whole lot. Transistors haven't gotten much better. However, we have a lot of room left on the model and architectural levels.
Speaker 2Does that mean we're seeing a commoditization of models? Or does that mean we're seeing actually a separation of model quality in those that are able to save energy and have efficiency gains versus those that aren't?
Speaker 1I think that's exactly right. So, you know, one thing I've been pretty good at is kind of picking where people are going to be focusing in the next couple of years. And I think this is going to be a big part of that.
Speaker 2That's exceptional, Naveen. And really, like, credit to you. Do you walk through when you're picking a market and go like, this is where I think the world's going to go. And so I'm going to position myself here.
Speaker 1Yeah, absolutely. And, you know, it also comes down just to interest. You know, this is, for me, one of the most interesting problems there is to build a better computer.
Speaker 2So you sat down here pre-unconventional in the idea maze, and you were like, problem set that I'm going to focus on is dot, dot, dot. What was that?
Speaker 1Well, I kind of look at it like, if I were to, solve a problem, what is the effect? Like, what does it unlock? In fact, it's kind of like being a scientist. Like, being a good scientist is all about asking the right question that if you answer it, it unlocks many more questions. That's almost the hallmark of a good paper. So for me, in this endeavor, it was clearly like, okay, well, we're going to run out of energy. That's clear. The paradigm we have hasn't really been focused on this. I think this is going to be where the focus will be because we need to keep scaling, right? Like, we need to deliver intelligence in many new form factors and greatly disrupt the cost in order to make it ubiquitous. So no one's thinking about this yet. I should start it.
Speaker 2Can I ask, we were talking about like model efficiency and how it can deliver in terms of energy. We see models getting more and more efficient today. How do we think about where we are on that model efficiency curve? Is there a lot of room to run there? Or are we kind of at the asymptotic point where you're like, we're eking out the final
Speaker 1increments? No, no, no, we have a lot to go. And I think the reason is, is we've not done this true design between the model and the physical side of it. So you can come up with a concept and you may be able to get, say, let's call it 5% efficiency gain just from software on a standard GPU. That sounds great. However, you can only exploit whatever concept so much because the physical thing is built in a certain way. If you look at our brain, they co-evolved. We co-evolved the intelligence layer and the physical layer. And we're able to exploit things in the intelligence layer in the physical layer. So sparsity, for instance, sparsity, means let's say just not having connections between computing units. I can get some kind of gain out of using sparsity in a GPU, 100%. But when I have a standard off-chip random access memory, the limit of those gains is capped. I can't go too far because that physical thing is not designed necessarily to exploit this kind of sparsity. So if I can build something that exploits it, I can get massive gains. That's really what the brain did. That's how it co-evolved.
Speaker 2When we think about that co-evolution or that specialized of architecture to the task almost, does that mean that we'll unwaveringly have the specialization of chips? We see Etched and we see Fractal and Olix and Cerebrus, obviously, but a number of people going after specific workloads. Is that the future of compute? How do you think about that? 100% it is.
Speaker 1And this actually goes back to even Turing's original essay. He wanted to build an intelligent machine. Yeah. I mean, that was his stated goal right at the beginning. So we went down this path of we took his machine, which was a theoretical construct, and we built physical versions of it. So anything we build is not truly quote unquote Turing complete because they have finite memory and stuff like this. But we were able to build applications that were super useful, right? I mean, we could run businesses off of them. We could edit videos and pictures and text and all kinds of great things. So that's a huge productivity gains over the last 60 years. But it deviated from that original goal. And now we've kind of come back. So the purpose of the computer, I think, is to provide intelligence. It's to me, it's just a physical manifestation of intelligence. And so now we have to figure out how to make that better. And what's interesting is that if you make a better physical manifestation of intelligence, you made something general. Our brains in some ways are general. However, our hardware is fixed in its workload. It doesn't do programmability in the same way that we think about it in a CPU. So the GPU is already down this path of building customized architectures for workloads. And I think we're just going to hyper optimize around this. And that's a good thing because now we can build intelligent machines that are actually capable of building anything at some
Speaker 2point. And so forgive my financial mind. You're a wonderful technologist. I am a delinquent trader for a living in many respects. But we mentioned like, yes, unwaveringly specialization of chips. To what extent does that massively reduce the TAM for a horizontal provider, NVIDIA specifically, versus dude, do not be so small minded. It is still so early. We have 50 extra room to run. It is that. The market is going to be so small minded. It is that. It is that. The market is going to be huge.
Speaker 1I mean, look, I think NVIDIA executed extremely well. on the path to getting us here. I mean, when I started Nirvana in 2014, we didn't really understand how big the market was gonna be. It was clear to me that we needed a new computing architecture around neural networks. That was a new way to describe compute. And to me, it was like the most important thing a computer could do. I think Jensen recognized this relatively early on as well. I mean, he started to talk about artificial intelligence at the GTC in 2014. And I remember it vividly because a lot of the graphics guys were pissed off about this. They were like, why did you spend so much time on AI? It's nothing, it's not a important workload. And graphics is, right? And he correctly recognized that this was gonna be important, invested a ton in that direction and actually executed very well. We had to go through this evolution of computing architectures from what we use those machines for to what they are eventually going to become. But I think now it's pretty clear to me that I don't need to care about anything else. I only need to care about running a neural network very, very efficiently, very, very scalably. That's gonna be the figure of merit.
Speaker 2To what extent do you think we see the commoditization of chips? You know, Microsoft last night released their chips. You've obviously got Google with amazing success with TPUs, Amazon with theirs. You've got Anthropic building their own. You've got the wonderfully named, I love this, Jalapeno. Do we see that just inherent commoditization of chips?
Speaker 1So I actually don't see chips being commoditized. I sort of look at it as it runs in parallel to what we saw. So I don't see any of that happening. I don't see any of that happening. we're not there today. And expertise doesn't really go out of style. It's just going to take on different forms. And what I think a future will look like is that we start building, we'll call it models, but I actually don't think it's a model. I think it's the whole machine. I think of intelligence as having a physical and having the intelligence layer as one. They're not separable. I have a machine. It's almost like an organism that I can teach to be an expert at various tasks. And the value will come from building that expertise up. It's like, which school did you go to? How did you learn how to be a neurosurgeon or a chef or whatever? That provenance matters, right? Like you went and studied under this person and you learned their methodology, their thinking. That's where the value came from. You or any human is an intelligent machine top to bottom, but it was the way you were taught that actually made you valuable. And I think we're going to start to see something similar in machines. And we can replicate that intelligence and all of that, but the way you were taught is actually going to be the important thing. And so part of that is actually. Knowing how to get to an end result that is real, that will work, that is verified.
Speaker 2When you think about the value in that learning, a lot of that is in the data that one accrues over time, in the years that you're doing something. We see data companies today, I'm in Macaw, but we see tens hit astronomical levels of revenue that previously were like astonishingly large. I mean, they're still astonishingly large, but in such short time. Naveen, dude, you're such a smart dude. It's such a. It's such a joy talking to you. I just kind of want to ask you lots of questions about the world. What happens to this market? Does it just consolidate over time? Does it unbundle more and more and you build more defensibility? What happens? I think
Speaker 1it specializes, to be honest. And I think if you specialize and actually start to work on important problems, you have a big market that's defensible. Again, it's a power law for skills. You could argue that all humans are capable of learning many skills, but certain skills are more valuable than others. And the same thing would be true, with data and curricula that you build, right?
Speaker 2And so we see the specialization of data companies. So you don't think they're commodities?
Speaker 1Yeah, I don't think it's commodity. I think it started as a commodity because it was sort of like at volume gathering of observations, right? But as you start to construct higher quality datasets, that's what we call it in the AI world, like higher quality for coding, higher quality for chain of thought or whatever, you actually start to have a lot of value in the data. The model is almost just a manifestation of what it learned, in a sense. So the data layer, I think, is going to be pretty important going forward. I don't see that commoditizing. I think building models and knowing how to choose that data is actually the game right now. The big model companies, it's largely a data problem. I mean, they are making advances in the model itself, but that's not been where most of the capabilities come from. It's been from curating data, using different sorts of loss functions. We had pre-training before, which was a very particular kind of loss. Then we moved to what we call mid-training now, which is reinforcement learning, supervised fine-tuning, all of these different sort of mixes. So that curriculum, how you went from pre-training to all these different sorts of trainings to modify the behavior of the model is actually the secret
Speaker 2sauce. Can I ask you, you mentioned that kind of pre-training, RL, and then obviously a real shift has moved to post-training. With that shift of emphasis from pre-training to post-training, do you think that is an enduring permanent shift in the efficiency of model improvements and how we get there? Or do you think it's just an ongoing shift in the efficiency of model improvements and will we move back to RL being the primary or pre being the primary? How does that look in terms of importance over time?
Speaker 1Well, I don't know that it's more efficient. Actually, I think RL is more inefficient if you look at it in terms of how much better the model gets per unit compute or per unit dollar. It's actually more inefficient, but it does keep moving forward, right? I think the idea is that we want to make the models better at the tasks that are important. And so I think it's probably for the next several years, yes, very important. I think we'll start to build specialization in those middle layers, maybe even in the pre-training layer. So I don't think the model companies are thinking very long-term in terms of performance, like, "Oh, I'm going to make some massive breakthrough on the way I think about an AI system that's going to give me an advantage." I would say the big labs are not doing that as much.
Speaker 2Do you think they can afford to in the day-by-day race
Speaker 1that is this current ecosystem? That's the problem. It's really fast, and you've got to be on top of it. If you're one month late on a model, that's. Yeah. That's disaster if you're OpenAI or Anthropic or Grok or Google. They can't afford to take those big risks just yet. You're seeing this. When Elon spun up XAI, he basically took, "Okay, here's the paradigm. Let's go hire the people," and boom, he got to the frontier pretty fast because he had the resources. So what that points out to me was that there wasn't a ton of very deep innovation. It was taking the pieces, putting them together with a team that knows how to build them and executing very fast. I think that's very different than trying to come up with a new paradigm. And it's not a new paradigm. It's a new paradigm. You look at Yano Kun's company and Fei-Fei Li's company. They were trying to do deeper innovations. And I think that might be in the realm of startups. And it starts to look like drug development. All the big risks are taken by startups, and all the incremental stuff is done by big companies. The big companies buy the startups when they show promise.
Speaker 2I sometimes ask simple questions, but I don't know the answers, and you're very smart. Why does everyone get so excited by RL today? You mentioned the mid-learning there, and I just have so many people, "Ah, I just love RL." It's all about RL. I'm like, "Oh, I agree. Yeah, I agree." There's several reasons.
Speaker 1If you think about the way we learn, it's primarily reinforcement learning. If I were to teach you a new skill, learn how to serve a tennis ball or something, it's really an RL loop. I would give you a paradigm of what it should look like. Here's how your serve should look. Maybe demonstrate that or show you a video or something, and it will allow you to build an internal model of what good is or great is. And then you keep judging yourself against that internal model. Yes, I'm getting there, I'm getting better, or I'm not, and I need to ask for help. So that is RL. And I think the reason it's important is we cannot gather enough data to solve every task in the world. We simply can't. You can't pre-train your way out of this by just gathering data. It's just too hard. Models, frankly, aren't efficient enough to make use of it. So you need to have a paradigm where you can start to say, "Okay, these are the 10 most important tasks." And then I have some sort of what we call a verifiable reward. I have that internal model. Like a good serve is I hit it really hard and it went in. It's verifiable. I can measure the speed. I can measure whether it was in or out of bounds. When I have that verifiable reward, then I can start to say, "Okay, did the actions I took, did those actions lead to that good outcome or not?" So now I have a structure that allows me to solve any set of general tasks. And that's why RL is so
Speaker 2exciting. Do you buy Yann LeCun's post-transformer world theory? And how does that change everything?
Speaker 1I don't think I truly understood this until maybe a couple of years ago because transformers conflate a lot of things. What we care about is reasoning, being able to reason through a problem. I don't know who said it, but it's like intelligence is what you do when you don't have the information, not what you do when you do have the information. So being able to reason through a problem from some sort of mechanistic or first principles understanding is what intelligence is. So transformers kind of, sort of have that, but they really do it through this agentic kind of flow where you're taking multiple samples and you're deciding what's good. So I take one trace and then compare it to another trace and I say, "Okay, this trace is better." So I'm going to call that my reasoning trace. And so the way we kind of implement this reasoning thing is through a retrieval mechanism. So attention basically has this context, which is a retrieval thing. And then we can sample off of that and eventually get to something that looks like reasoning. I think this is extremely inefficient for how we do reasoning. We don't need to build reasoning this way. Our brains do not work in this way. We don't build a huge trace, pump it into the input of another model and then produce another trace. That's not how our brains work. So there are way more efficient ways to do this and way more high performance from an algorithmic standpoint ways to do this. And so we will move past the transformer. Do you think what it takes to be a great founder has changed or not? I don't think it's changed. I think the basic principles are still the same. I think having a viewpoint on the few things that you can do in the future, your vision actually may matter more now than it did before. You know, I think in the pre-AI world, you could be a pretty good founder by just being able to execute. You don't have to have a very good vision. You just have to know how to execute. And, you know, if you just go out, execute a big company, which isn't that hard because big companies are slow and, you know, lumbering. I think you could actually build a startup and you could sell it or build a company out of it. I think these days, if you're just based on execution, I don't think you're going to get very far.
Speaker 2And you say, I don't think it's going to get commoditized in data. I think specialization, I think there is value there. Which layer does get commoditized over time?
Speaker 1Yeah, it's a good question. I mean, arguably, energy is already a commodity, right? So this is already commoditized. I think a certain amount of intelligence does actually start to become commoditized. We're already seeing it, to be honest, like models that are six months old. Let's say the Chinese open models, you know, that level, they're pretty useful, right? Something that's six months old is still pretty darn good. So I would argue that's a hallmark of commoditization, right? The frontier is not the only thing that matters. For certain problems, it is. But in six months, that frontier will be in the open models.
Speaker 2So then help me out. What happens if the delivery mechanism or the output of all the infrastructure is then commoditized?
Speaker 1Well, I think commoditization doesn't necessarily mean it's bad. It just means that margin falls quite a lot. But it also means that the market is very big. So you could argue that tomatoes are commoditized, but we still eat them We still need them, you know, so we will have a a huge market for intelligence and the market will continue to grow, but the margins may not be huge. I actually don't think the model companies will be the ones who own the margin of the whole stack. It's not even true today. Like NVIDIA owns the margin of the stack today, right? I would say they're probably the one who take the lion's share of the margin.
Speaker 2A lot of people say to me the cost for inference will actually go massively down over the next few years and we'll see the commoditization potentially of the inference market. Would you agree with that or not agree with that?
Speaker 1Well, let's break it down. What is commoditization of the inference market mean? What that means is that the operator who's running inference, I assume, yeah?
Speaker 2Yeah, absolutely. Your fireworks, your modal, your base tans and the services that they provide.
Speaker 1Because if you look at their value prop, what they've done and they've done very well is take a model that exists, an open model say, and then make it work well, you know, with uptime guarantees and make it, you know, relatively cost efficient to run on infrastructure. So they're an infra play, right? They're basically, mapping the model onto physical hardware and doing that well. Now, I think there will be value there in the future, but the price of inference, like what people are willing to pay per token will drop. And so they're basically, they have to squeeze more margin out of the hardware's capabilities to the delivery of that token. And that, you know, the amount of juice in that layer, I think will decrease. So I think it'll result in them having lower margins. And I think the model itself, like is commoditized. You have several different ways to get it now, but the hardware side of it probably won't be as good as the hardware side of it. And I think the model itself, like I said, probably won't, won't collapse. I think that part of it is still going to have an advantage because you can still squeeze more juice out of like, Hey, I convert energy to intelligence more efficiently than we have more, a lot more juice to squeeze there versus the infrastructure layer. In my view, I actually think the hardware is the place to be over the next 10 years.
Speaker 2Dude, it's quite fucking hard doing hardware investing. It's a very different mindset. Is this generation of investors equipped to be hardware investors?
Speaker 1No, I would say if you're a pure software investor, you're kind of fucked, honestly. And I'll say that pretty confidently because SAS, I don't think is going to be where you gain margin anymore. And we had this sugar high for the, since what, 2011, 2010 timeframe till now of, you know, I can very quickly spin something up on an AWS cloud service, test the market quickly, try things out, and then go after a hundred million or $200 million market. SAS was like massive fragmentation of the market because the delivery mechanism became very easy and frictionless. Now we're in this place where applications start to become more about the intelligence layer. And I just don't see software being a huge market in 20 years.
Speaker 2Do you not just think SAS becomes Lagora? And what I mean by that is like, is Lagora SAS? Well, kind of, yeah, enterprise SAS, but kind of not, no. So does like the semantics not just change? I don't think anyone really is like starting fresh being like, Oh, we're, we're a SAS company.
Speaker 1Right. So exactly. I think you're right. So I was taking the classic definition of SAS, I suppose, but I think companies where you deliver an outcome and you do so through a combination of software and services. Absolutely. You know, we're seeing that in, in several companies and I, I invest pretty actively in startups generate the seed stage. And, you know, I I'm seeing a crop of companies come where they're taking engineering tasks that have existed for a long time that were done in a traditional way and have to be done correctly. And they're making those tasks much more, much faster, much cheaper by using AI plus humans. It's great. That's, I mean, I would say Harvey was a first example of this for law firms where the old way was I have, you know, a set of lawyers who go and look at contracts and, you know, write contracts, that kind of thing. And they basically said, well, I can use AI and verify the outputs with humans and deliver the same thing with fewer humans and deliver it faster at the same quality or near the same quality. And I will disrupt that market. So I think we're going to see it.
Speaker 2This is sort of the new SAS. We see it, you know, I call it word wank, which is where like, you know, VCs come up with a, coin theory of the world. And it's like, you know, the trillion dollar services thing, you know, I think Sequoia did a brilliant piece on, and I love Julian who did it. It's very good. But like, are we not just slightly jazzing it up? That's just the future. People using AI to do things efficiently. There's like a whole glamorization of it. A bit overkill, isn't it? Like, aren't you just investing in a law firm when you do that? Effectively, yeah. But it's a better version of a law firm, right? But you'd be okay to do that? Because I meet a lot of companies doing exactly what you say. And I'm like, I don't want to do a law firm. I'm happy doing a Legor or a Harvey, which sell to law firms, but they're not actually a law firm.
Speaker 1Well, but we are seeing that like some of the large law firms are now investing in AI directly. So I think in its limits, you'll see basically Harvey sells to the law firms because the law firms have the channel, you know, is that channel going to be enduring? I don't know. But I could also see them saying, well, you know what, and we could just sell directly to companies like personally, I would buy that. If Harvey said like, hey, I have a solution that does all your NDAs, it does it reviews your contracts, well, like all these things, and I can gain confidence that it works, I would send 80% of my workloads, my legal workloads to them.
Speaker 2I am intrigued because you spoke about some firms that are building their own, I think you saw K&E in the legal space, commit like $500 million to it. You know, Lynn Fireworks, where I'm an investor, spoken very openly about owning your own intelligence, and the importance of companies leveraging their own data to build their own models. Do you think that is a very viable future? Or actually a slightly techno optimist prediction from a very specific
Speaker 1standpoint? I think it's a bit techno optimist, because I think you have to have a certain amount of scale in order to build your own intelligence and own it. Keep going with law firms, for instance, if you're Cooley, or one of these big firms, yeah, you can invest in these things. And you can make it happen. If you're a mom and pop shop in the middle of Nebraska, you can't do that. So what you're going to use that as a service of some sort, right, you may augment your attorneys with that service. And that's how you'll you'll provide service to your clients. So I don't see this as every company will use AI in a way of leveraging their past data to make their own models better. They're not going to do that. It's just it's too hard for them. They want an out of the box solution. So I think there is a market for selling those out of the box solutions. But it's going to be this long tail, the big players will start to customize because they have the scale to justify it. It makes sense. And it goes back to this idea that, you know, every part of our world is adversarial. Like, it depends on who you're fighting, right? If you're fighting another big law firm, and that big law firm goes and uses AI to supercharge their margins, well, you kind of have to do the same. The mom and pop shop probably isn't doing that. And the people they're, you know, fighting against are not doing that. So they're going to stay in this other other realm for some time.
Speaker 2I think one thing that underpins a lot of law coding is the percent of salary that we will then transition to technology spend. When we look at lawyers today, you know, I think it's about a billion dollar legal market in the US. And you're looking at that going, well, how much of that will be eaten by these providers? I think you see that same question there with coding, you saw Benioff say that they'll spend $300 million of a $6 billion developer salary budget on core code and coding tools significantly. My question to you is, will that stay at 5% for developers? Will that go up to 30% and 50%? Or will it come down to 1%? Because that is the motherlode question.
Speaker 1I actually think the percentages will remain maybe sort of stable, because we end up with this verification problem. It's hard because training humans to know what's good is actually the hard part. Developers, you know, it's actually not about what your direct output is. On this day, it's not like lines of code, it's actually like quality of code that matters. So very experienced, very top notch developers, those 10x developers, whatever, you know, they developed a sense of what is good, what looks good, what's aesthetically right. And they can, they can make things work fast. In this new world. You know, those coders, those people who know what's good, can actually verify the output of models and become more efficient. So I think what happens is we just produce more, and the market size, let's say that we're paying for humans plus AI doesn't change drastically, just their output greatly changes. That's the way I look at it, because play it out a little bit. So I'm spending in a company, let's say $10 million a year to develop my software. Now, all of a sudden, my customers of that software start having different demands. 10 years ago, they're like happy with one year release cycles. Now they're saying, Hey, I want my bug fixed in a week. And so the only way to do that is actually to use AI with the humans, I'm not doing less, I'm actually doing way more, I'm fixing those bugs in a week, and eventually fixing them within an hour. But I'm still paying $10 million to keep developing this whole thing. And I need the humans that can verify those outputs and actually move to a goal that's that's useful, fixing the bug, adding features. So I think the pace of delivery goes up because the demand side starts to demand it. And I think that is the key thing here that's being left out is that it's not that everyone's job goes away, because the same task can be done for less with AI plus humans. It's that now, yes, that is true. But I will I will now do way more of those tasks, because the demand side is increased, you don't get one without the other, when the supply gets cheaper, it changes the demand side as well. So I think that's what's going to happen is that we just end up doing more, we have more release cycles, more changes, like, here's a physical example, I think that people can get their heads around, you know, let's say today, we have a rate of building a building, if I want to build a skyscraper, I'm going to have to build a skyscraper, it takes two years of construction, say, and that's based upon x number of humans with, you know, y sets of skills. So let's say all of a sudden, I have robots that can do 80% of that, what's going to happen? Well, okay, there might be a disruption in physical labor for a moment. But then you'll say, you know what, all of a sudden, my cost and time to build that same skyscraper, it was two years. But now 80% of those tasks are done by robots, I can bring the whole thing down to eight months. So it's not that you're just going to build the same number of skyscrapers and just do it in to build twice as many skyscrapers. there's going to be more things churning, more buildings being torn down, more labor going into that market. So the total number of projects will go up. The amount of dollars going into all of those projects may be the same, but it'll be a combination of humans and robots to build
Speaker 2them. Naveen, you said that you obviously angel invest. You angel invest very successfully. You kind of downplayed it. I think where I invest today, a contrarian thought for me is I do not like robotics investments. I think we are so far from where we need to be. And I do think that timing is important. Now, I'm not saying the infrastructure to power robotics. I think the data sets around them are very interesting. We see a lot of people like Mecca or Micro AGI that are actually in the infrastructure build out for it. But like the actual application layer where we see like robot cleaners, robot builders, we are so far away from where we need to be. I don't want to be there and I don't want to take the timing risk. Am I wrong? Am I naive? You're much smarter than me.
Speaker 1Well, again, I'm pretty pilled on this one. So I am going to disagree strongly on it. Okay. When you say we're so far away, put that in terms of time. What do you think is the timeline? Five plus years. Okay. We're not so far apart on this. I think we'll have working prototypes for some of these tasks in the next two to three years, and the market will emerge in five to seven years. I think that's right.
Speaker 2I think there's also a clear line of delineation between humanoids and non-humanoids. Non-humanoids absolutely see that clearer pathway. The humanoids, I'm really just struck. Whenever I think something's really obvious, I'm always like, I must be very wrong. Because the humanoids is so obviously so far off.
Speaker 1You know, I think we're going to be in this weird world where we live in exponential development cycles. It's really hard to apply our old models. Something that looks like so far away today with an exponential curve ends up being much faster than you thought.
Speaker 2And like TAM, I think the inference market, what do I know now that I wish I'd known 24 months ago, that the inference market was just like unbounded.
Speaker 1It's unbounded. It's multi-trillions. Like if you asked me 10 years ago, you know, what were trillion dollar markets? It's basic needs. It's housing, food, transportation, right? You would never have thought something in tech would be a trillion dollar market. And I think just AI models themselves will be a trillion dollar market in the next few years. And the inference market is going to be a $10 trillion market within 10 years. So this is insane to think about, right? And to your point, we're moving some of that labor market and part of the value generation stack to technology. So that's where we're getting these giant TAMs. So the way I look at like unconventional, the reason that I think it's a, it's a fantastic market to be in is that if we knock off a zero, one order of magnitude of efficiency on delivery of a token, that effectively 10Xs the market in some ways. So if something was a trillion dollar market, now all of a sudden you have way more applicability, the base of potential uses just increases more than 10X. This is Jeven's paradox. So I think we're in this place where we can build these exponentials and just unlocks massive value and market. So I actually think robotics is going to be a hundred times larger than agentic coding.
Speaker 2Well, well, well, that is slightly disagreeing with me, Naveen. Can we just like, before we do a quick fire, can we just unpack that? Because I don't want to be completely moronic. Why do you say that?
Speaker 1I think we're still scratching the service of how to, how to see the output of intelligence. So the physical world is something we've just not been in at all for intelligence. Like I would argue robotics to date, like industrial robotics are not really intelligent. They're rote. We're imbuing them with intelligence that we have, like how to do a task and we're just doing it rote. That's not really novel problem solving. We will be in the realm of physical problem solving in the not distant future. Like imagine a world where I have billions of robots roaming around and they actually solve problems dynamically. That's a problem. That thing just broke. I'm going to go clean it up. I'm going to, I'm going to replace it. The cost of delivery of cleaning everything up in our world, building new buildings, you know, solving physical problems will not drop to zero. I hate those drop to zero kind of statements. It will greatly reduce. So I think that market will be enormous. And that's why I think it's just going to be much bigger than what we see today in the virtual world. Can I be blunt?
Speaker 2Where specifically then do you invest in robotics? In the model companies that power them, the data companies that power the model companies, the consumer application layer
Speaker 1where? I don't think the application layer yet. I'm more in the technology stack. So I actually think these guys who are doing the models haven't really figured out what they're going to look like when they grow up and that's okay. I'm not suggesting they need to, like, I don't think the value will be in the model. I am investing in companies that are building models. I think they are probably going to end up building end to end devices. They're going to build the robot with a model that does something very, very well. That's how they're going to sell it. My personal take is that it's going to go together. You're not going to have a separation. Do you care about price?
Speaker 2When the TAM is what you said it was?
Speaker 1Not really. No. I just see it as like, I need to put some percentage of my wealth into that space because it's important. And that's where there's going to be a ton of growth. And so it's interesting because we're in this place where value investors or whatever you want to call it, like traditional VCs are like, oh, you know, that's way overpriced. Like guys, when we're talking about trillion plus dollar TAMs, the price is almost irrelevant. You know, I completely
Speaker 2agree with you. The challenge is very often we have a customer too, or a boss too, and it's LPs. And the LPs are an even lagging indicator who think that ownership and price are very important. And when you rock up to your AGM and you say, hey, ownership's gone down and price has gone up. They go, fuck you.
Speaker 1They do until they start to see the big firms also have the same trend. And they're like, ah, okay, maybe, maybe that's where the way it's going. Right. If you could put all of your money
Speaker 2into one company that is not your own, what would it be?
Speaker 1Oh, yeah, probably like a physical intelligence. They are solving, in my view, something fundamental about how you define a task and then execute it. So conversion of a task to motor output. To me, that's the problem to solve right now for the physical world. So outside of what we're doing and building a new computer, I think that's the next big problem.
Speaker 2You've been incredibly successful across companies. What does no one tell you about having money that you wish you had been told?
Speaker 1I think it doesn't change your values. You know, you still have a finite amount of resources, but also at the same time, you still value certain things in certain ways. I'm still kind of cheap when it comes to certain things. There's no reason for it to be. What are you cheap with regards to? Well, if you ask my wife, you know, probably clothes, that kind of a thing. You know, I just like, what? I'm not going to pay that for a set of jeans. I think travel still, like, you know, I'm getting to a point where I will try to optimize my time. But overall, I'm like, ah, that costs too much. I'm just not going to do it. It's not a, you know, it wouldn't matter if I spent the money or not. I just, I just don't see value. So I think that's the thing. I put money into things where I see value and I can put a lot more money into those things. But places where I don't, I actually don't spend any more.
Speaker 2Dude, we're going to do a quick fire round. So I say a short statement. You give me your immediate thoughts. What have you changed your mind on in the last 12 months?
Speaker 1Yeah, I think in terms of investing, I've changed my mind on, on useful metrics for progress. So I think we always looked at it as like, oh, we need to look at market validation. As the main metric. And I think that's an old way of looking at it. In fact, I think it comes a lot from SAS because market validation makes sense when there's not a deep technology route. Like when you're saying, well, I'm building a product that serves a purpose. It's all about market validation. But when I have something that is on its face, almost true by inspection, like I'm going to cure cancer. You don't need to validate the market. You just need to validate the progress on the technical problem. So I think right now investors are in, are in two camps. Some are saying, well, let's go, let's, let's invest heavily in these companies. You know, valuation be damned. You know, we're one of those companies. And the other set is like, oh my God, that's undisciplined. It's like, you don't have market validation. I actually think that's fundamentally wrong. And a lot of growth investors are still in that bucket. So I kind of grew up, you know, reading all the books about how you look at risk and investing. And I've changed my mind quite significantly on this. Not just because my company is doing it, but I think now we're in this place where we're solving really deep technical problems. Robotics, computing, biology, and it's not about market validation. The market is there. The market is very clearly there. You solve that problem, a hundred percent people are going to buy it. It's about progress of solving the problem.
Speaker 2If you were completely resource unconstrained, professionally speaking, what would you do differently?
Speaker 1I would start to build my own manufacturing. I want to go closer and closer to the source of raw materials. We're in this world now where there are so many supply constraints from the ecosystem point of view. If I were unconstrained and I could build anything, like this is kind of what Elon's doing, to be honest. Yeah, we should build our own sorts of fabs because I can hyper-optimize that layer for what I need. I don't need to take a thing that is a commodity that's been built for a hundred other purposes and make it fit my needs. I would build exactly the thing I need and build a better product. And frankly, I see it as all the value of a computer comes from delivering intelligence. If I can hyper-optimize that all the way down to how I process the silicon ingot when it comes from the factory and how I slice it, I absolutely want to do that.
Speaker 2Great answer. What area of AI do you think is most overrated today where you would not invest?
Speaker 1Yeah, I think those quick hits where you have a standard thing that's done by humans, I'm just going to try to automate it. That's the worst place to invest right now, I think. I think it's very overrated. I actually do think that like agentic coding stuff is quite overrated in terms of a market because I think it moves to the model provider. I don't think you get much from the others. I know Cursor had a great exit and that's wonderful. But the model providers, the model itself is where most of the value is coming from. from. I don't think it's the harness or the IDE so much. It's not super sticky. I've seen it in my own company. People move between these things pretty fluidly. So it's the model that matters and what capabilities it delivers. So I think agentic coding as a market modulo the model providers is something I wouldn't invest in today.
Speaker 2What percent of token traffic will flow through open versus closed in three years time? I think it'll be 80% open.
Speaker 1What does that translate to in terms of dollar traffic? That's a better question, actually. I think it's going to probably converge on something that looks very much like the cell phone market. The mobile phone market is just roughly speaking, iPhone is 20% of the total market, but 80% of the of the margin, something similar to this, we'll probably see that there's a relatively small number of very important problems that frontier models are needed for. And you'll pay much more for those things. However, all of the like routine tasks that you do will be handled by open models. So I actually think there'll be a rough 50 50 split on spend. But in terms of tokens, it'll be only 20% to the frontier models.
Speaker 2Who is the single best investor meeting you've ever had?
Speaker 1And why? Well, all right, I can give a bit of a story for the first one. I think, you know, since I started Nirvana, I've probably had, I don't know, 1000 investor meetings, right? So after, you know, the 800th one to 800, the first one don't really leave an impression anymore. So very early on, you know, I had a meeting with Steve Jurvetson, who has now since become a good friend of mine, I was nervous as hell, because I'd heard so many, like amazing things about this guy, super smart, he's just gonna take me apart. You know, that's kind of what I was I was told, I had a conversation with him, which we just started geeking out on numeric representations. And I was like, this guy's an investor. And then I send him a couple of papers, he like marks them up and ask questions, I'm sending responses back to him, like, this is not what I expected from an investor ever. This guy's like talking to, you know, like a technical colleague, in a sense, so to me, that was one of the best investor meetings I had, because we were doing something so new that no one pitched it yet. No one had pitched a hardware startup in 10 years in 2014. This guy was almost champing at the bit to invest in something like this, because he'd been thinking about it himself. And so that was just like, boom, let's go.
Speaker 2I love Steve. I remember having him on the show. He was he was fantastic. You talk about like a willing to be non consensus is what I love about him.
Speaker 1That's right. You have to be to make to get make big bets that will pay off. By definition, this is already wrong. It's already old. So you have to be non consensus.
Speaker 2The challenge I find with being in Europe, though, is actually you have like the commoditization of ideas. And what I mean by that is actually now is being a founder is quite a touristy thing to do. Everyone wants to be a founder for the experience. The quality of ideas is reduced. I don't think Americans are inherently better, smarter, more talented. But I do think you are more innovative around idea selection. And that's probably because of exposure, what you see, how you think, where to play in the structure. You know, you're still doing expense management with AI.
Speaker 1Yeah, no, totally. I think I think we are given the permission to think bigger here. In Europe, you're not because the whole investor network and the way people think is so conservative. If you come in with a crazy idea, they're like, that guy's mad. I'm not gonna invest in him. Whereas here, it's like, that guy's awesome. He's thinking big.
Speaker 2No, I completely agree with you. It's absolutely right. The hard thing is, and I've got this, I've got this genuine, okay, let's do this. There's a Series A now, and they are building space stuff. The team is great. They're awesome. But dude, it's 20 million check from a 400 million fund. It's 5%. You can't lose that that easily. Dude, it's a super risky play in space. And I do not understand it at all.
Speaker 1I sort of look at these as in the early part of the technology cycle, like all of us who are technologists are going to give you a story. And that's what we need to do is build a narrative. That story is going to be wrong. 100% will be wrong. I don't know anything else, but I do know it's wrong. I don't know exactly where it's wrong. I don't know how wrong it'll be, but it is wrong. What you're investing in, I think at that stage is someone to drive the bus, not the bus itself, right? So the idea is right. The direction is right. And the team is what you care about. Can they drive that? Can they make the requisite pivots and, and, uh, and right technology decisions to actually drive the bus to something that matters? The challenge actually that becomes
Speaker 2in this market, you don't have the time with them. You don't. Yeah. I would love to say, yeah, I don't know. You probably got two to three meetings.
Speaker 1You're right. And you got to do as much work as you can talk to people that they know and whatnot, but it's hard. Yeah. I agree.
Speaker 2Final one for you, dude. What are you most excited for? I like optimism. I don't like the cynicism that we have around bubbles and bullshit. What's your light? I'm just fucking pumped for this.
Speaker 1I am personally pumped that I look at AI as a, a tool and capability that enables us to ask much bigger questions about the universe. There's so many things we don't understand in this world. And it's because they're unapproachable. They're unapproachable from, um, just the number of humans who are, who could think about that problem and actually make progress. The surface area of problems we can go after and start to solve starts to increase. And to me, that's extremely exciting. It's just, it's just idiotic to me. I actually completely tired of it now. And it's like, no, no, no guys. AI is, is the thing that enables humans to actually start to explore more of our universe. We can do that when we start to understand deeper parts of physics. You know, we, we have these ideas of like wormholes and these things, these are all like theoretical constructs. We can now start to use AI and start making progress. Can I actually build something that exploits that, that physical phenomenon or is it real or not? It's like, these are questions we can start to ask in the past. We just couldn't, we didn't have enough humans that were capable of, of approaching these questions. So to me, the ability to move science forward and to, to move human understanding forward is so exciting. I can't even tell you, it makes me feel like a kid. It was a world of wonder out there that we just couldn't even think about. We got browbeaten into our own capabilities of like, this is all we can look at. And now all of a sudden that whole world is open to us.
Speaker 2Shows like this are why I still love doing the show after 10 years. Like the breadth of your knowledge is, is just like wonderful. It's a, it's amazing for me to witness honestly. And so thank you for being so willing to engage across such a broad spectrum. And you've been amazing,
Speaker 1dude. It's a lot of fun. I love, uh, I love these conversations where we can riff on so many different topics.
Speaker 2But before we leave you today, founders and investors sleep on eight sleep. Mark Zuckerberg, Dr. Andrew Huberman, Charles Leclerc and the Aston Martin F1 team. It even tracks your heart rate, HRV and sleep stages, without a wearable. Thank God. The new pod six has a hub 50% smaller than the last one and starts at just $1,999. Visit eight sleep.com slash two zero VC. That's E I G H T S L E E P.com slash two zero VC. While eight sleep helps you sleep better. MongoDB helps you build better. Every serious AI company right now has the same problem. Agents need accurate context fast. Most infrastructure though, it wasn't built for it. Well, MongoDB is 11 labs runs 40 million agents on MongoDB. 75% of the fortune a hundred run their most critical apps on MongoDB. Trillions of dollars moving through MongoDB every single day. Instead of stitching together 10 different tools, you get one Jason native database vector search and voyage AI embeddings in the same system. And it runs anywhere. No lock-in. If you're building an AI MongoDB for startups helps you move faster with Atlas and voyage AI credits and technical support to help you ship. Don't build agents that answer once and forget build agents that remember and learn from your real-time data. Go to MongoDB.com slash agents. That's M O N G O D B.com slash agents. While MongoDB scales the product, Framer scales the story. So if your team wants a website that looks and feels handcrafted, but is really still fast to ship, Framer is built for exactly that. Framer is the AI website builder that helps creators, teams, and businesses ship production ready sites faster than ever while getting every detail right. Prompt, inspect, edit, and publish in one place at a whole new pace. Agents and humans work in tandem. Agents bring speed and scale. You bring taste, judgment, and control. The work lands on the canvas and stays editable. Build custom code components, manage CMS content, optimize SEO, and audit for issues all in one place. Enterprise-grade hosting, security, and 99.99% uptime SLAs trusted by leading brands like Perplexity and Miro. Learn how you can get more out of your site from a Framer specialist or get started building for free today at Framer.com slash 20VC for 30% off a Framer Pro annual plan. That's Framer.com slash 20VC. Rules and restrictions may apply.

Podcast Summary

Key Points:

  1. Energy supply is the critical bottleneck for AI infrastructure, with a one-gigawatt build costing roughly $50 billion and the U.S. grid's excess capacity expected to be exhausted within a few years.
  2. China has built energy capacity far faster than the U.S. for two decades, though the U.S. retains an advantage in frontier innovation.
  3. Physical efficiency gains from Moore's law and Dennard scaling have largely stalled, shifting the focus to architecture-level and model-level co-design.
  4. Naveen Rao argues the industry must become "energy first," co-designing models and hardware rather than treating the model as sacrosanct.
  5. Specialized chip architectures are the future of compute, and hardware—not pure software—is where the greatest value will accrue over the next decade.
  6. Data is not commoditizing; curated, specialized datasets and training curricula are the real secret sauce behind model capabilities.
  7. Reinforcement learning is key because verifiable rewards enable solving general tasks where pre-training alone cannot gather enough data.
  8. The inference market could reach $10 trillion within a decade, and robotics will eventually be far larger than agentic coding.

Summary:

3 billion and now leads Unconventional AI. S. grid's excess capacity will be exhausted within a few years, and China has outpaced American infrastructure investment for two decades.

He contends society must adopt an "energy first" mindset, since physical efficiency gains from Moore's law have stalled and remaining gains lie in co-designing models, architectures, and hardware together. Rao believes specialized chips will dominate, hardware is the best place to invest over the next decade, and pure software investors face a difficult future. He also argues that data is not commoditizing, that curated datasets and training curricula are the true source of model capability, and that reinforcement learning matters because verifiable rewards let systems tackle tasks where data alone is insufficient.

Looking ahead, he predicts transformers will eventually be surpassed, the inference market will reach $10 trillion within ten years, robotics will dwarf agentic coding, and AI will expand humanity's ability to pursue fundamental science.

FAQs

He sees energy as the critical bottleneck. Building one gigawatt of power costs roughly $50 billion and is harder than building a semiconductor fab, and the existing excess grid capacity will be used up within a few years.

Rao calls it a blip rather than a bubble being burst. He believes demand for intelligence is essentially infinite, similar to demand for a cure for cancer, so the long-term demand case remains strong.

He argues transformers implement reasoning inefficiently through retrieval-like attention and repeated sampling. Brains reason differently, so more efficient and higher-performance algorithms will eventually replace the transformer.

He believes hardware is the place to be over the next 10 years, since physical efficiency gains have largely stalled and there is far more room to optimize at the model and architecture layers.

He expects it to become a $10 trillion market within 10 years. He also predicts that open models will handle about 80% of token traffic while frontier models capture most of the margin.

He does not expect chips to be commoditized, but he does see a degree of model commoditization, where six-month-old open models become quite useful while the frontier keeps advancing.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.