Speaker 1We're so traumatized by the dot-com bubble that we've priced in exuberance right from the outset. We will be spending far more on running models in 10 years' time than we are today. It feels like we need thousands of times more compute than we have today. Frontier intelligence is not yet commoditizing. I think we'll have a couple of multi-hundred-billion-dollar companies built in that space. One neocloud that is competent is much like another neocloud that is competent.
Speaker 2This is 20VC with me, Harry Stubbings. Now, joining me in the hot seat today is one of the leading minds when it comes to Frontier chips, Walter Goodwin, founder and CEO of Fractal. The business has just raised a $720 million Series C, co-led by Thrive, Founders Fund, Lightspeed, and Redpoint, valuing the company at a $7 billion price. As I said, Walter is one of the leading minds in the future of chips, and we get into that and so much more in the discussion today. But before we dive into the show today, founders and investors sleep on 8Sleep. It is the intelligent sleep system trusted by Mr. Mark Zuckerberg, Dr. Andrew Huberman, Charles Leclerc, and the Aston Martin F1 team. The pod is a smart mattress cover that fits on any bed. It cools each side to 55 degrees, heats to 110 degrees. It even tracks your heart rate, HRV, and sleep stages without a wearable, thank God. The new pod, 6, has a hub 50% smaller than the old one. And starts at just $1,999. Visit 8sleep.com slash 20VC. That's E-I-G-H-T-S-L-E-E-P dot com slash 20VC. While 8Sleep helps you sleep better, MongoDB helps you build better. MongoDB has always been the database developers' love. Well, now it's the data platform AI agents need. Agents need accurate context, fast. MongoDB stores. Searches and reasons over your data in real time. JSON native database, vector search, and voyage AI embeddings all in one place. One system instead of 10. No data pipelines to maintain. Ugh, this sounds too good to be true. Build and scale from your first user to billions of vectors. Run on any cloud, on-prem, or your laptop even. That's why 75% of the Fortune 100 run their most critical apps on MongoDB, moving trillions of dollars every single day. And it's why AI-native companies like Eleven Labs run 40 million agents on MongoDB. And it's why AI. So if you're building an AI, MongoDB for Startups helps you move faster with Atlas and Voyage AI credits and dedicated support. Don't build agents that answer once and forget. Build agents that remember and learn from your real-time data. go to mongodb.com slash agents. That's M-O-N-G-O-D-B dot com slash agents. While MongoDB scales the product, Framer scales the story. So if your team wants a website that looks and feels handcrafted, but is really still fast to ship, Framer is built for exactly that. Framer is the AI website builder that helps creators, teams, and businesses ship production-ready sites faster than ever while getting every detail right. Prompt, inspect, edit, and publish in one place at a whole new pace. Agents and humans work in tandem. Agents bring speed and scale. You bring taste, judgment, and control. The work lands on the canvas and stays editable. Build custom code components, manage CMS content, optimize SEO, and audit for issues all in one place. Enterprise-grade hosting, security, and 99.99% uptime SLAs trusted by leading brands like Perplexity and Miro. Learn how you can get more out of your site from a Framer specialist or get started building for free today at framer.com slash 20VC for 30% off a Framer Pro annual plan. That's framer.com slash 20VC. Rules and restrictions may apply. You have now arrived at your destination. Walter, I am so excited for this, dude. It's lovely to be able to do it in person, and I'm so thrilled that you could join me in the studio, man. Very good to be here with you, Harry. Now, I want to start, for those that don't know, 60 seconds, like what is Fractal? What do we need to know?
Speaker 1We're a frontier AI company of sorts, but we make chips. We make these big rack scale systems that are, you know, fully focused on advancing the frontier of AI capabilities. And so from a team composition point of view, this means that we have, you know, ML researchers down to process experts that fly out to Taiwan every few weeks, down to people that actually look at the thermals and the mechanicals of putting together one of these rack scale systems.
Speaker 2I was listening to you on No Prize, which is joined by Sarah, and you were saying unequivocally that you were a chip company. And I was listening to this thinking, great. But then I'm looking at it going, everyone's moving full stack. No, we're seeing OpenAI have Jalapeno, we're seeing DeepSeek have their own, we're seeing Anthropic build their own, obviously Meta and Mark has his own. My question is, is the future won by vertical integration? And can you be a chip company alone
Speaker 1these days? It's a very good question. I think there's a verticalization in that as all of these different components come together, and especially as you kind of look into the future and you work out, you know, what are these models going to look like? How do the workloads evolve? How does the silicon evolve? There is this absolute necessity to kind of have a vision on every part of that. So Fractal is a chip company, but in a sense, we're also a frontier AI company. And I think what that means for us is, you know, and this will continue to be the case, you know, we monetize by selling chips, we monetize by selling racks of compute to our customers. But it's completely essential that we have a bunch of folks internally who are ML researchers who are looking at how model architectures might evolve, who are maybe even running, you know, smaller scale experiments than the ones that you would see in the leading labs. And so I think there is a distinction here between the expertise that is needed internally to win, and then the kind of nature of the business, what the business model is, you know, we never intend to compete with our customers, we will never monetize internal models, because that's how we've chosen to build this company, we've chosen to compete in the chip layer. But you can't sort of step back from challenging yourself to have expertise in all of these other places. Because, you know, in our case, that would mean you'd place the wrong bets, you have to synthesize everything that you're learning across the ecosystem. And you have to bring that back then to be obsessive about the part of this space that you're building. And if you do this right, you've ideally found yourself an enormous sort of piece of green space where you're incredibly helpful to your partners, to your customers, and to your suppliers upstream. And you've kind of got to know parts of their business as well. But of course, you know, you never seek to conflict with
Speaker 2them. Would you argue that everyone's moving to vertical integration because everyone's fighting against commoditization of one layer. If you think about what, you know, opening I did with Jalapeno in nine months, it suggests some form of commoditization of the layer. Do you think everyone is desperately expanding to run against commoditization?
Speaker 1I think there's definitely a fight to keep your margins expanded that do mean that, let's say, if you are serving frontier models, you know, it is important that you're not squeezed all the way down by very high margins on incumbent chip players. And that's where it's very useful to have, for example, a first party play or another third party play that can be a counterpoint to that. And so I think in a sense, there is this confluence of forces that do mean that, you know, if you're a frontier lab, if you're a hyperscaler, it's very important to kind of have your own retort to various parts of this. I do also think that there's this pretty crucial distinction today still between what is perhaps becoming commoditized and what we still see sort of at the frontier. What is commoditized?
Speaker 2And help me, I'm an ambassador. I look for value in amazing entrepreneurs and businesses. What is commoditized that most people think isn't? And what isn't that maybe people think is?
Speaker 1That's a great question. There is this sort of sometimes concern, sometimes excitement about the idea of intelligence saturation, that as models get smarter and smarter, they will become a kind of waterline that they exceed, where we don't really need greater intelligence for a whole set of tasks. And I think it is true that there are certain tasks for which we see this kind of intelligence saturation. And this is where you then see, you know, an open model is good enough. And these models become commoditized at that layer. I think where that becomes more nuanced, though, and where actually models, even at that level, sort of still fight against commoditization is that there is always economic pressure to, for example, reduce the cost of intelligence. And perhaps counterintuitively, you know, this is actually why I think the frontier labs and the frontier efforts remain so vital is, of course, frontier intelligence is not yet commoditizing. We see, you know, large margins there. We see big deltas against, you know, against sort of everybody else. And also these sort of learnings from the frontier, they kind of trickle down also into building better, you know, better smaller models as well. And so actually, if you look at that sort of fight across the entire gamut of frontier LLM capability, as well as kind of behind the frontier LLM capability, there's still this sort of ferocious battle. And I think, you know, I would distinguish or I would characterize that as sort of not commoditized insofar as if you are a market leader in something, you can still extract high margins. So a good example of this actually is DeepSeq released a few days ago their entire like inference stack, their entire software stack. And it had really remarkable, you know, people very quickly characterized it and realized that DeepSeq had done so well in building an inference stack to run their models on GPUs, that they were getting much higher throughputs, much better interactivity, better speeds than everybody else who had built a deep-seek inference server. And so something that you might have thought would have been by this point commoditized, which is the serving of an open model. In their case, by going full stack, they built a model which was more efficient and they built an inference server for that model that was more efficient. And they'd carved themselves out the right to have really healthy margins on that. So I think even in the sort of open source space, we don't yet see this kind of complete race to the bottom, at least for those folks that sort of, again, try to own a lot of vertical understanding of the space. I think what is commoditizing, and I don't think anybody has ever argued it won't be, is kind of the whole space of cloud compute. The reason it's so exciting today is we don't yet know which ones are competent. So I think there's still a lot of space there for there to be huge winners and maybe sort of huge losers as well.
Speaker 2But I think that's why you see all the neoclouds desperately trying to vertically integrate too, just because I think they're very aware of the commoditization of their own layer. We had the founder of Nebius on the show who was talking about the movement into model routing and inference for them as a core business and a core strategic, just because I think they're aware of that. I think that's true.
Speaker 1And you definitely see every neocloud is trying to build an inference server. A lot of these folks who have built an advantage in inference serving, which is a software advantage ultimately, are also thinking about how they can have more control of their destiny on the compute side. Obviously I'm biased, but I think this is where the kind of neo chip story and the differentiated silicon can actually become a really core differentiator for those players as well. Can you unpack that?
Speaker 2What is that?
Speaker 1So we're still in the very early innings of the kind of rollout of silicon that is not the mainstream conventional, you know, GPUs and TPUs and other XPUs. And it is still much bolder, especially if you look at the kind of way that these build outs get financed to lean into, you know, a new chip platform, a fractile, a cerebrus, and so on.
Speaker 2And we need these new, sorry, just to bring everyone along with us, we need these new chip platforms because workloads are becoming more and more complex and intense. And the specialization of these chips will allow for the execution of these workloads to be much more efficient.
Speaker 1I think these new platforms, you know, obviously we're one of these, you know, have a belief that, in certain workloads, you have this kind of more than 10x advantage, whether it's on cost, or in our case, it's cost and speed. And so, you know, in a sense, they are contrarian bets still, you know, we still don't see many gigawatts of new silicon built out, but they do kind of have this potential to fundamentally alter the economics. And so if you imagine you are a neocloud, and you're the first neocloud to work out how to break out of, you know, a particular bond with a particular GPU vendor, potentially you get to reap this enormous benefit. So we've seen this with, you know, FluidStack is a great example, right? They were, in some ways, the first neocloud to kind of decide that they were going to step bodily away from Nvidia, and they deployed a bunch of Google TPUs. And they've had an extraordinary year as they sort of crossed that Rubicon, and earned the right to start deploying, you know, other chips. And so I think there are opportunities to, as you kind of go more vertical, also take some of these sort of you know, higher risk bets, I guess, and reap huge benefits from that.
Speaker 2What does that do to the Nvidia business? If we think about kind of Nvidia as a horizontal provider, and you as a kind of specialized provider, is the future of chips specialized according to workloads? And how do we think about that market composition, horizontal versus
Speaker 1specialist? The way that I think about this, you know, Nvidia is an extraordinary business, and they build, by many metrics, in a sense, the best AI chips in the world, right? And the best systems, and they have, you know, the best chips in the world. They have extraordinary work on data center scale inference, for example. So in the inference space, I think they've done some brilliant work as well. What I think is different about this moment in time, and sort of core to Fractal's identity, is there is this idea of kind of an evolving frontier. And, you know, the frontier necessitates slightly different bets and a slightly different way of thinking, because you are necessarily setting up a business that is able to kind of look around corners, is able to make bold predictions, and is then able to steer the entire tech product, for example, towards those bold predictions. And I think if you look at Nvidia's business, what they excel at is delivering high quality products at extraordinary volumes that are, in a sense, unopinionated, right? These are powerful general purpose processes that are very good at various parts of, for example, you know, the AI compute world. If you look at where Fractal's bet is on inference, for example, which is building chips that are orders of magnitude faster than these GPUs, that's a particular focus. And for us, architecturally, it's meant this focus on finding paths to huge memory bandwidth, for example, and it's potentially taken us away in some ways from some of that complete generality. I think what's interesting, though, today, and the reason, of course, that we're so excited about this space is the frontier is kind of also an overwhelming portion of the total market. You know, if you look at what's happened just over the last few years, we have this enormous build out of specifically inference compute to serve LLMs at the frontier. And so, you know, what is in some sense specialist is also by no means a niche. And I think that's this sort of real transformation that if you look back, you know, 10 years, even five years, it would have been slightly crazy to build a kind of specialist chip, right? We didn't have any single workload that made up sort of more than, you know, let's say 10% of the total compute burden. Suddenly we do. And I think that demands slightly different company structures that can have that kind of predictive power built in that try to also orchestrate the entire org around what it means to chase after that.
Speaker 2You mentioned memory and HBM there. Why is the memory wall such a prominent problem still today?
Speaker 1Yeah, it's one of these sort of interesting facts about kind of chip physics, right? If you look at the two things.
Speaker 2All great days start with that line.
Speaker 1So if you look across the last like 20 years in the silicon world, there's sort of two things that really make up the kind of top level specs, the kind of top trumps of your chip, right? And like one of them certainly for AI chips is the flop count. How many floating point operations can you do per second? This is like a multiply or an add. And so that's your kind of mathematical power on that chip. And then the other piece is kind of the properties of the memory on that chip, the capacity. And in particular, the bandwidth to that memory. And that's how quickly you can shuttle data back and forth from you're looking at a GPU, you actually have that silicon in the middle of it that is running the maths. It's got that high flop count. And then there's other pieces of silicon around it, the HBM, that are kind of coming in on this sort of highway, bringing data in. And we've been extraordinarily good with Moore's law at scaling flop count. We've scaled it about a million fold in the last 25 years on these chips. Bandwidth, by contrast, is kind of much, much more intrinsically physics limited. We've scaled it about 40 fold in the same time. So, you know, it's lagged behind the flop scaling about 25,000 times. And so we naturally found ourselves more and more kind of bandwidth limited in a lot of the operations that we do. And for something like LLM inference, this has become a kind of critical bottleneck because of this interplay between the generation of text, which I think we all sort of have this sense that this sort of auto-aggressive, you know, sequential thing, one word after another is generated out of these models. Every time I go and generate one of those words, I'm loading from that memory over that limited bandwidth, all of the weights of my model. And today, also, all of the contexts that I'm operating on as well. So all of the past states of things that I've seen. And it's that that is then my bottleneck on generation speed. You know, the reason that if you take a very large frontier model, it might generate at 80 tokens per second, 80 words per second, is going to be not because of the limited maths capability on the chip, but because you don't have nearly enough bandwidth to that memory. I cannot load the weights of the model from memory into that processor more than let's say 80 times per second. And the reason I started with, you know, the sort of sexy line about physics here is it is if you zoom in at the interface, there's a bunch of essentially copper wires that are going from this DRAM memory into the processor. And we've been pretty good, the world's been pretty good at making them smaller and smaller and smaller. But there's only sort of so much you can scale that. And so you have, let's say, a few thousand wires.
Speaker 2And is that why, sorry, you have SRAM? Am I right in saying that? Moving memory closer to it?
Speaker 1So SRAM, yeah, so there's this sort of, you know, DRAM is like often a different building, basically. And you're carrying these boxes down the stairs and then up the stairs again into the processor building where the maths is done. And you have to carry them back again, unfortunately, because there's not enough space in that building, let's say, to store all the boxes. So actually, maybe it's a warehouse and then you've got the office where you're doing the work. SRAM is like having a bunch of boxes that are permanently in the processor, right? So it's a small amount of memory that you cram onto the same silicon where you're doing your processing. But that's the snag as well. The reason that over the last 30 years we've externalized memory into this thing called DRAM and put it on different silicon, these are built in completely different foundries. They've been specialized for very, very different things. So DRAM is completely focused on density. The DRAM bit cell, as it's called, is an incredibly simple structure. And you can cram, you know, many billions of these into a very small area of silicon. SRAM, by contrast, is built conveniently on the same exact process nodes. It's built on the silicon that you're building a processor by TSMC, for example, but it isn't optimized for density. So you might have less than 10 times as much SRAM capacity per unit area as you can get gigabytes of DRAM on that same area, and by extension per dollar as well, because your silicon wafer costs you a certain amount.
Speaker 2When you look at inference and the future of inference, which is then best suited? Is it DRAM or SRAM? Because latency is
Speaker 1important. That's right. So latency and bandwidth as well, like how much can they just cram through this? So this is one of the things that we've been working on over the past couple of years at Fractile is, I guess, the rejection of this trade-off. So SRAM as a memory is the only one today that can be built on the same processor. It's the only one that can be incredibly close and have incredibly high bandwidth. DRAM, much cheaper, much denser, much more capacity, but it's the only one that can be built on the same processor. So it's the only one that can be built on the same processor. So it's the only one that can be built on the same processor, but fundamentally limited in the way that we access it. But if you look at that sort of physics, the fact that it's coming in over wires, it's because we've kind of got it in the wrong place, right? We've got the DRAM sort of sitting in plane with the chip, and we've got this like limited, it's called a beachfront, right? It's like literally the limited kind of real estate on the edge of the chip. What we've been working on at Fractile over the past couple of years is essentially stacking that memory on top of the logic. So looking for ways to, rather than having a lot of memory on top of the logic, we're looking for ways to, rather than having a lot of this kind of limited beachfront, you know, everybody gets to have a seaside property, basically. It's like the entire memory layer sits on top of that logic die. How is that possible? So 3D in the chip world, it is a dimension that people have been scaling for a long time, but it's got all of these complexities. So actually, if you look at an HBM, a high bandwidth memory, it is a stack of DRAM chips, but it's a stack that has certain properties that, you know, don't make it amenable to just being taken and plonked on top of, of your logic die. And so we don't do that today. But there are these process technologies, and this is something that sort of, you know, Fractile's team has a long hinterland in around 3D stacking.
Speaker 2So you're building condos on the beach instead of beach houses to allow for more capacity?
Speaker 1I guess we're building one of those kind of, you know, new built archipelagos where everybody has a bit of beachfront, right? It's strips of land, everybody's right next to a piece of coast. When you put this on top, suddenly you have this whole two-dimensional area where you can have essentially those copper wires are now not going laterally. You have millions of them coming straight down.
Speaker 2When you look at differentiation between other providers, is that your differentiating factor? And how do you analyze the specialized providers that also compete with you in this realm, like an edge? Like what is the differentiation?
Speaker 1Yeah, I think there's a few ways to look at this. You know, one is, one is sort of philosophical, and then the other is technology today. So I think, philosophically, I do think that Fractile is, you know, we have a mission, right? And the mission is to advance frontier AI capability through building better hardware. Today, that's taken us over the past, you know, past couple of years to this particular solution. And I think this extraordinary work that the team has done in unlocking a future where you can have the bandwidth of SRAM, but with the capacity and the scalability of DRAM, it is a world-changing kind of innovation. And it's going to have longevity as, as well. You know, we're, we're very enthused about taking this forwards over multiple generations and kind of compounding a technology advantage there. But at the same time, you know, I think with a, with a mission, if the workload changes, you also want to be able to, to change your response. And so, you know, I think one thing which we're cautious not to become is the kind of the company that cracked 3D DRAM. We want to be challenged to continue to innovate on, you know, whatever comes next. If you look at mechanistically, what it does mean for us though, over the next few years, there is this sort of challenge today, if you look at the chip space, where on the one hand, you have all of these HBM based chips, and at some level, it's very hard for them to diversify, whether it's, you know, the GPUs, TPUs, the jalapeno chip from OpenAI, etched MATX, they have often really brilliant kind of architectures on the chip, which allow them to do things slightly more efficiently than one another and so on. But they are fundamentally kind of boxed in by the like of the fact that you can get only so many flops, and you can get only so much bandwidth, the HBM that sits off to the side of them is exactly the same piece of silicon, it's the same module from the same vendors for each and every one of those chips. And then on the other side, you've got these SRAM based chips that have kind of high memory bandwidth, but also have this huge capacity constraint, right. And so if you look at you know, if you look at Grok, or you look at Cerebrus, what they can do today, which is extraordinary is allow you to run powerful models much, much faster. One of the things that I think suddenly, I've always been excited about with Fractal, it was sort of what I set the company up to do four years ago, is build chips that are better at LLM inference, but in particular build chips that allow us to run that inference much, much faster, because we always believed that we would get to this space where we had what we actually do have today, what's called test time compute scaling, right, reasoning models, long running agents that run for longer and longer, to get better and better at better at what they're doing, you know, a better answer for you. And this is the kind of profile of all of the frontier workload that's sort of changing the world today. You know, if you look at what happens when you turn a frontier LLM to the problem of solving a Millennium Prize problem, it will go off and generate many millions of tokens sequentially, and you scale up this swarm of them as well. So there's sort of parallel element to this, but the many, many millions of sequential token generation, means you're waiting days and days and days to get one of these responses. And it's a space which somewhat tantalizingly hasn't been cracked at all by any of the silicon, any of the chips that we have today, because there's this sort of tantalizing mismatch, I guess, between the fast inference chips, the SRAM based chips that have the ability to, you know, access memory quickly, generate tokens quickly, and the needs of this new workload, the kind of long running agent, which needs a huge amount of capacity, because you have these really long contexts that are emerging. And so the world urgently needs this kind of up and to the right thing where you have simultaneously really high bandwidth and high speed, but also high capacity, so the ability to scale to longer contexts and larger models.
Speaker 2Is this one of those where it always takes longer than you think? Or is this one where you see dramatic inflection points in the acceleration of technology?
Speaker 1I think what's exciting about this one is, and this is true, I think often of sort of, you know, technological breakthroughs, you are always standing on the shoulders of giants. In our case, you know, some of the technology that we're leveraging to do this 3D integration, for instance, has actually been around for quite a long time, it's just not been turned to this exact problem to the memory stacking challenge. And so it's a space wherever you're willing to do the kind of both the architectural effort and redesign your chips to make use of this. And then you're also willing to do the kind of manufacturing operations legwork, and you're able to persuade your partners that you're sort of not crazy along the way. There is actually the possibility here for these kind of overnight transformations.
Speaker 2Can I be blunt? I was talking to a founder of a brilliant chip company last night, and he said there's a huge amount of hot air and BS in the industry. And there's a lot of marketing fluff, but there's not much actually in production. When you look at your etched or your fractals, are we actually in production? Like, how do we actually think about on the ground being in production? And you know, I think that's one of the things that's really important. Is he right with that statement?
Speaker 1It's a great sentiment. There's two parts of this. You know, yes, Fractal has silicon that's gone through the foundries has come back to us. Yes, we're running further trial runs and we'll be doing sort of full product tape outs in a few months as well, which is quite exciting. I think the scale of production is something which is very much worth challenging, right? There is a real challenge, especially for kind of new chip entrants at the frontier, for instance, TSMC capacity, HBM capacity. And I think in particular, you know, for some of these efforts, there is a race for exactly the same resource. You know, a lot of these chips today are built with the same back end partner, for example, you know, whether it's the TPU or jalapeno, MTIA from Meta, Matex or etched, you know, all of those are rumored to be using Broadcom, for instance, same or very similar TSMC process nodes, same HBM technology. And so you can see why there might be like a real kind of gold rush for that particular finite resource. And I think that means that it can be challenging to get to really meaningful scales in a short space of time. And one of the things that we've sort of, I guess, always been quite focused on at Fractile is because we've had this sort of almost unfair reach into the stack, right? The point of a back end partner is they do the kind of last mile, the physical design, they interact with the foundries with the manufacturing side of the business. Fractile has always had that in-house. We've had this extraordinary team that has kind of always worked that way, you know, even before Fractile for decades in how they've delivered their chips. And it's become a really core strategic advantage for us because it means we've both always had direct communion with our manufacturing partners. We've had to do that legwork from the outset.
Speaker 2And just so I understand, the alternative is you go through Broadcom who then have the manufacturing partner relationships. That's right. Yeah. Got you. So Broadcom is the intermediary layer that most people work with that you don't. And that we don't.
Speaker 1What does that enable you to do then? is build completely different kind of building blocks for these chips, right? I think if you look at a Broadcom, you know, I was at a conference this summer where Charlie Kawas, who's, you know, one of the leaders there, held up a kind of Lego model of the next-gen platform. And he said, this is going to be in production with six major customers in 2028, which is a sort of strange thing to say, because what he means is something that looks identical to this is going to be made with six of the major players and given different names. It will be the TPU, MTIA, Jalapeno, you know, 3. Dorito 2. Exactly. And so it sort of reveals something about the structure of this market, right, where each of those chips will have a different name. They will also be different. There are incredible chip teams inside all of those companies that come up with insanely clever kind of architectural optimizations. But in the end, it's going to be exactly that same platform. It's going to be the same stack of HBMs, the same number of chips. A lot of the sort of fundamental building blocks are dictated by the back-end partner because it's a platform technology play. You know, it's a kind of you get to change the color of the paint, basically. And so for us, what it's allowed us to do is like maneuver and build these new building blocks. So the memory stacking work, for instance, is incredibly complex and painful and requires convening multiple, you know, much larger supply chain partners to get them to do something that is kind of highly custom. But once you crack through that, as we have, you know, you then have an asset that nobody else has. And this was a slightly long-winded way of getting all the way to your question about like volume production that also more likely than not has meant that you've gone and found a way to use resources that others haven't. So in our case, a different process node, maybe a slightly different DRAM node or DRAM generation as well that might be going into Frontier HBM. And so you sort of find, you know, the opportunity to build gigawatts of capacity sort of down the back of the sofa with things that other people and this sort of, you know, set of Frontier platforms have already moved away from.
Speaker 2Can I ask you, when we think about the kind of production line, so to speak, that a lot of people told me that I need to ask about tape out. What is tape out? And how do you think about it?
Speaker 1So tape out is the moment where the final file from your chip design process, which is actually basically like an enormous image, we call it a GDS2 file. And the image dictates. you know, where are all of the copper traces? Where are the transistors? Where's the logic that gets sent to TSMC or to your logic foundry partner? And they commence the process of building the mask sets, the kind of stencils that will ultimately produce the chip. And then you start doing your first kind of trial runs, your first production runs as well, potentially. So it is a moment in time that suddenly if you're a chip designer is a bit of a kind of seminal handoff, right, because at the end of this long process of architecture, front end design, verification, physical design, layout, you send the file and if there's a bug in it at this point, you know, it's too late. The mask is getting printed. And so that's why it's kind of a, you know, it's a major date on the calendar sort of thing. It's also though, just sort of today's step one in quite a long process, right? Because that silicon then goes through the fab and you wait months for that to come back and you have to be poised and ready then to receive that and run, you know, your actual bring up process, you know, this package silicon returns to you in the lab. You then sprint to get, you know, a workload running on a single chip, on a tray, eventually on a rack as well. And so that's sort of part two, I guess, is that, you know, at Fractile, we track progress, you know, there's tape out dates in the middle of it, but the thing that we're continually shooting for, which we call task zero is, you know, the day that a particular reference workload runs on an entire. Rack of Fractal chips.
Speaker 2What does no one see about this process that is most prominent to you? If we think about tape out being like a seminal moment handoff, where is the, God, no one quite understands the difficulty, the momentousness of the post tape out date.
Speaker 1I think, well, I think that's it actually is it's what comes next because there is an awful lot that you can fail to prepare yourself for. If you're ultra fixated on that one thing, the kind of handoff of, of the first chip to the, to the fab. And so I think, you know, for us, it's continually tracking, you know, what is, what is it that we're actually trying to build here? We're trying to build systems that yield well. So the underlying economics are great because, you know, we've, we've designed silicon that has the right levels of repair and redundancy it's, you know, highly manufacturable. It makes a lot of sense as a product and then fits into a system that again, yields. Yields well is reliable. And so the sort of building of a product that is manufacturable in volume where you can then find a path to ramping as quickly as possible. That's the goal in some ways. And I think tape out is a, um, I almost feel like maybe over the last few years, there's been this sort of accidental shared obsession between sort of chip startup founders and vent the venture capitalists that are backing them on this phrase. Cause it's such a tangible moment. It's such a tangible date, but a lot of the work that we do at Fractal, you know, all of the work we do on building software, all of the work that we do on systems and a lot of the silicon team as well is focused on those parts of the chip that in a sense are kind of like intangible, ineffable, but are about what it looks like to be able to get this stuff up and running and reliable and to volume as quickly as possible.
Speaker 2When we were downstairs, you said that actually silicon maybe no longer. The bottleneck in the same way that it was before, and maybe data centers were a data center capacity was, can you talk to me about that relationship and the transition of shortage versus not?
Speaker 1Yeah, it's been, I mean, this is why the last four years have been so exhilarating because it feels like every quarter, the thing that is most constraining, let's say our ability to run more frontier models that scale in the data center has shifted. There's these underlying properties that we urgently need, like the ability to run inference much, much faster. The ability to run at longer contexts with better economics. But then we've had this sort of, you know, if you think back three years ago, there was no question of other chips. It was an NVIDIA GPU shortage. And then I think we found ourselves in a underlying silicon and supply chain shortage, you know, acute memory shortages today for HBM and so on, which stretch years into the future, logic silicon shortages with TSMC as well. And I think, you know, if you look at the space today, land and power is definitely another limitation. And, you know, we've had all of the kind of low hanging fruit for industrial sites that can be retrofitted to be data centers, places where, you know, in the US you can get permitting to bring gas turbines close to spin these things up in, you know, 90 days. These kind of often one time tricks have kind of all been exhausted. And so there's sort of we're now exploring the kind of longer tail of, you know, harder, harder ways to find land and power to install these chips. And so on the one hand. Actually, it is data center capacity that is our kind of fundamental bottleneck on how much compute we can build out. And so this is the kind of, you know, if you want to be completely agnostic and invest in, you know, your long AI, you know, by energy basically, because that is obviously the kind of most fundamental. But at the same time, it's precisely that kind of constraining limit that is why it is so vital that we have the best possible chip. For any given job, if you look at Frontier Labs and Hyperscalers today that have like a fast mode on their models, you know, if you're running that on an HBM based chip, you're doing so at this kind of extraordinarily inefficient point on a kind of trade off curve on these chips, you can either be sort of slow and efficient in generating tokens or fast, but but deeply inefficient, sort of far fewer tokens per second per megawatt of power, for instance. So what happens if I want to take a multi trillion parameter model and run it at thousands of tokens per second, but I have a 500 megawatt data center, and I don't have enough aggregate throughput, you know, I can't serve my user base on those chips, there's a real pressure to now install a set of chips that do retain, let's say really good aggregate throughputs, even as you squeeze them to much, much higher speeds, you know, Fractal would be an example of that. And so, although it's land and power that may be the kind of. Most acute bottleneck, even ahead of kind of supply chain constraints, it's actually exactly that that kind of incentivizes chips that are much, much more fit for purpose.
Speaker 2How much of an energy deficit do we have? I know it's an incredibly macro and bold question, but when you look at energy requirements versus energy supply, what do you think the deficit looks like?
Speaker 1I think if you look at backlogs in the US and the UK, it's this extraordinary, you know, the cues for grid connection will be like an order of magnitude more than there is energy in the grid. So viewed in that way, you know, and the extent to which all of those are actually legitimate projects that would get off the ground is obviously something that we can we can question. But it does look like there is this this staggering level of demand. And I think energy is one of these real fundamentals.
Speaker 2Going back to what we said earlier about DeepSeek and their incredible innovation on the inference stack. When you look at China's policy and regulatory environment towards energy, I mean, it's green light anything it's green light anything, including moving millions of people out of their homes because they need the sites for energy provisioning. And in the US you have NIMBYism and in the UK you have. you know, bats and butterflies preventing sites of scientific or natural beauty. Are we not just inherently disadvantaged in a world of energy requirements versus China?
Speaker 1I think this is one of the places where China has somewhat stolen the march on the West, right? And in particular, you know, these mega projects, you know, huge hydro projects, for example, they're ideal for AI compute buildouts because they have this particular kind of signature where, you know, it might have once been kind of inconvenient to have 15 gigawatts of power all arriving at a single site that then needs to be distributed. You know, an AI data center doesn't really care where it's located. And so why not go put 15 gigawatts of AI compute right next to this, you know, hydro plant in the middle of nowhere. And I think that there is in the US and in particular in the UK and in Europe, a lot that we can do to look a little more like that. And I think that the good news here, and this is something which, you know, I think the government in particular, you know, here and European government should be doing more to play up is it's actually exactly that fact. It's that a lot of the power sources that could be brought to bear on this are, you know, renewables can be a huge part of this story, right? You can locate a data center, you know, in Scotland next to an enormous wind farm where previously maybe there just would not have been a story as to why you wanted to have energy generation there.
Speaker 2Do you know what's really interesting for me? Because we often, we do a lot of Instagram now in short form. And the minute we talk about data centers and they're not happening, we did it the other day with Microsoft pulling out of the N scale project. Everyone goes, good riddance, good riddance, you tech bros, good riddance. And it was astonishing to see the universal backlash from bluntly moronic civilians. Can I ask you, you saw inference as a market before so many other people. Wish I'd listened years ago. The thing that I know now that I wish I'd known 12 months ago, it's just the infinite demand for inference. What do you know now about the market for inference that you think isn't that obvious or you hear other people say?
Speaker 1I think the thing that remains, in some ways, it's an even more pronounced version of my original bet. So, you know, I think four years ago, the insight was sort of twofold. One is as you start to get these kind of foundation models that have been trained on everything and then generalized to anything you might turn them to. Of course, you have this kind of hard logic, where training becomes a fixed cost and the deployment, you know, inference is just the use of these models to do useful things. It's the only part that matters. If you were spending more on training in the long run, like why are we even building these things? You know, you really could kind of pack up and go home.
Speaker 2I always describe it to my mother like kind of the Olympics in your same boat. He trains in the gym. Great. And then decides I'm not going to actually do the race.
Speaker 1Right. What's the point, dude? Exactly. This is why we're here. It's like imagine you spent far more money, building a factory than you ever actually recoup from generating something out of it. Right. It's the hard logic of building something with a high fixed cost is that you must believe you're going to be doing enough deployment to justify that. And so, you know, I think that was sort of was part one. And then this bet that we would have, you know, what we call kind of test time compute scaling, the advent of reasoning models, they didn't arrive until two years later, for which I'm very grateful to Noam Brown and the team at OpenAI that was the first to kind of, you know, publish one of these. I think what's happening today is we continue to see this sort of vector of these test time compute kind of scaling laws, right? It's this stack of different ways that we pursue greater capability. So one is, you know, 10xing and 10xing the amount of compute we use to train these models. But the other today is how do we how do we pour more compute into deploying these models into the inference side? And it's a huge part of how you get to greater capability. If you look at, again, the examples of solving a millennium prize problem, that Navier-Stokes solution was, I think, 80 hours of execution from 10,000 agents in a swarm. And so you have there these kind of two axes that trace out this enormous kind of volume of token generation, right? Where I've both got 10,000 parallel agents all generating their own chains of thought, and then they're running for 80 hours. And so it is so different to the kind of, you know, intuition that we might have about an LLM, where we sort of think we ask a question, and the single threaded thing will go and generate almost immediately an answer. When we bring these models to bear on the hardest problems, and I think generally the hardest problems, and this is a bit of good news, you know, are also the ones that have by far the best, you know, the most positive societal impact. It don't generally it is solving a science problem, or, you know, something in medicine that we maybe have a problem with. We haven't been able to solve before. To go and do this, where you have this incredibly economically and societally beneficial thing to go solve, which is beyond the reach of kind of just, you know, applying the model and asking it one shot, you know, what do you think? What's the answers to this? Inference compute is the axis that we have. And the Navier-Stokes example is kind of evidence of that. And so I think the thing that is.
Speaker 2What do you mean inference compute is the axis that we have? It's the axis that we have to.
Speaker 1It's the. It's the knob that we have to turn to get greater capability. So if you imagine that at any moment in time, we have in the world the best frontier model. It will come from one or other lab at any point in time. And the best frontier model is extraordinarily capable. You talk to it and it's smarter than any you've spoken to before. But it turns out that when you ask it, I am working on this, you know, I'm working on this challenge. We're really trying to find, you know, a cure for this rare disease. We're not at a place, and of course, we're not at a place where it's just going to say, oh, here it is. Like, I've thought of this. It's, you know, in my neurons, this is fired out immediately. But the thing that we also discover is if you do throw enough compute at it, if you say, hey, how about rather than just trying to give me an answer, you propose many hundreds of candidate potential solutions that you could go and explore. And you create multiple copies of yourself and you go and traverse the Internet and you read the literature. And you debate with copies of yourself and you exchange information and come back to me in, you know, basically a week's time. Once you've generated maybe billions of tokens in pursuit of finding an answer, you know, then a little bit like if we gave a bunch of smart people a lot of time to solve a problem, you do have a chance that you've come back with a solution. And we really see this. This is the key. It's what every single application of a frontier model to solve a hard problem is going to look like.
Speaker 2So the question to you is, I am a venture capitalist. I am an investor. And I see the crunch, the debt cycles, the credit, the problems that we are hitting now and about to hit. There is not enough money in the world for the requirements of this cycle and everything that you just said there, which I agree with. And then I look and go, and then we haven't addressed the physical world and physical data and robotics and everything in between that's required there. And so I'm holding these two seeds. And I'm seemingly opposing thoughts in my mind going, what the fuck happens now?
Speaker 1Well, this is why we need continuous innovation in every part of the stack that leads to serving intelligence, right? And the good news is, you know, for all that, obviously, we're focused over here on chips. You know, if you just look at the algorithmic side, like model efficiency, the cost, assuming no hardware change to serve a given level of intelligence is coming down precipitously. Deep 80% improvements
Speaker 2in efficiency with no hardware change.
Speaker 1Yeah. We see, you know, longer context becoming a challenge. We see incredible innovations on how you can compress context and do sparse attention and other ways of handling this. So there is sustained innovation on what you might call kind of the compute efficiency of the models, you know, with the same chip, ISO compute, you know, how much more intelligence, how many more solutions can I arrive at? And then, of course, we do have the same on hardware. And we're working very hard to make sure that we have that hardware effect as well. And so. I think what's really thrilling, and it's the reason that I think on the whole, you know, this build out is entirely justified and we won't see, you know, a collapse in this market, is that, you know, far from the fiber build out of 2000, 2001, we have complete full utilization of the GPUs that are powered on in data centers. We don't have dark GPU fleets because, as you've seen, you know, we have this enormous number of problems as a species. That we still have to solve. We now have tools that we can throw at those problems. And we have many orders of magnitude more thinking that we could be doing at sort of inference time to solve the hardest challenges if we had the compute. And so, you know, I agree with you on the one hand. We're not going to get thousands of times more data centers. And I think that's probably a good thing for the world. What we will be able to do is take, you know. Roughly that kind of land and power instead of data center sites that exist today. And over the next few years, through both model innovations and hardware innovations coming together, squeeze more and more and more, dramatically more intelligence per megawatt, per rack in one of those data centers.
Speaker 2So I have Mike Cannon-Brooks on the show from Atlassian. And he said to me, you know, I think the cost of inference is going to come down dramatically over the next three to five years. And that's why. That's pretty good, actually. That's why I'm an ordinate. - Yeah, you can do me in the next one. - Well, I mean, you in the next one is relatively close to just me. You want me doing Russian. That really is a good one. But with the Mike Cannonbrook theme in mind, what happens to the cost of inference?
Speaker 1So I think there is, you know, if you look at the cost, what will a token cost in 10 years time? It's one of those sort of questions that, you know, begs another question, right? Like what is the model that we're running? It's probably orders of magnitude larger than the model that we have today. And so that token is also delivering in some ways, you know, substantially more capability. I think the margins and inference is something that will, you know, will fluctuate and they will accrue in different places. We will be spending far more on running models in 10 years time than we are today. You know, that part I think is clear. But who is going to win in this space? You know, you've got today, I guess you've got chips, you've got clouds, you've got frontier labs, building frontier models. You've got folks that are building, serving open models. And I think one of the reasons why, you know, I am very long on chips in terms of where that value accrues or being one big place where that value accrues is actually because of the nature of the challenge of placing the right chip bet. You know, it's got this intrinsic kind of lag, that lag to design the chip, to get the chip out, to have it built. And then the degree to which it has to have a useful lifetime in a data center means that you don't have quite that same kind of You know, potentially a race to the bottom in some ways in the chip space as you do elsewhere. If you have a better idea, if you can build some sustained advantage in some way, I think you see a lot of the value accrue there. I think you also see a lot of it accrue still at this frontier, you know, the construction of frontier models, which have this sort of sustained advantage over some of the others. It's a costly process. It requires incredibly talented teams working with huge amounts of compute, but it is a process that requires incredibly talented teams working with huge amounts of compute, but it is something which then delivers, you know, who doesn't want the greatest intelligence on earth? Of course, people will continue to pay for that. And so I think the overall kind of prize pool expands. There will be kind of relative winners and losers within that, but it's a positive sum game and inference.
Speaker 2We just had Satya on the show the other day, and he said that we would see a future where every company has their own specialized model with their own unique and proprietary data. And actually a cool kind of value segment would be evals and benchmarks for themselves. And I think that's a really good point. And for that company and for their own objectives. Do you agree with that world of many specialized models on a per company basis? Do you think it's relatively techno-optimist and wrong? How do you think about that?
Speaker 1I think there's absolutely space today. And just if you look at the technology, there is positive impact in taking a model and fine tuning it for your use cases, but it's a transient thing, right? Like generally, the advantage of having something that is internally fine tuned will, disappear as soon as the next release of the next open model arrives. I think that can still become a bit of a kind of yes and, right? Like as long as you have a model where this, as in a model of constructing these, where this stacks on top of the best frontier capability, then I can absolutely believe that there is an advantage to having a kind of an internal, a specialized variant of something that is otherwise constructed at the frontier. But I do think this is one of the hardest questions to answer, because over time, it may well be that there's going to be a lot of uncertainty in the way that, you know, you can only do this with the open models, for example. And if that's the case, you know, what does it mean for, you know, this sort of trade-off where is it better for me to specialize an open model or just to go all in on the thing that's been released, you know, at the frontier that is still pursuing these better compute efficiencies and by extension might actually be very cost-effective for my challenge.
Speaker 2So when you look at that value stack, you said, obviously chips, no shit, where you think is the most valuable, where is the least valuable? I think there
Speaker 1is a danger for anybody that is working, you know, purely in the software layer. I think, you know, I've seen a few plays recently on kind of orchestration, you know, there's a coming wave of heterogeneity in the chips that we're going to have in the world, you know, GPUs, other XPUs, these new chip efforts. And, you know, some clouds and some software providers have set themselves up to be the kind of orchestration layer over this sort of coming wave. And I think there's a world in which that kind of heterogeneity is a slightly transient thing, you know, that, yes, today, you know, we have, for example, a world where NVIDIA having acquired GROK will deploy two chips alongside one another, you know, Rubin GPUs and GROK chips, and they'll do what's called pre-filled decode disaggregation and attention MOE disaggregation, where these pieces of silicon are doing different parts of the LLM inference workload. And so there's this sort of sophisticated, orchestration between these two.
Speaker 2At the same time, it's kind of like model routing, but chip routing.
Speaker 1Yeah, exactly. Chip routing. And, you know, I think there's a kind of technologist view that maybe, hey, why stop there? Why not have like seven different pieces of silicon? And maybe they're from different vendors, and they're each specialized in different parts of this. And I think there's a belief that this might be a huge kind of wave of value creation will be in being specialist in how you orchestrate over these. I think a lot of that could collapse back down over time, you know, that is a fundamental existential reason why we desperately need to have seven different platforms to run a workload. And there isn't really a strong economic case for kind of why we would be able to sustain that as well. I mean, if you imagine, even just the the NVIDIA GROK data center deployment, how many NVIDIA GPU chips do I put in there versus GROK chips, you know, the the bet today would be so this attention feedforward disaggregation, if you'll give me a minute on the tech here. I put the weights of my model on the GROK chip, because I access them really fast on that high bandwidth. And because every user that I'm cramming through that data center, they all want the same model weights, right? So I can kind of amortize, I can share the cost of that much more expensive memory, I can recoup it, make it up on volume, attention, which is the part that operates over the context isn't shared across users, my context is different to your context. And so for that piece, I'm incredibly exposed to the cost of my memory. And we talked earlier about how SRAM is far too expensive, DRAM is much cheaper. So I need to fall back to the GPU and the GPUs memory to run that piece. And so that's the logic of, you know, an NVIDIA data center with these two chips in today. But as I spec that data center, how many GROK chips do I put, you know, what if my context lengths get longer over time, suddenly, I probably wanted to have more NVIDIA GPUs than I did last year, and the context lengths have got longer, how much longer will they get, maybe the model gets larger suddenly, and actually, I wish I had more GROK chips. And so you're very likely to end up in a place where some portion of that capacity in your data center is stranded, you're just not filling it because you've got the wrong mix. That's unacceptable. And it's completely unacceptable, right? Because you have a certain number of megawatts, you're trying to juice as much revenue as you possibly can out of them. And so this idea that you're placing a fixed blend, and you're not going to be able to change that data center for the next few years, right? That's fixed, it's capex, it's being amortized over time. Heterogeneity is actually a really. challenging thing. And I think it's a transient response to the fact that you don't today have a chip that can kind of do it all, right? These SRAM-based chips solve this fast inference challenge a bit, right? But they don't have the memory capacity to do attention, and so we still need to bring that GPU in. They also don't have the flops density back to the top trumps to run the compute intensive part of LLM inference, the pre-fill portion, the prompt processing, because you've got this one piece of silicon, which cost you a lot, and you've had to share the. the area that you might put maths in with the area that you put memory in. So you definitely haven't got as much maths as your counterparty that used HBM, who could use 100% of that chip area for the tensor cores that run matrix multipliers. If you look at, let's say, what Fractal is doing, you suck the SRAM upstairs basically, and you have a DRAM layer stacked on top. All of that compute die on the base is now free to put tensor cores down. You can have as many flops as anybody. You can be a top trumps maxer on that. But you also. You also get the SRAM level bandwidth from the 3D access, and you get the capacity from the DRAM. And so suddenly you have this holy trinity where you've got all of these things in one, and it becomes clear that you don't need heterogeneity and all of this complex orchestration and running the risk of stranded capacity. Was NVIDIA's acquisition of Grok an insurance buy? There's so many fun rumors about this. I think the Grok team is an incredibly smart team, and Jensen is a smart guy that wants smart people on his team. They also did build a platform that, I think, answered a big challenge that NVIDIA faced at the time, which is how do we move into the fast inference space? I think Jensen is quite positive some in this space. He said some nice things about Fractal last week when he was in London, and is in some ways a bit of an ecosystem player. The way that NVIDIA captures the kind of bulk of this market is that they will sometimes be deployed next to other systems. Next to other platforms, other platforms that augment parts of their capability. There is another rumor that obviously Cerebrus got a deal with OpenAI. I think that can be an alarming moment if you're running a multi-trillion dollar GPU company and one of your largest customers suddenly signs on for something that you perhaps hadn't previously thought was existentially necessary, which is a fast inference platform. What's your response? I think both deals were signed on the 24th of December, which must have been a fun time to be a lawyer as well.
Speaker 2And so you think it's a knee-jerk reaction to that diversification from OpenAI?
Speaker 1I think it's a. acknowledgement that some parts of this space at the time required a different solution. And this SRAM play from Grok was the right solution at that moment to be able to run LLMs faster. Today, though, we've already seen ourselves, I think, move on from this. You know, you cannot solve LLM inference with these SRAM-based chips. There isn't enough capacity. The cost to achieve speed is too high. The constraint on the context lengths you can operate on is very, very stark. And so, you know, I'd sort of characterize that solution as, you know, it gives you sort of snappier chatbots, but it doesn't necessarily give you this thing of being able to take kind of overwhelming frontier intelligence, these agents that run for days on end, and truncating that to be orders of magnitude faster. I think that really is where the huge prize is. And today, that's not something that that provides an answer to.
Speaker 2So the different players in the inference space that people know are your modals, your fireworks. Your base tens, your togethers. You bet on one. Which one would you bet on?
Speaker 1I'm friends with founders of multiple of those companies, so I'm bullish on them all. But it's like children.
Speaker 2You always have a favorite.
Speaker 1Yeah. You know what I like about all of these guys is they still have the capacity to dodge and weave, right? They haven't yet locked up huge amounts of their own capacity. They're not a neocloud. They have the potential to bet on a new chip as it comes up. They have the potential to carve out new portions of the market. I think there's a world in which this piece doesn't fully commoditize, right? You can be specialist in serving multimodal models, text models, fast inference. So I won't place a specific bet. I will say that I would rather be an inference provider today, you know, a specialist in running these models, getting them up, than I would a neocloud, actually. I think there is more mileage in being able to create the sort of business flywheel, of signing up these enterprise customers that consume these tokens.
Speaker 2Totally get you. Will they be multi-hundred billion dollar companies? And is it an Uber Lyft market, where winner and significantly less valuable second and third place? Or is it a GCP as your AWS market?
Speaker 1Even that's only three players, I guess. Sure. Yeah, I think you're not going to have 10. And today, you probably do have a long tail that takes you towards that. And if you add the neoclouds into that, you definitely have, you know, tens of players. 10 players, all of whom are maneuvering around billions of dollars. Are they multi-hundred billion dollar companies? I think we'll have a couple of multi-hundred billion dollar companies built in that space.
Speaker 2Can you talk to me about the new funding round? How much did we raise? It was a $720 million Series C. Wow. That's quite a lot of money. How was the process? Was it fun? Because now you're
Speaker 1like the belle of the ball. You know, if you go back two years, you know, we were still teaching people about the word inference. But it makes for a kind of trickier investor pitch, right? Because. When you educate an investor, it's just, it's kind of like leave then. You've got 30 minutes, right? And you have to spend 20 minutes. And four years ago, you had to spend 10 minutes explaining what an LLM was. And then you had to spend another 10 explaining why you felt that training these things wasn't going to be the be all and end all and you'd deploy. I'd then spend five saying, and here's why I think speed is going to matter. We hadn't yet seen reasoning models and so on. So, compared to then, it's been very fun, I think, to be able to go straight in and talk about how we have a completely unique and extraordinary team that can do things differently to everybody else, how we have a very unique position in the supply chain and can maneuver differently. I think a different set of philosophies on how you build to win in this space, you know, that's what I get to spend my time over coffee talking about. And so I think there is a real advantage to now being in the eye of the storm in a way, in a space where I think there is an appreciation of why this is such a crucial problem to go
Speaker 2and solve. 720 million. Talk to me about, is that enough? And suddenly you have that in your account. What changes? Like what do you do that you couldn't do before?
Speaker 1So suddenly there's a weight of responsibility, right? Like there's a number of folks who have staked a lot of capital on us. And I think one of our responsibilities there is to ensure that we are always having, you know, putting multiple shots on goal as it were. So one thing that capital allows us to do is kind of distribute ourselves to look at multiple things at the same time. You know, we're now able to start looking more generations ahead. We're able to explore more parts of the stack than we maybe, maybe previously would have, you know, more innovation at the systems layer, at the networking layer and sort of building this entire roadmap of, of future systems is one thing that capital allows us to do. This is an expensive space, but I think it's a space where, you know, the underlying capital that is needed is a space where, you know, we can do a lot of things. It's a space where there's a lot of capital that is needed to kind of get that chip out, get those first systems back. That's been a kind of fixed cost for the last few years. And in some sense, that's not what most of this capital is used for. I think what a lot of this is now important for is, is that ability to maneuver faster in the rest of the world, in the supply chain, you know, the placing of long-term agreements over certain components that we need to have years into the future and the ability, I think, to build more of that muscle of predictive power as well. You know, we can now start to think about how we run more of our own experiments, work out what might be important in the future. And as I said, Fractal will always be a chip company, but we do need to have strong insights so that we can be more helpful to our customers about what scales well, where model architectures might head next, how you can handle very long contexts, what it looks like to run an agent swarm versus one very long serial agent. So I think that's, I think that's one of the things that we need to start thinking about, you know, all of these almost ML research challenges that we also get to now go and tackle. And we're also expanding geographically. So obviously we've got folks in Taiwan, we've got a team in the States. I'm very excited that we were joined just a few weeks ago by Dan Rabinovich from Meta, who worked on a lot of their AI compute systems there, driving our kind of systems build out. So everything from the chip upwards, you know, the rack scale systems, the networking, the integration with the data center. So it is this very, very long stack and we get to go and build parts of the business, you know, where it makes sense to and pull in the world's best talent as we, as we go after that as well.
Speaker 2Can I ask you, speaking of investors, if you could invest in any one of your suppliers,
Speaker 1who would it be and why? So TSMC is kind of an extraordinary business in that if you look at everything that is happening today, everything traces back to them, right? And it's sort of, I mean, this is not a, this is not a scoop for your podcast, right? I think it's pretty much, you know, pretty well known that TSMC underpins all of the world's chip production. It is cheap at the price. I mean, I would buy that stock every day of the week. All of the world's chips that are at the frontier or the majority of the world's chips at the frontier are built there. They are an extraordinary business, an extraordinary team. So I think we'd go in there. If you could invest in any of your customers, who would it be and why? So I really like some of these scrappy Neo labs. Again, I'm thinking like an investor here. Sure, yeah, that's what I want. And I think we are going to see some. Which one? Well, friends of mine have started Inherent in London. I don't know if you've come across them. They're looking at both, I guess, the sort of the RSI challenge, but also in particular, what it takes to build systems that automate science, automate research. And I think there is a lot of mileage here in, you know, it's not quite the same as pursuing the best frontier model that goes and tackles everything. But I think there are so many interesting questions in what it looks like to orchestrate and deploy models, build systems that are better at solving these challenges.
Speaker 2If you were forced to invest in one of your competitors, final one, before we do a quickfire, who would it be and why?
Speaker 1So I'm a big believer in 3D DRAM and one other company called D-Matrix is pursuing a different architectural direction in the same space. I think both companies are going to be very
Speaker 2successful. What was their last prize?
Speaker 1I don't know. Are you ready for a quickfire?
Speaker 2Mm. So what have you changed
Speaker 1your mind on in the last 12 months? Maybe I'm not quite ready for a quickfire. Do you want to read me the quickfire? That's a good one. Sorry, Ian. I'll just. Is that not an honest quickfire? Please can we keep that in?
Speaker 2Hey, no one's ever ready for the quickfire. But so quickfire, which we're definitely keeping that last version in. What have you changed your mind on in the
Speaker 1last 12 months? I think the underlying pace of capability advance in models has been extraordinary. So these maths results that we saw this summer, in terms of kind of, you know, people talk about the truncation of their timelines, right? I think what has become clear is anything that is kind of verifiable, you know, maths and coding tends to fall into this, will be solved in the next 12 months. We will be able to tackle any of these kind of hard verifiable challenges with front-end models.
Speaker 2Who do you not have on your board that you would love to have on your board?
Speaker 1This is an easy one. I think Demis Hassabis is both an academic hero of mine and a bit of a British hero. And I think he should come aboard. Have you asked him?
Speaker 2Not yet. That's a good way to do it. Demis? What's the biggest misnomer about the chip industry today?
Speaker 1Misnomer is about technology businesses in general, that they're very technological. And it's a very, very, very human space. So much of it is about relationships. So much of it in our case is about having this extraordinary team that can collaborate across the entire stack. So it's that it's powered by humans.
Speaker 2Will Nvidia be a $10 trillion business in five years? Yes. You said that pretty confidently.
Speaker 1It's a very positive sum game. Nvidia today trades, this is a soundbite, so it may be unscored, but I believe it trades at a lower forward multiple than Cisco after the crash, after the dot-com bubble burst. Why is that? We've talked about this bubble before it even started booming, right? And so, you know, I think we're not yet appreciating how sustained this build-out will be. And you don't have to have incremental data centers added necessarily. You know, we talked about this advance in compute efficiency. If I have finite numbers of megawatts, I need to replace my chips every few years with better chips. And so that's a huge advantage for Fractal as a market entrant, because every chip that you see in the data center today will definitely be out of those data centers. Six or seven years tops, many of them won't last more than a couple of years. And so that's an enormous market opportunity for the market leader. They have extraordinary supply chain relationships. They're going to do very well. There is then this enormous. And I would say new opportunity in being the chip vendor that serves the needs of the frontier. And so I'm very long on Fractal's chances over that horizon as well.
Speaker 2You are in charge of the UK. Congratulations. What do you change to make the UK a superpower in technology?
Speaker 1I think, you know, sadly, so much of what unlocks growth is unlocking energy. It is doing everything that we can to bring online more renewables, more nuclear, more sources. of power that allow us to have build out of industries that matter here.
Speaker 2Yeah. I love Alan Chang. I don't know if you know Alan. I love Alan. Yeah. Phenomenal. Final one. What are you most excited for about the next 10 years?
Speaker 1I think it's, again, we're using these models now. What are we using them for? We're using them to solve problems we couldn't solve before. And I think we're only just starting to see that pick up and take off in terms of what it means for the real world. And so it is going to be all of the ways in which life gets a bit better. We find cures for diseases we couldn't previously tackle. We have education that is high quality for everybody. These things are unequivocally going to come. We have the technology to make it happen now.
Speaker 2I totally agree with you. I think it's so awesome. You look at class sizes traditionally, 40 children in a class. You have personalized education for some that like multiple choice, some that like visual learning at their own speed. I was shit at maths, probably unlike you. I was always too scared to put my hand up and say I don't understand because I'd look stupid. And having personalized learning to that point is one tiny example. I so agree with you. Listen, Walter, I love doing this show because I really get to learn from amazing people like you. So thank you for doing it with me. And this has been so much fun.
Speaker 1Thanks very much, Harry. It's been very fun.
Speaker 2But before we leave you today, founders and investors sleep on 8sleep. Mark Zuckerberg, Dr. It even tracks your heart rate, HRV, and sleep stages without a wearable. Thank God. The new pod 6 has a hub 50% smaller than the last one and starts at just $1,999. That's E-I-G-H-T-S-L-E-E-P.com slash 20VC. Every serious AI company right now has the same problem. Agents need accurate context fast. Most infrastructure, though, it wasn't built for it. Well, MongoDB is. 11 Labs runs 40 million agents on MongoDB. 75% of the Fortune 100 run their most critical apps on MongoDB. Trillions of dollars moving through MongoDB every single day. Instead of stitching together 10 different tools, you get one JSON-native database, vector search, and Voyage AI embeddings in the same system. And it runs anywhere. No lock-in. If you're building an AI, MongoDB for Startups helps you move faster with Atlas and Voyage AI credits and technical support to help you ship. Framer is this incredible, complete website platform, not just a builder, so teams can launch and keep improving their sites in one place. Enterprise-grade hosting, security, and 99.99% uptime SLAs trusted by leading brands, like Perplexity and Miro. Rules and restrictions may apply.
PodcastTranscript AI