Speaker 1Hello, everyone. Welcome to another episode of The Circuit. I am Ben Beharin. Greetings, programs. I'm Jake Oldberg. This is a great week for semiconductor conferences. We have the return of Semicon West to San Francisco, which I think many people are fans of. Arizona was great, but there's a lot going on locally here, which is coinciding with the Applied Materials launch of their EPIC Center, which is going to be cool. We get a tour of that. I'll be there on Monday getting the tour. Yeah, Monday, which will be cool. AMAT's actually been in the news quite a bit lately relative to some stuff to BESI, hybrid bonding and memory. We've obviously talked about WAVE 4. So that's going to be a good, good event. And they're having their investor breakfast Tuesday, which we'll report on. And then we're going to go to OCP and have an insane amount of meetings at OCP where IR Labs, our friends, Mark Wade and team, is actually going to have a working demo unit with GUC and YWIN. So we're going to see some working bits transferred with CPO. That will be exciting. It's going to be a big show. Like, I'm surprised how many people are coming to this show. It's sold out. It's sold out. I mean, I've personally had people be like, do you got any more tickets? I'll buy some off you.
Speaker 2I've had multiple people ask me for tickets. So just so everybody knows, we're talking about the Open Compute Project Summit, which I think has become one of the top two, three events in semiconductors. Right. It's this and Computex. And it's. Yeah. It's a big show. It's kind of hard to comprehend how big it is. It has outgrown San Jose.
Speaker 1It's. Yeah. Yeah. They can't be at San Jose next year. Like, the amount of people asking me to come by their booths, even like the Boyds and Eatons of the world, right, who are doing Liquid Cools, Sidecar 800, VDC. Like, they're all, we're going to be there. We want to show you all our working goods. So it's a big show. Anyway, we will be there. If you are there also, feel free to reach out or snag one of us when we're walking the floor trying to get nuggets from engineers on the floor who hopefully don't know who we are. And that's the shtick for this week.
Speaker 2Yeah. If you go and hit us up on email or Twitter, DMs are open, all that. It's going to be a busy, busy, busy show. Busy, busy show. Busy, busy show.
Speaker 1Okay. And on to this week's news. Probably the biggest event, I think, just going on was Marvell's Investor Day, which seemed to get a very good reaction. As a matter of fact, the excitement and kind of dollar lifts that they shared helped a bunch of names, particularly in Optical and Interconnect, both competitors and adjacent to Marvell. They showed some very aggressive numbers that I don't. To be honest with you, most of the sell-side notes I had read were not as high, even close to their estimate, so very bullish. I'm not sure I found anybody who was projecting this kind of size for Marvell. But you were there, so tell us, boots on the ground, how it was.
Speaker 2So, yeah, I was at the Marvell Analyst Day on Tuesday in New York. It was – the energy in the room was really good. I mean, it was just like everyone was very upbeat, very positive. Marvell talked about a lot of interesting topics. To your point, they put out these five-year revenue and EPS targets that – yeah, there's no five-year consensus number, but if there were, these would have been above them. So they seemed very, very upbeat and very constructive. I think, you know, one of the many things they accomplished is they have shifted the narrative around the company significantly. Right? Because we've talked about this before. For years, this has just been – the only thing people wanted to ask about Marvell was how much share are they losing at Amazon. Right? That was the entire conversation, Alchip versus Marvell. And now it's, you know, they have a much – They have a much wider breadth of customers and a much wider breadth of products, and they talked about a lot of those. And we can, you know, we can quibble with any particular point on, you know, some of these products are better than others and whatever. But it was, you know, it was good to talk about something other than Amazon and Alchip. There's a lot going on there, and there was, you know, we don't have to talk about the same topics now. There's much more going on there.
Speaker 1So my top-level question, and I guess this is also from, like, folks you talked to because I sort of brought this up, but nobody I was talking to cared about this as much as sort of I did. You know, we've talked about this before, that essentially as long as we've been going to analyst days for Marvell, I don't know, four or five years, it kind of felt like every year they had a different way of doing it. And I guess the question now is, do we think they've found their lane? Like, we talked about this a little bit after earnings. They talked a lot about XPU attached. Now, to me, again, that's not just the straight lane because it's not just XPU attached. They have networking silicon. They have parts of that stack that I think are important, and we'll talk about some of their new ones. But do we think that they have positioned themselves, like, accordingly now? And from that, this is the lane. This is the lane we think they belong in. Because you're right. They don't really seem to be talking up XPU anymore or hardened arm. They can do that, but that's not what customers are coming to them for, largely. It's the periphery of those accelerators, periphery of the racks, and then increasingly network connectivity and IO, which is great. There's a lot of people in that space. Fantastic. But is this the lane we think they belong in?
Speaker 2I'm going to look at it a little bit. I would say. I would say that they've had the pieces all along. I think they have, a year ago or so, they sort of shifted the messaging more around just, I think they started calling it data center infrastructure. And, like, they had all the pieces. They've been heading in this direction, and now a number of things have improved that the story seems much more fleshed out, right? So I don't think it's, like, this is, like, something out of the blue. I think they've been. I think they've been working toward this for a while, and it's just they're able to sort of put more substance to that. I mean, maybe the substance was there all along, but this. I think they've gotten there. So I don't think. I don't see this as, like. I know what you're saying. Like, for a few years, they were sort of changing the story every year. I think they've been on this track for a while, and it just. They have more to talk about, and they've delivered on a lot of things. They've delivered on some of it. And I think one of the tricky things about what Marvell and Broadcom do around custom. You know, we've talked about custom ASIC for years, and for a long time, the custom story was more than just, like, a processor. Again, for both companies, Marvell has always been good about saying, hey, we can do this, we can do that. We can do all these different things. And it just. The story seems to have congealed much more coherently now, because. Yeah. And my point is, like, they have all those other things, but they also have processors, too. And now, like. I think for a lot of people, there's a struggle to understand what custom is, and what. And importantly, how you differentiate in that market. How has Broadcom been able to hold on to that business for so long? We don't really get a lot of that. And Marvell has been, you know, fighting to get into this market. And. Has been better at, like, giving us more visibility into what's there, and all the different pieces, and all the different IP blocks that they can offer. And it just. The way they're telling the story makes a lot more sense now.
Speaker 1Yeah. I mean, I'm not disagreeing with the parts. I just. I. You know, as a mental model, I like to know how companies think about themselves, position themselves, see the swim lanes in which they compete, create narratives. You know, just at a high level. And so, that's been my only point, is, you know, they've gone through evolutions of, you know, who are we, and where are our strengths, and where should you think we compete, and whatnot. And so, I do think this is right. I mean, I do think the inference-slash-true scale-up compute domains are going to benefit them a lot. Again, I think people forget about the silicon. part of their story, the networking silicon, the switch silicon, a whole slew of things that they have. Um, but that to me is, you know, the, the, the timing, I think for the way they're positioning themselves is lining up well. And then I think that's why they kind of had much larger numbers as a part of what they think their dollar amounts in their swim lanes are, um, which, which again seemed pretty high, but, uh, you know, for a lot of people, this is a new, new sites, new types of models for Marvell across, uh, across compute infrastructure, um, and all of the connectivity bits. And, you know, the other thing I think about too is, I mean, it really, they really are up against Broadcom in a lot of ways, but they're also, you know, tight with NVIDIA in terms of NVL and pretty partner friendly across some other parts of the system. And I just wonder, you know, in some of the areas where I think Broadcom is not going to go in particularly, I would be shocked if they ever adopt NVL as a standard, even though I think that, uh, the market's going to demand this, uh, in terms of connectivity and interoperability of compute, um, how much of that opportunity where, where, where either Hawk or Broadcom doesn't want to go, that Marvell can go and starts to meet some bigger pockets. It's of, uh, hyperscalers and a whole slew of other folks who, uh, who again are, are going to compete in the connecting the scale up domains, um, across a range of periphery, right? From, from memory, from CPUs in the scale up domain to wanting to work with your NVIDIA infrastructure, all of those things. Um, they seem well positioned for a very heterogeneous infrastructure world, I guess, is the point I'm making.
Speaker 2Yeah. Yeah. I mean, I think, uh, there's, there's a lot going on in the data center market that's, that's making this possible, right? A lot of the things, not all, but a lot of the things that Marvell is talking about today are capabilities they've had, chips they've had for years, or at least segments they've played in for years. And this is like, we're now at the point where the data center spend is so big that even sort of small products that used to be sort of little niche things they did in the corner are now real. Real volume or significant amounts of revenue are possible. Um, right. Because they, they went through a whole, a whole bunch of products that they make. And I, you know, they've, they've acquired some of those, but like, there's just, there's a lot going on there, right? If you think about a server, there's, I don't know how many chips there are in a server, thousands of chips in a server. A lot of those they've been making for years. And now suddenly like the volume matters, it's big enough. And I think that's, that's how they get to these big revenue numbers is it's not just they're helping somebody design a chip. They're, they're selling, you know, millions and millions of chips into the data center now. And it adds up, right? And I think that's, that's what's going on here. So I, right. But then to your, to your point about like the hetero, I agree. I think the heterogeneity of racks is increasing and will continue to increase. And I don't, I don't know that every, every hyperscaler and every AI lab is, you know, long-term going to have their own accelerator, but at the very least, I think they will, they are all going to do it a little bit differently. They're all going to have different servers. They're all going to have different rack architectures and that's good for, that's good for
Speaker 1Marvell. Yeah. But, but this, this piggybacks on, you know, kind of two, two, two things that we've done reports on and, you know, tried to estimate the dynamics of who's best positioned for this, but you're, you're never going to have a homogeneous set of infrastructure in the world, right? Unless, I mean, I think you could even make the case that if the tier one neoclouds are successful long-term, they probably won't be solely in video shops. They might be largely, but I don't think there'll be solely in video shops, which really means that you, you know, need heterogeneity, but all of that needs to be networked together. That's just one of the things I brought up in this, you know, report we did called the future of compute is fungible in that, you know, what needs to be fungible is the workload so that it can go to any compute domain and maybe heterogeneous compute domain, which is something I don't think people have thought about yet. Cause you know, when I talk about scale up domains, it means connecting one, two, three, four racks together. And those racks look like one brain instead of right now it's one rack is it looks like a brain and you shard the model across what that's connected to. So the goal is to get to these compute scale up domains that are four or five, six, you know, compute pods or more, depending on how much power you have. Can you cool them all in a row? Can you network them all together? It's all sorts of engineering variables. But you want that to be, you don't want to shard the model across that you want shared context, shared compute, all of those things to serve inference at scale. And that might be multiple things, right? It might not be all from one vendor. You might have CPUs from an x86 provider or arm that aren't NVIDIA Vera. You might have NVIDIA GPUs, might have NDD GPUs. You might have all of these things. And the only way a hyperscaler or a neocloud can have fungibility in that workload is that they all work together. And I'd have no idea how they pull this off, but you talk to everybody in the cloud space and they're like, yep, this is how it works. And they're like, yep, this is how it works. And this is what we'd like to do. We're going to make software that tries to orchestrate these models because we need to do that. But it's a complex problem. So that's why I just keep landing on scale up domains in a heterogeneous world are going to require perhaps more interoperability across variants than people realize. And, you know, I think that's a much more economically viable world for those who try to monetize this infrastructure versus having islands of compute, even if it's scale up domain islands across vendors. All of this needs to work together. And that's the end goal, I think, but also extremely challenging.
Speaker 2So I, you know, I was in New York. I also, after the Marvellous Day, I also went on and did a whole series of investor meetings and shout out to everybody, to all the listeners I met with too. And a couple, a couple of things were very clear. Like, I think most, a lot of people think networking is increasingly an area of focus around the AI spend. It's people, I mean, it's not exactly news, at least to our listeners, but it's, it is certainly a topic that kept coming up again and again is, is networking is important. This is, this is where people are spending a lot of time. And it's good reason the networking domains across these data centers are becoming more important and more diverse and more complicated and technically much, much, much more interesting. I think that's, that's certainly true. The other debate that I find I'm having more is the extent to which future AI compute workloads are done by sort of general purpose GPUs, or do we start to see further sort of differentiation around the fact line, meaning, meaning, are we going to have, you know, I think it's clear we're now moving towards a realm in which we're having different chips for inference and for training. That trend is clear, but now the question is also, are we going to have different types of inference compute as well? Or is it just going to be like, you know, one GPU can do both, both sides of it. And I think a lot of people I talked to, at least here, who are on the sort of the industry side of things sort of assume that like, or at least had assumed that, you know, in the future, I'm going to have a bunch of GPUs and to your point, they're going to be fungible and I'll, I'll use them for whatever task I need. Maybe I get some extra special stuff for training, but like for the inference stuff, I just have to sort of wire enough of them together. And this comes out of the work I was doing last week around, around token pricing. I'm, I really am starting to believe that we're going to be able to do a lot of different things. We're going to have a much broader array of inference, silicon, and that you're going to want to have special types of chips, GPUs, accelerators, whatever for a grade for input versus output. Right. And this sort of fills goes into this, you know, pre-fill versus decode conversation. And, and we may have other variations come up above or you think, and you think about it, like there's now what three flavors of TPU going to, uh, four or five, right? There's two, but yes, I think there will be more over time. I think, I think there's, I think there's three now with, on, on Ironwood and then we get to generate Gen 8 and Gen 9, we're going to have more. And I think that's Google sort of picking out very specifically, you know, coupling different chips to different parts of the AI equation. And certainly with, with NVIDIA, we see that as well. We have CPX and, and Grok LPUs to do the pre-fill, and I think we're gonna see more sort of variation there as well and who knows what you know there could be more iterations as we as the software develops so all of which is to say that like it gets it gets more complicated i i think it is interesting to think through how that what that means for the competitive landscape nvidia certainly you know is is leading this and has a pretty wide array of products already and they can sort of can continue to expand their skew count to get into that i think it also opens up the door for other people as well though um and i'm not sure i was convinced of that before and i'm increasingly coming around to this idea that you're going to have more openings for competitors and i don't know how that all plays out but i think it certainly gives gives room to other other forms of compute other you know other people are going to get slurred i don't think that means that we're going to have 70 different startup xpus filling in lots of every corner but sort of i'm i'm still working through how this is going to change my the way i think of nvidia broadcom marvell amd and how they sort of mesh together here it's getting it's getting a little bit trickier to to track share yeah in a good way yeah no i
Speaker 1mean i think i've been on board with this for a while which is why i'm still working on it but i think it's going to be interesting to see what happens in the near future and i think it's going to be interesting to see what happens in the near future and i think it's going to be interesting to see what happens in the near future and i think it's going to be interesting to see what happens in the near future and again it all needs to work together in an ideal world because that's the best economic possibility for hyperscalers to have fungibility of the workload across variants of infrastructure that is not a world we have today i don't think it's one we're going to have in the next few years but i feel like that's the inevitable end state because their their economics favor it greatly from a capex as well as a tokens per dollar per watt as as their services their software is the orchestration layer of where to monetize which workloads based on based on its request um so that that becomes a you know again but just to say like we're not even in the single vendor scale-up domain yet like we're not truly as much as google has x many you know 90 whatever 9400 9600 tpus networked together those are still sharded models that's still not one model across that many racks so this is where they all need to get to is that you can load huge models with massive shared memory and storage serving as one contextual compute domain for many concurrent users and massively long context windows like that's what they're trying to get to but we are we are not there we are not even close to one vendor scale-up compute domain yet um so my other you know as you know i just nitpick on this and i continue to will orville also likes to talk about scale-up scale-out in different terms than others there's no ubiquity of my feedback again has been and just so all our listeners know like scale-up or scale out i don't care which term you use but if you add domain to the end of that sentence then we all know what you're talking about it's connecting racks together in a shared compute infrastructure so start adding domain i don't care which term you might all land in other areas but the only one that's actually clear is scale across as the only one which to be honest with you some future year should also be a domain but i realize that's going to be much harder to do um so just start saying scale up or scale out domain and we all know you're talking about connecting racks together as one shared compute contextual window uh there we go okay i do want to talk about this uh mysterious little hint that marvell launched as a new class of ai infrastructure management silicon that they did not say was a base management controller or bmc but they put it next to uh that um this is actually something i think is super interesting and we did a report uh as a part of our gpu tsunami series talking about bmcs and fpgas and the total attach that's happening on compute boards you know you look at any compute tray all the ones we're going to stare at in close proximity this week at ocp and you see a whole slew of fpgas from lattice and bmcs from a speed and um that is increasing as boards get bigger as boards need uh have more compute on that board um that's network that's true of networking and that's true of uh of of cpu and uh asic slash gpu um xpu slash gpu boards so my i've talked with this a bunch with engineers and it's kind of like you know what are you using you know your your your base management controller pairs very closely with your fpga which is why lattice and a speed have a deal together these are these are things that work in harmony um but like what's sort of the role of these things and and it does keep coming back to uh an engineering theme which is you know bmcs not necessarily fpgas because they're a bit more flexible or fungible you could call it a fungible programming gate array actually um is uh bmcs were were kind of designed for other things like they do work but they have limitations in kind of more modern high wattage high complex compute design trays and so what engineers have been doing is like they throw more fpgas at the problem to solve different things is it security is it power management is it you know x number of things and they're they're trying to solve problems in fact i brought this up with uh as i was talking about at hpe when they were talking about how hard of an engineering problem it was to design kind of this this complex compute tray with helios and they used a lot of additional microcontrollers and things that again were not necessarily custom designed to this but they made it fit their needs and i think that's the context that marvell's bringing with this a infrastructure management board is to say and a new type of silicon that solves some of these problems from management controller to um again security system performance monitoring all of those things is is necessary and they want to play in that market now i don't know if we should just come in and size the opportunity here like bmcs which depending on whoever's you know forecasts you read could be anywhere between a three and eight billion dollar market because these are not very expensive chips by 2030 um maybe it's not the same maybe it is but it's it's my my intuition is marvell's trying to solve kind of this problem that modern day you know bmcs and even even to some degree some of these things are not going to be able to solve some of the things that engineers are throwing more fpgas at to solve problems for could use something a little bit more custom made or for the modern era of these complex compute trays while we know anything about knowing anything about it that's the that's the intent i think they're coming from and i think that's interesting because it's exactly true that uh you know bmcs serve a purpose it's pretty straightforward fpgas have been increasing and attached to solve a wide variety of problems that we don't have custom silicon for which is why they're flexible and programmable and i don't think that goes away to be honest with you this is still a compliment to bmcs uh and fpgas but my point was the problem i'm addressing is why people were seeing more fpgas almost every year come to the board was because they're trying to solve a whole slew of engineering problems so i i wonder if this kind of you know where this fits but i but i understand the problem because we are trying to solve very complex engineering problems with a lot of let's just say legacy approaches to board design and management um so i'm intrigued by it and i don't know if it goes anywhere but that's the context
Speaker 2that intrigues me so stepping stepping back a second there there is a chip that has existed on a server board for years called the board management controller and it is the the there are a bunch of companies that do it different different ways there's a company called a speed in taiwan that is sort of the merchant leader in bmc and bmcs as the name kind of suggests sort of manage a lot of the low level stuff that takes place on a board so not the big compute flow it's not doing the ai math it's just like keeping track of all the electricity that's flowing around the the server board and right and you know we don't think about it that much but there are you know hundreds of sensors on on a board that measure and you know yeah and well on a traditional server board let me like that's just a traditional server board that we've used for years in the cloud we'll have things that monitor voltage and temperature and current across the board and the bmc is like the place that you can that will track that right and so for those of you who have ever messed around with a a server that's been used for years in the cloud we'll have things that monitor voltage and current across the board and so for those of you who have ever messed around with a server with firmware you you you can like turn on and off the fans or you can like get temperature readouts from across the board um that's on a traditional server, right? When we start moving to entire rack designs, you don't just have a multiplication of, you know, you don't just add them all up, like hundreds of servers, now we have that many BMCs. The task involved gets much more complicated. The modern, you know, the current generations of GPUs and accelerators can have dozens of sensors just within each chip, right? So there's a lot more data taking place and you're having to monitor and sort of track all kinds of signal across, again, not just a server, but a rack or an entire row of servers into these big memory clusters. And so the job is getting, there are more sensors and the tasks being asked of them are getting much more complicated. And it's meaningful in that it, you know, the way these things perform can have a real impact on the performance of the server. And you start talking about saving 2% of electricity with these things and across a gigawatt of capacity, it adds up very, very quickly to big dollar amounts. And so there are lots of ways that people are looking at this. You mentioned a few of them, FPGAs, microcontrollers. Marvell clearly has something here. They teased it. They don't call it a BMC. They call it AI infrastructure management, which is not a product. Which is this, no, it's not a product at all. So, yeah, I noticed that too in the presentation. I was like, I was really wishing they would tell us more, but they just kind of teased it. Right. And I think, like, I know we're getting deep in the weeds on this, but what was interesting about this event was they had similar conversations and positionings for products around like a dozen different categories that were equally in the weeds as the BMC. Just to show, again, how complicated and how why, how complicated the servers, they're becoming and how wide Marvell's opportunity is to play in a lot of these categories that, again, they've had these, they've had these kinds of capabilities for years. They were just never big enough to really merit attention. And now they're all becoming probably a hundred, a million, multi-hundred million, if not billion dollar categories. Yeah. Agreed. Okay.
Speaker 1Well, last thing I'm just going to say, and it's just because I just want, I just want to land this point. I don't think I did. I, it is, it's just, I keep coming back to, when you talk to the engineers creating this world of compute that we're talking about, which is Watts up and to the right, more chips per board, incredibly complicated networking. You just hear them constantly say like, like these are very hard engineering problems. Like we're up against just engineers have to go solve problems. And it's just interesting to me what they do. How they do that today, how they rethink logic board design, for example, how they rethink that in a world where power goes up drastically. How can you fit all of this in one board, right? And how many capacitors and MLCCs and additional silicon, like all these things that are needed, sensors, analog, whatnot, it keeps getting more complex. And it just keeps compounding engineering challenges. And so I just want to point that out, but this workload as it scales, and again, as models get bigger. And you need more compute and you need to serve more people like weren't like, these are problems that we don't have solutions for again, like this at IR labs, right? He, his whole point was, you know, the engineer mindset is, is optimistic, but also understands these are hard problems and there are sometimes not clean answers to this and that's what they have to go. They just have to go solve hard problems. And so that that's why I think these things are interesting. There's opportunities. There's opportunities for players in semis and analog and whatnot to create new products that didn't exist before, because we didn't have the needs. Like you said, a general purpose server, I think we had largely settled a general purpose server, like that was fixed. You knew how to do it. There was no new challenges. And now this is creating all these new challenges, which again, my point is creates the opportunity for new silicon innovations in both analog and, uh, and, uh, logic. So I just find that interesting all to say, there's still a lot of opportunity. We don't know where it's going, but that's why I like what they did. Maybe it'll work. Maybe it won't, but create new stuff that solves new problems. Don't try to solve new problems with legacy silicon. And I think that's a fine, fine place to be.
Speaker 2Makes sense. Although you said analog analog too. And I just want to point out analog or semi semiconductors analog semis or semis. Yes, I know. I, I got, I got to say,
Speaker 1I love, I love the analogs as well. I just, I like calling it out because it's a very different problem. Um, also again, and I think you're going to see it this week, to be honest with you, you're going to see innovation and analog because they recognize this and they are working to solve problems that didn't exist before and those roadmaps took time. Over the last two years as this became clear, and now we're out taping out new silicon. So. So fascinating times on that. And, uh, anyway, all right, let's move to, uh, open AI. Some news this week that I guess hurt, uh, hurt, hurt, hurt. Yeah. News quotes, quotes around that news. But there seems to be some confusion between the recognized revenue and annually recognized. So I'll let you explain it and then let's see where we landed.
Speaker 2So there was a story. I forget. Where I think the FT that said, uh, or somewhere someone, someone said, uh, they had gotten ahold of opening eyes, latest data and their revenue was on, was like something like $20 billion below where people expected it to be. So, all right, so let's just start off with, um, expectations. Like who's this, who, who established the expectation, uh, to $70 billion versus $50 billion. So, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, I wasn't, I don't know how much damage it did, but I know that like semis were off pretty bad the day that came out because it looked like, oh my goodness, OpenAI is, numbers are way below target, the world is ending, right? So, okay, it, one, it turns out that the, the, the revenue numbers are just off. People, like, there's different accounting treatments, you can look at gross versus net, so you, you get a bunch of revenue, but you have to pay for that revenue and traffic acquisition costs, right? It's just different, different treatments of the same, same trend, and, and I, I don't know if OpenAI came out. I've not explicitly said it, but certainly, uh, lots of other people were pointing out that no, no, no, if you actually compare like for like, it's, it, nothing has changed, right? So, there, there turned out to be no substance to this story. It was a mild, mild panic, freak out on something that lacked substance, or at least was making a bad comparison.
Speaker 1I mean, but at the, at the end of the day, you know, people were pointing out, I mean, look at what their revenue was last year, which I think was estimated to be in the 20s, and so, even if it's 50, you know, you're, you're doubling that. There's obviously a difference between what they're annualizing versus what they're recognizing. Um, but regardless, like, any numbers you have, which include, like, numbers we've been trying to do on the hyperscalers, dedicated AI spend, obviously not just Anthropic and OpenAR, and then you look at, you know, the sales forces of the world who are doing, like, we are doubling from where they were last year. I mean, you can make a strong case, and most people have these estimates, that even if we're tracking AI, we're not going to be able to do that. Um, the, the dedicated AI spend could be between 250 and 300 billion dollars this year. That's up. That's up over last year. So, it's growing. Um, and so, you know, the, we quibble on numbers. That's still a lot of money to be doubling. That's still a lot of growth rate, but I think the sensitivity is something we keep coming back to, which is people are really worried about this, and they want to make sure we're funding, right, that money's flowing, that people are paying for AI, and that's a, that's a sensitivity. And that's a sensitivity in the capital markets.
Speaker 2Yeah, and I, I think to me, that, that was the main takeaway here, because the, the numbers themselves, I don't want to argue about imaginary numbers. Like, just file, everybody just file your, file your numbers so we can see the real ones. Um, but the, the real issue is the, the fragility of the semiconductor trade. There is growing awareness that all, all of this spend is being driven by six companies, really two, and the, the two, those two are, uh, massively loss generating, and there's a lot, you know, there's an awful lot in both the equity and the debt markets, uh, riding on open AI and anthropics prospects, um, right? But I don't know that we have any reason to, to, to freak out about their numbers, but like, there is this growing awareness that the trade, the whole semi-complex is, is fairly
Speaker 1fair. Yeah. And I think, I mean, I think that the intertwined nature of it, how it is going to be a sensitivity, and I saw somebody, there was a note, I cannot remember who, who put it out, that was like, that basically said, um, Um, you know, it's not, it's not demand that crashes any of this, which, which I agree with. It's some unforeseen issue, like for example, a massive cybersecurity attack or a complete data breach, something that causes you to say, maybe we should slow our roles. It could even be that we just literally cannot energize gigawatts in an annual basis to keep both compute. Something that I'm most worried about these days, just trying to be as in the weeds we are is tracking how much is energized at what time based on what the gigawatt of compute forecasts are. Those numbers do not line up and that's just a reality of it. Now, does that ease in a 16 month to 18 month timeline consistently? I don't know, but right now what's energized annually is not matching up with what's being forecast in accelerator sales. So that's just the reality of it. Now that, that could hurt. What happens if they're like, eh, we're going to slow down until it's energized. We can't rip and replace. And. That hurts somebody's numbers. It's again, it's not because it's demand goes down. It's just infrastructure takes time. It's hard. We don't have power. Right. And that, that will hurt the trade, but it's not the demand. It's the interlocking of all of these supply chains that one or two things could slow a lot of it down. And that will trickle into people's numbers.
Speaker 2I, I, yeah, I think that's fair. I, you know, my, my sense is there's a debate about how much electricity we can add next year. We being the United States electrical grid. And I've heard estimates from 20 gigawatts to 30 gigawatts, which an absolute, right. Which is in absolute terms, a staggering number for something that's barely grown for decades. But we're going to add 20 to 30 gigawatts next year. Um, the energy people I talk to, or I don't really know many, but you know, the, the research I read on that is towards the low end. Of what we can actually do and the compute people I talk to are all bulled up. They're hoping that we could do, you know, they're, they're, they have secretly have aspirations that are above 30. Right. And, and so, yeah, that, that's not gonna, that doesn't seem likely to happen, but like here, here's the hypothetical I've been posing a lot to people lately is next year. Somebody, some big, big data center, one of the big gigawatt scale data center, hyperscaler clusters gets pushed out for six months. We can't get enough. Transformers, we can't get enough turbines, or we had some regulatory hiccup and you know, the, the, the appeals office isn't open on Tuesdays, whatever, something gets delayed for six months for some reason beyond control. And w what, what happens to semis at that point? I think if that happened today, the street would look, look through it. No one would care. Okay. Six months. Right. What, how are people going to view it a year from now? I, I don't know, maybe, maybe they won't care. Maybe they'll see, oh, six months, no big deal, but there's, there's so much complexity built into it and so much expectation built on top of it that if let's say one of the big semis company says, Hey, we were, we were going to sell, you know, a zillion units into that cluster, they don't need them right now. So we're going to push our numbers out for six months. I don't know how people react to that again today. I don't think people. I don't care. They say six months, but like, you know, because, because inevitably, inevitably we're going to have that kind of thing happen. And I think so, so long as we see open AI and anthropics numbers continuing to go up the way they've been going up to your point, there's demand that shows there's demand exists and people sort of look through it. But I think, you know, there, there is next year, I think it's going to be pretty tricky because you think about it, like back in January, February, March, all ever. what he was talking about. We're all the bottlenecks. We've stopped talking about those as much, but that's, I think, a function of the industry has now scoured the landscape. We found where all, we know where they all are. We all have schedules for them. But that doesn't mean the shortages went away. We just aren't, we've sort of thought through them. We're still going to get to a point next year where we won't have enough gas turbines, just because you look at the production schedules, they can't produce enough.
Speaker 1So I think, and then that's like, I had put out a report and we talked about it, you know, that I think next year is peak of the constraint. If you just look at what's happening in 28 and 29, it will ease some, but, but every, it took a while for everybody to build right out everything they need to build from a clean room space to start to planning for more gigawatts, like all the things that took years. So I think you're right. I think next year gets a little rough, how long that, that cascades right into 20, 28. I'm not sure, but I want to, I just want to share a general point. Cause I don't, I haven't shared this before as food, food for thought about, um, why we might give the benefit of the doubt to this cycle compared to other mega cycles. And it has everything to do with every other prior mega cycle.com, uh, railways, cars, I mean, electricity, name a transformational technology. Um, you, you had to wait for adoption for those to scale. You were basically building out in the hopes that people adopt it. And by adopting it meant in electricity, you had to have your house wired. You had to spend money or you had to buy a new house in order to go get that right. Or in the.com phase, you had to get a computer that you didn't own that costs thousands of dollars and then sign up for a plan. And so people, had to go get hardware kind of mobile was the same thing, right? And telco, you had to go get a phone in order to use the backbone. This is different. Everyone can use it right now with existing hardware. There's no adoption cycle of new costly hardware or things, widgets to drive this cycle. It can be used today. And that's why I think this is a very interesting, um, time, which compounds demand and because it's freely available for everybody to use in some capacity, you just have to start using it. And that's a very different thing. And I think that's why we keep talking about, and you hear everybody talk about demand is there. We're actually building to meet the daily use of billions of people who could use it right now, which is again, very different than any other cycle. And I just, I'm sensitive to that. This just is going to take time to build what we need. We could argue at the end of the day, the only two things that matter is how much compute is needed for 6 billion people to use it on a daily basis. And how much energy does that take? Those are really the two things. If you could figure that out, you could land on how much we need and are we close to it or not? And the reality is we're probably not, but at some point perhaps, right? We are, we've, we've built what we need to serve billions of people, agents on a daily basis. But my point is it's different because we're building to meet the daily use of billions of people. It's different because it's, it's available to use today. There's no waiting for hardware. There's waiting for infrastructure to serve this at scale.
Speaker 2That's a little concerning to me, because if you think about the internet boom bubble, a big factor there was people had big plans and grand ambitions. A lot of companies built up on the idea that we could, they could do all of this. And I think that's a big factor. And I think that's all kinds of interesting things. And the problem was the infrastructure wasn't there. Both the physical infrastructure. Well, everybody was interested in it, but like the physical infrastructure wasn't ready. I mean, it was physical infrastructure wasn't ready. So like web, web, web van, you know, is, had the right idea. They just were too early and didn't have enough warehouses. It took Amazon to deliver on that. Yeah. No, no pun intended. Maybe a pun intended. But then more, more importantly, I think the other issue was the, the bandwidth wasn't there. Right. Not enough people had high-speed internet in their home, uh, until, you know, 2004, 2005. And we definitely got to a point where, where we maxed out the infrastructure and it was insufficient. And that, and that sort of like caused a lot of problems and everyone started missing their, their numbers. And because like users just couldn't access the service in the right way. I don't know that this one, this time is going to play out like that, but it is a concern that you have, you have a lot of frustrated demand because the infrastructure isn't ready. And it's going to take a few years for the infrastructure to be ready. Do we, is there a mismatch in, you know, expectations versus.
Speaker 1Well, and I think my, my bigger concern is that in that is, um, as you start to roll this out to, you know, your employees and all of the things that, you know, government officials, whatever, whoever your customer base is, like, are they frustrated? Like you said, because it's slow or it's costly or it's, you know, X, Y, or Z does, does it not deliver the expectations or hope because they're up against, you know, I can't use it or it's slow or it's failing or it's not doing what I want. And these are functions of both the technology and, you know, perhaps its ability to serve at scale. And so I think that's, you know, parallel to, of this, but like I said, the, the, the reason that I think the benefit of the doubt exists in some capacity is just because you have potentially billions of people interested and willing to use it today and it does not require any new hardware and that's very different than every other mega cycle that required the customer to acquire something new that cost them something in order to start using it yeah but i just think of
Speaker 2sort of the basic functionality of the web things that we take for granted today you know think about like javascript right the the 1990s web didn't have javascript today it's like built into every website we don't even think about it you're able to do all kinds of things before the website is fully loaded uh you can you can execute code as sort of in on a website um like there's all kinds of little intricate little nitty-gritty details that are today a fundamental part of how we experience the web um which were not possible on dial-up modem right and so i'm just wondering like and what i'm i guess what i'm getting at is to me i still have big questions about how people are going to use ai more broadly i don't think we're quite there yet and i don't think there's a lot of functionality that there's a lot of things that are just like like you don't really need a lot of ai for a lot of the things we do today it could be much better it probably will be much better but we aren't there yet because of infrastructure reasons right and so will we get to a point where i mean certainly i think the average person at least in in the u.s is like really sick of hearing about ai because all they all they hear about are the costs the data centers and the cyber attacks sky net and and and they're not like really seeing much personal benefit from it and i'm optimistic about ai in that i think we will get many many more good things but i think but i can understand how people will be frustrated and kind of just get disillusioned this just goes
Speaker 1back though to the excellent conversation we had with our friend benedict devins who was like who kind of made some similar you know points that i'm making now but he said um it really comes down to ui like you have to make it easy for people to use and as a matter of fact i tweeted this the other day like i you know i think you and i know you're you've been a clod camper and i've always been in in gpt um but i'm not using clod a lot because to be honest with you i'm not using clod i think their interface got better for projects like now that you have one central chat agent and then it organizes your threads in your projects like for like that's great and like and and my wife is using this now for teaching because she has the clod uh teacher uh plan and it's actually a very good ui it helps it helps you your mental model use it better and so that's like the harness the thing that shows you what you can go and do and how you can go and do it and makes it easy is is the other part of this right because i while you don't need new hardware which is which is my point you do sort of have to rethink all of this and it needs to be a ui that helps you get the most out of these things that is still kind of the we're still in those stages how do you make it easy for people to do this um in fact i just noticed today with chat gpt's uh update yesterday um it is starting to pre-fill pre-fill pre-empt prompts for you based on what you were doing in that chat thread it's like a little ghost you don't have to use it but it's kind of showing you a suggestion on what you might want to say or ask next based on what you were just doing and so i think again this is just this is the ui these are the interfaces things we're going over to get not just people onboarded but to get them using it and a more regular fashion and this is the the ui battle at an enterprise and a consumer level is is still early we're early days of that so i think that's interesting too um okay yep last time yes can we do one more thing
Speaker 2all right so this week uh lisa sue from amd was in asia and she she said um i think she said to the press or maybe she was at an investor conference but she basically said uh they expect to get greater than expected capacity from tsmc next year right which is which is good news for them uh right because obviously tsmc capacity is very tight it's the key constraint on all of this and so all right that's good for amd but i also think it is a positive signal for intel in an indirect way okay this is my latest conspiracy theory right we have talked about this over the last three four earnings calls tsmc has gotten has has gotten much more serious about how they discuss intel right a year ago they were like oh intel's a great partner we love them they're so you know because intel is a big customer now in the last year lipu has come on and intel foundry has solidified considerably to the point there that tsmc now takes it much more seriously and you see that in their language today they talk about intel coldly exactly but much more third you know sort of more distant right yes intel is a partner not there's no fondness there and i think tsmc is now i wouldn't say worried but recognizes that intel foundry has gotten to the point where they are technically very capable of competing with tsmc long term both in terms of emib and 14a they have i would say on on page paper at least some some clear competitive advantages over co-ops and a14 from tsmc right tsmc sees as a competitor now there but they probably recognize or they probably are thinking that the biggest constraint on intel becoming a more serious competitor to tsmc is intel's ability to raise cash and so wouldn't it be nice if intel suddenly had a harder time generating cash flow from cpus because it was seeing a lot more competition from amd because you know there's huge demand for cpus and if the constraint on cpu demand is is tsmc capacity i think tsmc is giving that extra capacity to amd to do what it can to throttle intel's cash flow generation they know they can't stop it but maybe they can slow it down okay so so i guess i have one question
Speaker 1that i'm going to add to this conspiracy theory um where are those wafers coming from it's a good question but i mean
Speaker 2there's lots of lots of potential people probably i mean for one thing intel right intel still is going to do a lot of a lot of wafers at tsmc next year they're not they're not getting the upside right tsmc is obviously adding capacity at a pretty fast clip tsmc is now making those decisions about who gets that capacity and intel's not gonna get um okay
Speaker 1second to this theory is um almost everybody who tracks this space us included even though i've been saying this year for next year for pcs and the year after is going to be absolutely horrendous um has been upping their estimates of how bad this is going to be off uh idc i think now is at like 20 for next year um that frees up wafers and what's intriguing to me is to this intel point like intel actually this is gonna sound so ironic and i and i know i'm possibly stretching but let's just again throw this out as a as a um as a hypothetical uh in no other prior cycle had intel even a down pc cycle would have been awful for intel like not nail in the coffin but a wayfaring example of that is when you have a fountry and you have wafers that you need that you're happy to shift to a higher asp server cpu perhaps for the first time ever abysmal pc sales is actually a very good thing for intel in two entirely different scenarios and i just think that's insane to think about but no joke pcs are going to be really bad the next two years and that frees up wafers and to the end of the day i think that's going to be a really bad amd point amd actually did have a lot of wafers continually for pcs so if they allocate some of that also to server and uh server cpus and um helios asics um that could explain some of this too um but it's interesting to think about but there will be a wafer spraying up because people are gonna really take back what they're gonna make in pcs think this market's gonna be pretty terrible yeah i i think
Speaker 2we're already seeing that um in other in other sectors where you have companies that have excess uh excess tsmc allocation for this year even who are not selling as many of their traditional end products and are repurposing that for you know say custom custom asics for one of the hyperscalers i think we're seeing that absolutely we're repurposing tsmc for something beyond core business i think yeah okay well
Speaker 1that's our time everybody hope to see folks uh this week i know we will have a very busy podcast next covering a ton of ground from this week we appreciate your time and thanks for listening we will talk to you later
Speaker 2hit us up this week we'll you know love to meet you all leave us a review tell your agents to leave a review and subscribe thank you for listening
PodcastTranscript AI