Speaker 1Hello, everyone. Welcome to another episode of The Circuit. I am Ben Beharin.
Speaker 2Greetings, programs. I'm Jay Goldberg, and we're joined today by all sorts of avian noises. Ben's chickens. My cockatiel.
Speaker 1I told my rooster it needs to be quiet during the podcast, but it is a disobedient rooster. All right, let's kick this off with the earnings of the week, which was Micron. And a couple of things I think are interesting to talk about. There is certainly no shortage of opinions on memory. So one, let's talk about the earnings. And then two. Does anything that came out on this change, like the thesis for memory? Because I backed into this, and before we talk about the earnings, but I backed into this. And essentially, while I think everyone agrees that the dollar value of memory has gone up, right? So if you've looked historically, it was a peak. Ebb didn't float as memory did, but as a peak, it might have gotten to $100 billion. And now people are saying, right, our model included. You could be at between a $600 or $800 billion, depending on ASP, floor now. So the floor has changed, right? Bit growth glows. The floor has changed. Nobody disagrees with that. However, it seems like the way the market is valuing these is that they're assuming that crash comes in 29. Either that that cyclicality value goes up and down between that new floor. But that essentially the wheels come off the train, like not that it's as it is an industry that needs to go up into the right, that's fine, right? But if bit growth continues, and pricing degree leverage continues to some degree, and you have this new floor, that's where I sort of think the exercise is, okay, like, are we thinking about, even in some degree of cyclicality, the right kind of forward assumptions here for micron and some of these other names? You used the C word, crash. I mean, but that is what's.
Speaker 2It's called a crash in 29, everybody.
Speaker 1But that is what's being modeled. Like, what's being modeled is demand destruction and a high degree of margin compression, right? Which could, again, if they normalize to the 60s, great. Like, we can run those scenarios. But I think the reality is it's not being the sustainability. The sustainability of bit growth and some degree of pricing elasticity is certainly not in kind of the long-term modeling.
Speaker 2I think, yeah, I mean, there's all kinds of arguments people are making right now about where memory goes. Like, I think your scenario calling for a decline, or I understand you're not really calling for that, but, like, that's the scenario you laid out, is which this keeps going. It's into 2029, and then conditions worsen then, whether they crash or not is a whole other story. But through 29, this keeps going through 29, and then we have some kind of, you know, beyond that, it seems it's too hard to forecast, and we'll probably have some kind of reckoning. There are people who are even more bullish who say that, no, memory's fully structurally changed now. We have all these long-term agreements in place, and that's going to. We don't have the same whipsaw cycle, boom and bust. It's going to be. We'll be cyclical, but it'll be much more muted cyclicality. I think that's the sort of very, you know, most optimistic outlook. And then there are other people who are saying, what we're seeing now is, with the memory results, are being driven by pricing, and output, bit growth, is much slower, right? I think the company said DRAM and NAND are growing sort of in the 20s. Low 20s for. DRAM and mid-20s for NAND into 27, 28. And so I think the bear case would be, we get to supply-demand equilibrium in 28, the market sees that and starts forecasting, you know, starts getting much more pessimistic in sort of some point next year. And I'm not taking a stand here. I don't know what's going to happen. But I do think it's notable that. Prices doubled, right? Prices are basically doubling this year. And it sounds like there are agreements in place that pricing goes up again next year. But output is only in the 20s. And set against that, you also have all the memory makers adding significant amounts of capacity. So I think we get to some parity earlier than that. But how that actually plays out in the shape of that downward curve when we get there. Is very unclear right now. Right? You know, I think past cycles, everyone assumes that, like, it's memory cyclical and it goes for two, three years and it crashes and we go from 80% gross margins to whatever, 30% gross margins. And I think there's a reasonable argument people could make that, no, that's not how it will play out. There's lots of things that will smooth it. And I think, you know, I think for many people, what matters most is the cost of memory. It's just like everything is getting so expensive, right? They were talking about because of agentic AI, which is driving CPU demand, that's driving demand for, you know, not the fancy HBM memory, but sort of cheaper LPDDR memory and other stuff that goes into CPUs. So even that category is getting squeezed. I mean, ultimately, to me, this comes down to how long it takes them to get capacity online. And when we start to. You know, build up a lot of capacity, possibly overbuild capacity. And I think that comes sooner, but not soon. Right? And then we haven't even talked about, like, where China fits in on this, right? Because we also had lots of news reports coming out of China. CXMT is pulling forward all its capacity targets. They're going to have a lot more capacity next year. They're building fabs very quickly. I think that's the thing that people miss most about Chinese memory. So much that they have. You know, they're as constrained as everybody else, but they are able to put capacity online much faster. It takes two and a half, three years to build a memory fab in the U.S., Korea, or Japan. It takes 18 months in China. And so how does that play into it? I think that leads to. We're going to have a lot of memory in 2028 is, I think, where we get to. And how that affects pricing is the big question. I don't know the answer.
Speaker 1Yeah, I mean, and I think there's two parts of this also. And we should talk about the Micron sort of specific report as well. But there's pin-compatible dynamics. And I think if you just look at where pin-compatible standardized industry memory maintains segments, right, which is going to be phones, PCs, consumer electronics, maybe to some degree accelerator attached, like if it's SOCAM or whatnot, and you can just put. Anybody's DDR right next to your CPUs, fine, right? That's not a design-in dynamic. That's going to continue wherever you believe that is, how big you believe that is, what segments you believe that standardized industry pin-compatible dynamic is relative to, is going to remain price-sensitive, price-competitive, right? Whoever has the best quality, so speed, reliability, all of those things, plus the ability to go pricing, you can just mix and choose your vendors, right? So that part, again, however big you believe that market is, like we have assumptions on that, that's going to vary, right? That part could either be cyclical and or that part could have ups and downs in pricing and margin because it's more competitive, right? The other side where you're designed in, you're specced in, you're in the advanced packaging parts of this, so HBM-ish. And then I think there's. Because the other point I make here is like memory innovation is not stopping. Like what we have today in memory around HBM or, you know, just again, let's just say DDR and the whole thing, like that's not slowing down. Like there's memory innovation coming that will change this as well, that will new products, new process technology, new transistor designs, new stacking, all of these things will come. So there'll be a segment that maintains its value, right? That I believe. Again, but then you got to come back and say, okay. How big is that, right? And what are your assumptions? But I think that's the part where you, you know, meet those in the middle, right? I would not expect margins to go back into the thirties. And again, right? This is an industry that has historically been single digit average to low double digit average in margins, right? So even if you're at 30, you're more than two X where they were. But I think there is an assumption around like, okay, could it normalize? 60% range, you know, mid sixties, low seventies, something like that. When you do have the most expensive bits. remaining the most constrained tied to massive accelerator shipments um and tried tied to trying to solve massive memory pooling and i think the other interesting element is you're going to want to keep again assuming that bit growth goes up by demand at just a high level right because you're right big growth has slowed now but it's because we just can't make as much like big growth would be almost infinite if you talk to anybody who makes logic around ai accelerators they just want us it's it's memory hungry um all of that right means bit growth continues to grow so then again right premium bits commodity bits i think is a healthy debate a part of that as you think about where that attaches to but if but if bit growth continues to go up why would the you know their your memory makers do what they did in the past which was just you know do what they did in the past which was just take advantage of it and do what they did in the past which was just take advantage of it and do what they did in the past which was just old green you know clean room space and replace it with new ones keep that clean room space keep making ddr4 if that's going to work for cxl and you can build a whole bunch of disaggregated cpu you know solutions keep making ddr4 keep selling it and they are they're doing that right so there's not a take legacy memory offline like it was in past cycles and because they're still going to need that bit growth everybody can still use that memory in some capacity like not all of it needs the premium hbm5 or ddr6 or or whatever like that so that's part of the capacity expansion and again why i say the category raises in tam like the tam is growing for memory where it settles is then the question of where you then balance right those new revenues for the main players within some ebbs and flows which is going to happen right in certain pockets maybe not all um but that's why i think it's interesting when they talk about capacity coming online what's going on with older generations is that it it's it's not it's a little bit more like tsmc right you still monetize legacy legacy for a long period of time that's never been a dynamic for memory and i still think the demand will necessitate some degree of later generation memory fabs keeping going because there will be customers for that memory
Speaker 2i i get that i would like more more signal from the memory makers that they hear that um because in all of what you described there's an opportunity cost like i right you're you're you're saying that they're going to keep making legacy older versions of memory i'm i'm not confident of that right i i you know i would you know we talked about this a little bit last week there are some reports that like memory prices for consumer memory have have plateaued which i would read as a deliberate decision by the memory makers but i'm not sure that's true that's just my interpretation of it and i didn't get clear any confidence in that from from the micron call and so i i worry that they're adding all this new capacity they're investing all this money in r&d they have to have this very strong temptation to dedicate that capacity and that r&d to the to the fancy memory at the you know i i would just like some relief for the consumer side of things right um right we talked about this a couple weeks ago with hot chips everyone was complaining like don't don't make it higher just make it just make more of it right there are there are real trade-offs right you start making you know 16 high hbm memory the the effective yield on that the amount of wafers that it takes to produce that is significant and it all comes at the expense of other categories of memory as long as they're constrained and right i mean we had this this this news this week that uh tesla is despecking memory on i think for for for its robots maybe for the cars too but i think certainly for the robots like they're they're despecking memory they're using less memory in the robots um largely because they just can't get enough memory and so i i i i worry about the incentives here for the memory makers and how how much that leaves for everybody else everybody else who's not using memory makers and so i i i worry about the incentives here for the memory makers and so i i i worry about the incentives here for the memory makers and so i i i worry about the incentives here for the memory makers and so i i i worry about the incentives here for the memory makers expensive fancy hbm
Speaker 1well and that's why i say like again does every does every new product or even you know consumer demand like you said like the like the the most cutting edge memory like why could it not keep using if you've got a billion smartphones a year and you know whatever 300 million just shy of 300 million pcs again granted if if if pcs are going to start running a whole lot more ai then i get you might need you know some degree of leading edge right ddr um probably not fun like i don't know how much you're going to run on phone someday but let's assume that long that that's like that legacy memory right still works for some of those scenarios and then i could keep those prices down right it doesn't need what goes on in leading a charge but but my point is i mean ddr4 is a good example it is staying in the market right ddr4 is still going to be sold there's people who are going to use it that that that need it and that will help normalize once they build new factories now again how long would you keep this on is going to be the question but but but but my point is i think there's logic to keeping at least a generation or two going in older fabs but but my point is this is not the way memory worked before memory was clean room reuse and so if that is the argument for some degree of structural change to memory again a little bit more like like i said logic where trailing edge should we even say trailing edge memory i don't know like is that a term we're just going to invent yeah but you know what i mean but something like that stays monetized for longer periods of time and like i said i think you can make an argument that that's a good use of their clean room space when they're dealing with this kind of capacity constraint and you know to some degree of a much larger tam because that's reality like the tam is larger it's going to be bigger how big it gets is an unsettled question but it's a structurally larger market for memory and that's not changing like the memory demand doesn't go away it might normalize and be somewhat static at some period in the future but it's not going away and that's the difference between how big this market has gotten and now we just talk about how do you price it what gets priced in what's your asps where do you have premium leverage etc right so that's again that that's why i like the the main report i did this week was called bet on bits not history because bit growth demand is inevitably going up which just means you've got to be able to manufacture that in a bit growth scenario up up and to the right everything else in between price vendor share etc is is negotiable i i i take
Speaker 2your point i just i i worry that you that the the memory makers i don't i'm not convinced the memory makers have internalized what you've just said right the idea that they're going to now run a tsmc model and maintain trailing edge capacity um when it takes you know let's let's call it i don't know four you for every four sort of trailing edge memory you get one hbm leading edge memory i actually i think that's the big difference here is if you think about in in like traditional moore's law leading edge is denser more capacity than trailing edge and here we're kind of talking about the reverse because the really expensive leading edge hbm stuff is is less dense on a per wafer basis right then like a like a node shrink than the training it says right it's it's not yeah it's a reverse node shrink you're actually dedicating more capacity to right something that is less you know less productive i i'm you know doing the math wrong but you take my point yeah and i don't i you know i it certainly makes a lot of sense for them to maintain that trailing edge capacity to keep everything stable and sort of the long-term view of it but right when you know let's say it takes four the capacity for four consumer chips produces one leading edge hbm chip and they charge a hundred times more for that leading edge chip the the math works very heavily in the favor of going just throwing everything they can at hbm that's a very
Speaker 1strong temptation right if again if you are up against some degree of of clean room capacity constraints right if you have more options like if you're sitting here looking at your your your land you know space and you're like look i've got all of this capacity and how do i then allocate right now that i have more my point is you can make different decisions if let's just say and i'm not saying this is generally the case but everyone doubles memory capacity right in clean room space you essentially could say okay where then is the best way for me to normalize this meet my demand support bit growth because because again you're right like leading edge is going to give you more capacity like whenever they move to the next transistor designs it's going to give you more capacity within the same space so you'll have those factories for that my point is if you're you can make different decisions than you did in the past about how you balance both your bit growth and then who the customers are sustained Assuming sustained demand for, let's just say, something like trailing edge memory.
Speaker 2I take your point, but I also think that there is real damage being done on the demand side for non-data center memory right now. If you look at the – I was talking to somebody this week who has been forecasting mobile phone growth, and they think we're in for a decade of no growth in mobile phones. Because if you think about phones now, the upgrade cycles are globally with four years, and they're going to push out. Because this year, demand is down. Phones are getting more expensive. They're less good. Nobody's going to buy a phone now. They'll get to the point next year where their five-year-old COVID phone is completely unusable. They'll upgrade then. But then it's going to be another five years before they buy a phone. All of this is lengthening, structurally lengthening the PC mobile. It's going to be a global phone upgrade cycle and something that will have lasting impact going into the future unless we get to a point where everybody upgrades because they get AI phones actually mean something someday. But there's real potential there for structural shift in the demand side as well.
Speaker 1Okay. We've spent a lot of time on memory. I do want to just make sure. Was there anything specific to Micron we wanted to highlight other than the fact it was a great quarter? Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. They did everything they could to say the business is structurally changed. We have long-term conversations with our customers. We've got SEAs that are taking pay. And so it's not the LTAs of your father's past. Anything specific Micron-wise takes?
Speaker 2I would say number was good. Quarter was – guide was good. Stock is down a little bit for the week. So I think it was up right after results. But it kind of tapered into Friday.
Speaker 1I mean it was flat-ish. I mean a couple percent which I just would consider flat.
Speaker 2Yeah. Let's call it flat. The market is like, yeah, okay.
Speaker 1So. Yeah. The market is still debating everything that we just debated for the past 20 minutes. And literally until that changes, until your assumption of what happens, none of this is going to change. I don't care what the memory makers do. It's not going to change. They have to reconvince people and whatnot. So, okay. A couple of other events this week. Let's just say investor events. I was at HPE's networking investor event. And I think I remember bringing this up when I was at HPE's big event, vendor event in the fall. Their acquisition of Juniper has essentially had them – I don't want to say – I don't want to say pivot, but a drastically increased focus on networking as a part of their company strategy. In fact, I had several executives that I talked to at that event basically say you should think of us now as a networking company, which to me just meant like we lead with networking. Networking is the hook. It's the conversation with customers that we have that then pulls the rest of the businesses through. What we build in racks, what we build in infrastructure for neoclouds or on permanent. And that's fine. I understand what they're saying. In fact, Dell would similarly tell you they're a storage company and storage is the hook that brings in infrastructure and software. I get it. Fine. What was interesting though was the time that they spent just talking about the overall demand for let's just say enterprises. Needing to completely redo their networks. Like this isn't even just a point of hyperscalers and neoclouds, which I think is a good conversation itself around how all of this gets fabriced and switched and clustered and whatnot. But just that like enterprises themselves and what they said though, and I think was super interesting, was it's security vulnerabilities in the age of agentic, which we've talked about before in cybersecurity, is what's frequent. So, you know, I think that's a really good point. I think that's a really good point. the hps event um are you just an engineering company at the end of the day like is your value that you solve you just solve these problems with great engineering because because so i was in this in fact this was a story i've been telling twitter you'll appreciate this so i was in like the helios tray is there and i was there and eric woodring was there from morgan stanley um a couple of the support for ubs and whatnot and so we were all so i was asking supremely complex technical questions and then they were asking financial questions because they're trying to get at like how much should we think of amd's dollar value like here and and across the board but the thing that kept coming back to like the every investor i listened to like i just stood there and listened for a while who hadn't been a part of this conversation come back and be like so what exactly did you do here what is what is hpe's unique ip here and that was a very uncomfortable conversation because it's broadcom switches somebody else's copper it's you know i'm a people person i take the i take the paper to the engineers you know so that so that was an entertaining one um but anyway all of that just to say that was the hpe discussion um but they did increase their guidance right per year so they are growing but i did again i am i'm not saying that i hate them leading with networking i'm just saying it's it's an interesting pivot from what we would have talked about hpe last year versus this year now you know post juniper and and what they're leading with and you know good bad or indifferent that is the tact they are taking all right i think we've maxed out
Speaker 2our hpe quota for the year okay we've talked about it as much as we need to talk about it for the rest of the year
Speaker 3okay that's i mean that's that's fine it but infrastructure infrastructure all right uh balance sheets as a service i don't mean to cut you off i don't mean to cut you off you're good
Speaker 2i'm trying to restrain myself so uh yeah balance
Speaker 1sheets as a service has entered the equation again we know that uh we've talked about this with nvidia in fact nvidia had a uh non-deal road show where i think they hit at least six of the the big sell side banks maybe more uh there was a deck that got passed around for anybody who tracks this and i i want to say this is the first time it showed up on a slide like balance like we will use our balance sheet at least as a part of this deck obviously they talked about it but now it's like a formal part of of the presentation um but then also broadcom announced this week they're going to help anthropic uh fund some of these things so i know there's a ton of negativity viewed on this and and you don't think any of this is a great a great sign um well i get i get the yeah anyway go ahead rant on balance sheets
Speaker 2so i think i think it it's it's complicated it's nuanced it's not complicated nuanced right i i saw that uh in the r deck that uh nvidia passed around to investors this week and i found it very um humorous that they do have a slide talking about their balance sheet and the way that they're using their balance sheet to fund growth of their of the ecosystem and i and this the slide is basically just a picture of their of the footnotes of their 10k that show all their obligations right and uh i have seen people use that exact same slide that exact same image in a in a negative context to say look at how much this has grown look at how big these obligations are getting and here's here's nvidia saying look look how much we're look how much our obligations are growing their brackets are growing and they're growing and they're growing and they're growing about it it's like it's like that meme from uh the big short like why are they why are they confessing they're not confessing they're bragging right it's so it's it's like this is like peak peak jensen like he's like oh you're complaining about this i'm going to show you why it's fantastic so i i think i mean so i'm obviously somewhat cautious about nvidia's ambitions here they're spending so much money they're putting so much into the balance sheet they're they have a legitimate counter to that which is that no we're investing in growth we're investing in the ecosystem um we're you know we're we're enabling this huge layer of customers and i i think that there is truth in that up to a point and it's very it's going to be very very hard to know like how much is too much and so i i'm not going to say like this is a terrible thing that they're investing in all these things it's it's not circular financing it's it's vendor financing um i i think people should look at this and just go this market is incredibly competitive uh and capital constrained along with everything else constrained and then you we can have a legitimate conversation about like how much is the right amount for nvidia to spend here um um because up to a point it's it's going to help them but it's it's just like it every every you know every every step ratchets up the risk
Speaker 1all right so so a couple i have a couple questions um well obviously and we made the point so broadcom's doing this too sorry so throw throw that into the
Speaker 2context um so yeah so so so yeah so anthropics uh reuters got a hold of a political group that's been doing this for a long time and they're doing this for a long time and they're doing this for a long time and they're doing this for a long time and they're doing this for a long time and they're doing this for a long time and one of the really interesting things in it was that there's a section in which Broadcom has committed to loan Anthropic up to $40 billion, $42 billion. And this is around sort of the construction of TPUs, right? Google TPUs, licensed, produced through Broadcom, going to Anthropic, and Broadcom is now putting up a fairly large amount of money for that, following in NVIDIA's pattern.
Speaker 1Yeah, okay. So I say that to say what you then have is your two undisputedly large AI infrastructure players in both categories, Merchant Accelerator and XPU, who are merchant fabless semiconductor companies. And they're the ones making these decisions. So I'm curious, I mean, and I have a theory, but I'm posing the question, why does it make more sense? Okay, so again, knowing that this, your customers, this is a high fixed cost scenario, that is the semiconductor industry, high fixed costs. Why does it make more sense that your fabless companies here who have those customers or want to sell to those customers do this instead of your foundry, TSMC or Intel, for example? You mean, you think TSMC should be putting out its foundry too? Somebody, I'm saying somebody is looking at this saying, the demand is inevitable. I want you to make these things with us. You need help because it's expensive. I can monetize that in a decent way. Why not the foundries? Now, again, that's a high fixed cost business. I get it. But I'm saying like, but the point is there's a distinction here. These are fabless. Fabless merchant companies or custom who are doing this. Why, why, why, why, why are they, or why is it the right approach for them to do this versus the foundries where this all ends up anyway? And they know that demand and they get additional money.
Speaker 2So TSMC is going to spend what? 50 something billion dollars in CapEx this year? Probably 60. It's probably going up. Yep. So they're going to spend $60 billion this year in CapEx. Yep. Because they want them to take on more risk.
Speaker 1Okay. But again, their margins, what are they going to, what are they, they're, they're trying to raise margins, but they want to stay low. So my point is, my point is if it, if it has to be done, if you know that demand is there and you're going to make money on it, you're going to make a return on that. So I don't, I don't think, I
Speaker 2don't think TSMC doesn't know that, right? TSMC is too far up the chain to really, really know true end demand. Right. And this is, this is what. trips up foundries for for years is they don't have a good forecast of how many phones everyone are going to buy this year their phone they're you know they come to people like us to get those estimates right and and yeah right i get it they they don't they don't have a great forecast there they're not touching they're not talking to the end customer every day um you know it's it's like it's in the news when someone from google or sam altman goes to taiwan to talk to tsmc that's it's rare enough that it's in the news um and so how how they don't have a good idea of how much the end demand is and i would also i think it's a pretty good argument that nvidia is and broadcom are better at monetizing ai demand if you look at the end they have higher well sort of higher gross margins than tsmc does right so there's a lot more a lot more cash coming off nvidia every quarter than tc i don't know that but i you take my point like there's a lot of the value is going to nvidia and the chip makers rather than chip designers rather than tsmc so you know and i think if i'm tsmc it's like you know to me it's like an investor who spent his whole career investing in you know auto the auto industry and now is being asked to make you know for his personal account is investing in internet stocks smart smart person knows how to invest but doesn't really understand the sector um and i think that's that's what tsmc is is is you know i i don't think they're gonna get too involved they're pretty conservative too like they don't even invest they don't have like like a venture arm like to me that would make a ton of sense that's where they should be investing is like investing in young promising companies right because that's going to feed their
Speaker 1demand long term yeah okay all right so i'm getting i'm getting somewhere we're going down a path i promise okay do you remember a long time ago we talked about uh this was before i'm going dude this was like year one of the circuit we talked about broadcom and how hawk operates this company and do you remember how you described that he runs broadcom yeah it's he runs it like a private equity fund okay so if you know that who who then do you think has the best risk reward analysis here for these investments uh yeah broadcom okay so then at least in that scenario why would not someone like tsmc then just share some of that risk it's gonna get made there anyway
Speaker 2um because that entails a high a high degree of trust in whoever that intermediary is not saying that hoctan is not trustworthy but uh it's it's a it's a third party you have a that third party risk right it's a third party risk there right and he said like tsmc doesn't understand the end markets they can't they're far away from it what they do understand is the risks involved in their own supply chain in their own production what they do every day is incredibly incredibly risky and they understand that risk and they know how to manage it they don't know how to manage a large investment portfolio or a large strategic venture portfolio right and so they're going to have to trust somebody else like i think if i were a tsmc shareholder i'd be like why are you you know giving all this money to to to hawk or whoever sure when you should be
Speaker 1just build another fab which they will right and they are they're going to increase capex like i'm certain of this okay okay all right last question on this then does broadcom say they do this with anthropic and let's say they do this with more companies and they start to use balance sheet as a service more aggressively the same way that nvidia does does that knowing what we know about how hawk does these these does that give us any more confidence in the long-term structuralness of this build out maybe
Speaker 2you're asking the wrong person for for structural confidence in the ai industry but i'm
Speaker 1just saying like we know we if he we know his mindset we know how he's very good at making money running broadcom like you said like a pe firm and so for them to be willing to do this and perhaps be more aggressive i i just i take as an additional vote of confidence to the structural degree of this build out i don't think anybody knows where
Speaker 2this is going right i don't think anyone knows where this is going like yes yes ai is going to be very important and it's going to do all kinds of incredible things someday but how the economics of that actually ends up is still super unclear and i guess i guess you could say the risk is you spend all this money you're going to invest in anthropic or you're going to invest in open ai is it only going to be those two are there other other labs that might come out of nowhere that will capture the value here i think that's still possible right maybe it's a new lab maybe it's a new business model maybe it's a new product like it's it's also new um that there's there's a lot of risk involved right
Speaker 1yeah there's a lot of uncertainty agreed i would just my main point was i think it's significant that hawk tan has entered the equation with balance sheet as a service and i think that's a sign of more to come um and now that is a dynamic that is the competitive dynamic of the day so i i
Speaker 2look at that slightly differently and i think it's very much uh nvidia is driving driving the train driving the bus i guess and they have been setting the competitive terms for ai data centers silicon for several years now and they keep pushing and pushing and pushing right they've they they added networking they added server design right they added data center design they have all kinds of software right and you see amd is following suit they've they've they've greatly ramped up their software efforts they acquired zt systems to do server design and they're following suit everywhere they can and the same thing for broadcom they have the networking to begin with they're they they don't they sort of don't do server design for a reason but like um but they're using balance sheet like nvidia is sort of forcing everybody to do more and more and more in order to compete with nvidia right yeah and that's how i see this right and um and and broadcom can afford to do more
Speaker 3and more and more in
Speaker 2order to compete with nvidia right yeah and that's how i see this right and um and and and broadcom can afford it they have i don't know how much cash they have in their balance they have a lot of cash they have a lot of debt too though because they've done all those acquisitions but they're generating a lot of cash um but their balance sheet is is less less room than nvidia's um amd has a much smaller balance sheet much smaller cash flow stream so they're they we know that they're providing backstops in a number of data centers but i think it's getting increasingly difficult for them to compete because they don't have a balance sheet right they have i think 10 billion in cash and tiny little bit of debt nvidia has 60 billion in cash and i what their debt is is a complicated discussion now but not a lot of formal debt right now and then video is
Speaker 1generating they're generating it's growing and it is
Speaker 2growing and i think free cash i was just looking it up yeah i mean nvidia is generating 20 times as much cash per quarter than amd is yeah right so so amd can can cannot compete when it comes to balance sheet yeah even broadcom is is going to struggle to compete here against nvidia but in broadcom does have the advantage that it could call on deep pools of of private equity capital
Speaker 1to step in right okay uh all of that to say this is it's this is one of the most interesting competitive dynamics that has entered the equation balance sheet as a service um it's at some point might be worth like i don't know a whole episode on this and the strategic implications and beneficiaries but it is one of the newest dynamics that i think is extremely fascinating um so more on that but i think what we've established is it's not going away uh perhaps more players may may enter the equation or broadcom will do more but it is now a key part of i think how you analyze who's got longevity and uh and and the bar of competition and how it's changed so i think
Speaker 2if we did an episode on data center finance we would have to title it running into the street screaming with my hair on fire in sheer abject terror because right because i've been covering semis for a long time as have you and like if you if you told me five years ago the the center the central competitive dynamic in semiconductors was use of balance sheet and taking on debt i would have i wouldn't have believed you i would have because like for years like semiconductor companies didn't take on debt for good reason it's cyclical and debt is very very constrained and it's a high fixed cost business and it's a high fixed cost business right and if you miss a cycle in semis if you miss a product and you miss the cycle that's really bad if you miss a cycle and you can't afford r d because you have debt payments and so you can't invest in the next product cycle that's that's terminal right and and so this industry in general has been very very cautious about taking on debt and obviously today's semiconductor companies are very different than they were 10 years ago they have a lot of cash and a lot of balance sheet um but you know i think that's a big part of the reason why i think that's a big part of the reason why i think that's a big part of the reason why i think that's a big part of the reason why i think that's a big part of the reason why i think that's a big part of the reason why i think that's a big part of the
Speaker 1reason why i think that's a big part of the reason why i think that's a big part of the reason why i think to dig in tomorrow. All right. Let's talk tokenomics.
Speaker 2Tokenomics. So one of the things that has frustrated me for a long time, and long-time listeners will know this, is I get really frustrated by the way that when we talk about AI, even like seasoned industry professionals talk about AI and different features of software and silicon. Everybody gets a little bit hand-wavy, right? Remember earlier in the year, the hot thing was reasoning models. Everyone was talking, oh, reasoning models, this is going to change everything. Did it? Or even just like tokens. Everybody's talking about tokens all the time. But what is a token? What is it worth? And it frustrates me that it's all so imprecise and sort of qualitative conversations. And it occurs to me that if you look at all the data. The data that's now available out there, you can construct an analysis that lets you piece it all together and actually quantify a lot of these things, right? Because you can look at token prices, and you can look at token prices for reasoning models, for example, and realize that like a year ago, the labs were charging a premium for reasoning models, and that that premium has gone away. It's not a differentiated feature. It's sort of bundled into the frontier models. It's just a feature. It wasn't really worth much. As opposed to if you look at input tokens versus output tokens, they have very, very different pricing structures, because the nature of compute that underlies it is very different and very different cost structures. And you can sort of piece together the ways that that plays out into the end customers, the AI labs, economics, and then you can work backwards and you figure out, oh, this is why things like NVIDIA buying Grok for $20 billion make a certain kind of sense. And I think there is a way now to quantify that economics of AI, it's possible now. And it is something that I think everyone's going to be doing in a few months. It's an important form of analysis to really just be able to talk about these things in like hard dollar terms and not just say, oh, I'm going to have this feature in my model, and it's going to make everything faster and better. No, it's going to reduce your cost of DQ. It's going to decode by 20%. And that means you can price accordingly.
Speaker 1Well, and I think that highlights the OpenAI's Dev Day this week where they announced, I think the brand's ultra fast for Astra, the higher tier, which is I think 250 tokens per second, which then very quickly was like, is this Cerebrus? Because I'm not sure because they showed a much higher token model. Yes, it was on Cerebrus. And then it came out afterwards that it is a low batch optimized version of NVIDIA GPUs that are actually driving that. But two things. One, impressive that they could optimize NVIDIA GPUs to get a higher tier without Grok or any specialized premium accelerator. I get that it's not 750 tokens per second, but it's still more than once. And that OpenAI can and probably will charge more for that. So I would add. I would add to your tokenomics point that there's the base level of that I agree with you with. And then the kind of greenfield model opportunity or for what you would consider premium tokens or premium pricing and how big of a market that could be, which OpenAI is trying to justify, right, with a higher tier.
Speaker 2Yes. Yeah. I'm not quite there yet in my analysis. I've put a lot of these numbers together. And I think there's a lot of interesting things that are popping out about it. I'm still not totally comfortable with the idea of premium tokens. I get where people are going with this, but I think there's a better way to think about it. Tokens will be price discriminated by segment and end user in some way. But I think it's going to be a little bit more nuanced than just saying premium tokens and cheap tokens. But it's very interesting to think through this because I imagine that Anthropic and OpenAI at this point have economists on staff and are sort of thinking through pricing decisions. Or I don't know, maybe they're just throwing it into their own models. But there's a lot of nuance there. And I think it'll become especially interesting when or if the Frontier Labs go public. So to really understand their economics, you can get a fairly good insight just using data that's out there today.
Speaker 1No, I agree. But I do think this is an interesting beginning of the fragmentation of the market where you might see pricing tiers that are both speed-based and or possibly specific model-based. Like if models fragment and a good example I've used is like I think someone, it's inevitable that someone makes a dedicated, frontier cybersecurity model. That's not something everybody needs, but that's something that a lot of people who do need it will pay a lot of money for. And so there's the, and I would also argue it's a very different type of token. So that is a price slash model. So it's not just pay OpenAI and Anthropic, get X, Y, and Z. It's which one do you need? Oh, I need cybersecurity or I want the best engineering model. And you start to see those choices with economics applied to them, I think is a, it's a very real, real possibility. Yeah, but that's a product, but
Speaker 2that's a cybersecurity product. I don't, I don't think people end up, I mean, I'm quibbling over wording here, right? That's a product. It's a cybersecurity product. People will pay for a cybersecurity product. I don't think we're going to get to a place where there's like 20 flavors of tokens. Here's a fast token. Here's a, here's a voice token. Here's a security token, right?
Speaker 1Yeah, I think that's fair. Yeah. Yeah. You pay for the model. The underlying implications could be one of those things, but I want it fast. Fine. You're not lying. Like if you're not thinking premium token, you're just thinking, okay, I'm going to use a lot of these and I need it to happen quickly. That's valuable to me. So I'm willing to pay a little bit more.
Speaker 2Yeah. Yeah. I think that's the other, the other side of this too, is like, I'm sure the AI labs are pricing things appropriately. We now need on the other side of it, I think not just investors and analysts, but like customers are really going to want to know what they're paying for because there's going to be a lot of, a lot of room here for, you know, if you're, say you're a large enterprise and you go to Anthropic and you're like, Hey, I want the site. Cybersecurity model. That's going to be a very complicated negotiation. And I, I think you're going to want to understand Anthropic's economics in such a way that you can negotiate better. Otherwise Anthropic is going to say here, we're going to, we're going to give you the super fancy luxury VIP tokens and you're not going to know what that means.
Speaker 1Yeah. That, that makes sense.
Speaker 2Because, because we know we can, we can calculate now, like we know what their hardware is and we can, we know roughly what the, what the specs are and how many tokens per second it is. Yeah. And we know what that costs and you can do a lot of interesting stuff once you have that
Speaker 1data. Yeah. Agreed. All right. Well, we had some other things we're going to get to, but we're up on, on some time. So we'll save those nuggets perhaps for another one. And we've got some nice guests as well in the next couple of weeks. So stay tuned for that. And we will talk to you next week. Thanks for listening.
Speaker 2Thank you, everybody. Leave us a rating. Tell your friends, tell your agents. We'll see you next week.
PodcastTranscript AI