Memory Pricing Pressures, Meta's Muse AI, and the Agentic CPU Crunch

[MUSIC] >> Hello, everyone. Welcome to another episode of The Circuit. I am Ben Behren. >> Greetings, programs. I'm Jay Goldberg. >> Well, you returned to our regularly scheduled programming of just Ben and Jay after back-to-back weeks of guests, which we appreciate the feedback on all of that. But now it's us to crank it away on hot topics of the week. >> The two of us and the rooster. >> No, my rooster, I shut the window and it's midday. They're not crowing so much. I appreciate their feedback. I know some people don't love. We have very loud roosters sometimes in the morning, and they're not far from my window. So be it. All right, let's start with memory. And in particularly the, what are we going to say, the fourth and fifth enforcement of the memory apocalypse, the Chinese players, who are obviously ramping capacity? We see a lot of news come across the chatter sphere, the discord in the slacks that we're in around that. So they're moving quick, the Chinese memory guys, I know. So you had a take. >> Yeah, I mean, I saw a lot of headlines this week, or a lot of chatter about both CXMT, who does DRAM and YNTC, which does NAND, are accelerating various parts of their program. It sounds like CXMT has pulled forward some of its capacity expansion. >> Yeah. >> No, sorry, YNTC has pulled forward some of its capacity expansion. CXMT is on track to add another 100,000 wafers per month by the end of the year, end of next year. So that's a 33% increase for them, year on year, and capacity. And then CXMT also sounds like they're making a lot of progress in sort of just developing their quality of their product. I've heard different rumours of what that means, but it sounds like their yields are improving, and they're starting to branch out in different flavors of DRAM. So not like the sky is falling, China is coming kind of news, but just a period of reminder that these are serious providers of memory, and they are getting better and bigger pretty quickly. >> Yeah, and I think you're already hearing, I mean, look, at the end of the day, there's a lot of OEMs, and I know it's not just the Chinese OEMs who would love some release to the pricing pressure. I saw two different forecasts come out earlier in the week, downgrading consumer devices in volume, really across the board. I mean, it was everything. It wasn't just PCs, it was phones and everything because of the memory price bug, which I think is a fair downgrade in consumer electronics. It's rough, it's rough out there. It gets going to be ugly in consumer hardware for the foreseeable future. >> I don't know, I'm starting to hear different rumblings. Like, I mean, most of this is mostly coming from interested parties like memory company CEOs, but it sounds to me like there is some plateauing of consumer grade memory pricing, you mean? >> Yeah, pricing, yeah. And I've heard that from a couple of people, I don't know how much I believe it, but I think the consumers have to really, really appreciate that now, just a little bit of, because I think if the memory makers are smart about it, they'll realize they really have two very distinct customer groups. One customer group is very price sensitive consumers. And the other side doesn't really care about price, and we'll take all the memory they can get in the data center. And I think the memory guys will see the way to sort of balance that. I mean, it's not an uncommon problem, like market segmentation. So I think there may be, at least, I mean, I don't think memory's getting cheaper in consumer, but maybe prices stop going up again. As opposed to data center pricing, memory is just gonna keep going up. >> Keep going up, yeah. Well, okay, so here's the nugget that I had found out. So there are certain skews of memory size, you know, gigabytes that aren't normally in a skew or at a base slash entry skew that are cheaper than some memory that's less than that gigabytes. But the OEMs don't wanna move up into that price because once you increase your storage or your memory, there's really no backtracking from that. Even if it's less expensive than a lower tier of memory. And so you kinda have this dynamic of, like future proofing planning to make sure that you're not stuck at a certain tier that historically is more expensive than a lower storage or memory tier. And that's creating some really interesting dynamics as to where pricing increases happening, what they do with skews, how those prices go into certain skews. So it's just different navigation, right? You've got to manage through. But I thought that was interesting 'cause it's not necessarily all pockets of memory. It's pockets of memory that someone doesn't wanna bring into a certain price point because that locks them in. You can't downgrade from that, going forward. - Yeah, I forgot to do the comparison for the iPhone 'cause the new iPhones of course came out. And if memory serves me correctly, memory tiers on those stayed the same as the last generation. - Yeah, storage went up. - Storage went up, okay. So, but I know that that dynamic is true in China where you're seeing a lot of phone specs sort of flat line, memory and storage wise over the last year. Nobody increased their, some vendors actually, some vendors actually decreased both in a lot of skews 'cause you don't wanna get stuck, I take your point. - Yeah, yeah, but chicken is fine. It's just a, you gotta navigate the right, that landscape. But no doubt, right? And even if we see some pricing stabilize, which I hope you're right, the, it is a higher pill to swallow. And I think that's the concern in consumer electronics right now is that we're at a already inflated state and prices, entry level prices have gone up to what you just consider at mid tier. And everybody's having to just navigate how they set these prices. And fall is gonna be really interesting. Like, what happens with promos? What happens with things like PCs and smartphones and other consumer electronics? In fall, I think it's gonna be really interesting to see where they land pricing, what promotions happen. 'Cause yeah, it's gonna be rough. And then we'll have to, yeah. - I think about this strategically. If the Samsung, Hynex and Micron are smart about it, they're gonna, they're gonna do what they can to protect their long standing consumer customers because if they don't sort of look out for them to some degree, that is gonna open the door for those customers to look elsewhere. And I think the fact that two, three weeks ago, we heard a lot of reports that Apple had tested CXMT memory and was pushing very hard to get a waiver to, from the US government to be able to buy it, I think that sort of caught a lot of attention at the other memory makers. Because if Korean Idaho can't provide the memories China can, I mean Wuhan can. - I mean, but the end of the day, right? They should be concerned because once you're in the door with another alternate player, like Apple's not gonna, all of a sudden go, "Oh, we're gonna stop using them." Like now they're another supplier. And that would hurt. - That would hurt. Barn doors open. - Yep, and I think the other thing too, is you think back, what was it, three weeks ago with hot chips? One of the sort of big takeaways I heard from a lot of people with hot chips was, the memory presenters got a lot of very tough, they faced a pretty hostile crowd. People were pretty angry at them. And in particular, you know, this is a technical presentation and the technical, like, there were a lot of people pushing back saying, "Why are you moving to hybrid bonded "memory, why are you going higher?" We don't need that because that is just gonna be more and more wafers that are gonna be needed. - Yep. - Like, let's try a different approach to memory. It doesn't have to be so high. And so many stacks of HBM. Let's try a different path forward. That was very, very loud feedback I heard. - Yeah. - So I think the memory, the memory players have to, you'd think they would just like, sell everything they can in this great times. And no, I think they're gonna be a little smarter, a little bit more strategic about it. - Yeah, and I think that, you know, we've talked about the memory pricing, you know, before. And it's interesting when you, you know, talk to them. to some folks in the region, different regions. You hear them be sensitive to this, knowing that margins are going up, and again, there's a different margin profile for all three of your big three. But you hear particularly from the Korean players, and then this is actually true of a lot of folks, deep in the semiconductor supply chain in Taiwan, in particular, and even in Europe, where they don't want to press margins to look opportunistic and trying to take a consumer-friendly view or a customer-friendly view in this, I think, will be appreciated by those companies long-term. I mean, just use the example for Apple, right? I mean, if you think the memory players are up in arms or have a grudge against Apple, like you just wait to see the grudge that Apple will hold, if you continue to be hostile to them, and they are your biggest scale customer, like, this is a non-negotiable, you should try to still be somewhat customer-friendly, and this is just a fascinating, fascinating dynamic. Anyway, but yes, you're right. As long as you know why I'm seeing CXMT, do continue to scale, we know they're going to bring relief to China-based OEMs, who will hopefully have some pricing because their prices are going up. Their low-end tier to mid-tier is getting rough, but, yeah, even for Chinese OEMs, they can't run quick enough. So we'll see. Memory dynamics remain extremely-- >>Express-- >>Express, yeah. Let's talk meta. So I have a couple of broad opinions, and then we can talk about some of the implications of this. So everybody's already muses the greatest thing ever, I don't know if you all played with Instinct 2, but essentially what you're seeing is a very interesting, let's just say, maybe more consumer use cases, I don't want to say it's only consumer, but things like, what would it look like to have a chatbot that can go do product research for you, or what would it look like to do a chatbot that can help you find cheaper flights or manage this in X, Y, and Z? And so people play around with these, and they see that element of this, which an assistant is trying to, for lack of a better word, be an assistant, right, in a central location. And people have been very favorable to this, like muses sort of taken off. I get it. I've tried it. It has some great value. In fact, one of the things that it does the best, Instinct fails at this, but Muse does this the best, is try to anticipate some things for you. Like, oh, hey, this just happened, or I see this on your calendar, you know, you might be hungry around this time. Should I find you a restaurant? Like, it sinks together what it knows to try to like be helpful, right, or anticipate. And there's value there. I think that's, you're starting to see the release of this, but I'm going to comment. So our friend, Ben Thompson, was on TVPN or whatever, and I think he was talking about an article that he had written. In fact, in my last interview with him, he made the same point, which is, you know, consumers don't want to be productive. And to some degree, consumers like shopping, which I agree, you know, he used the example of like, people go to Walmart to go to Walmart and look around and shop around. And like, for me, that's Costco. Like, I enjoy Costco for some obscene reason, but I really like going to Costco. But those are the things like those aren't going to be automated away. And I do think there's some areas for consumers who will engage in these things and be okay with something helping them, again, maybe flights or planning a vacation or things that aren't enjoyable or tedious. So I don't think it's a blanket statement that consumers don't want help or help shopping or whatnot, because they want to just go shop themselves. They want to do that for things they care about. They might not do that for things they don't care about. But I also think that these little agents or muses actually also showing you what a personal enterprise assistant could look like. And that's where I think things get super interesting, because not all employees have had a true kind of connected assistant who can help them be more deeply productive. Do things like manage things they don't want to manage, etc. And we haven't really had that before. You wouldn't say that you're engaging with chat GPT or a cloud to do some product stuff yourself as a true kind of quote unquote assistant, the way that you're seeing muses. But I think that's interesting, right, just this degree of, you know, could every employee also have their dedicated work assistant that does some of these things. And then again, the implication of all of this, of course, is that it just creates more compute demand, right, muses doing kind of its own computer, even tells you that it says I have my own computer. I can browse the web. I can do all these things. And that's running in a VM and it requires a whole host of compute and CPUs and GPUs. But that added dynamic of I'm not just using, you know, a research service, open AI and THROPPIC, whatever, to kind of use documents and I am the orchestrator of those things. But now something that could be another part of that and an assistant, a smart assistant of that from an enterprise standpoint, like we, I don't think we'd really had anything like that before. So I just think it's, I just think that whole 10,000 foot view is, is interesting. Yeah. So maybe back in up a second, Facebook had its annual conference this week and they announced a bunch of AI related products, the most interesting of which is, well, the most interesting of which is a little Toma Gucci, AI toy that you have to keep alive. The second most interesting is their AI agent muse. And first, first of all, I want to say, I called this two months ago. I said, this is what the product looks like, this is what the AI consumer product is going to look like. It's an AI assistant that does all these things. And I mentioned that not just to brag, although partly to brag, but also because what I've been saying for a while is the hard part now for AI is the product side. Yeah. You have all this great technology. How do you make it into a product that users engage with and want to use more? And I think muse is a very, like, I think OpenClaw is the first one that you've looked at it. You're like, oh, this is where this is headed. And now you see muse is a much more refined, usable product that, you know, anybody can sign up for. But I still think it's only a step in the right direction. I don't think we're there yet. I don't think the AI models can really, really do it yet. But I think there is immense consumer, and to your point, you enterprise utility. But I really think there's a strong consumer product in the work. Somebody's going to come blow up with something. It's probably a couple generations of models away, but where it's actually really, really useful to the consumer. And it will do, it'll do these things. It will, there's a lot more it can do. And I mean, I think your point and Ben Thompson's point is valid. But that's to me, just in the category of product definition, right? Yeah. Figure out what users actually want. And I also think, like, you and I aren't necessarily the target demographic here, because I started using instinct. And the second it started asking me for all my passwords, I was like, who is this? I still have privacy concerns, but apparently, lots of people out there don't. I do not. I am happy to just give my life, no, it's like, I was telling there's rumors that other other people will do these assistants too. And I was like, I'm ready. I'm ready to give it all my entire life, but yeah, but okay, but I want to make this point, because I think this is interesting. For me, like, what Muse did, I think instinct tried this, but to be honest with you, like, I just don't feel like it landed. Muse was the first one that kind of felt like, once you let it to connect to some things, it started saying, I can go do these things for you. And to me, this is actually a critical part of the onboarding. So, so what I'll back up and say is remember, remember when, you know, AI was kind of early, and I had already like gone fully down the path and then you were saying, like, I sit there and I'm not quite sure what to do with it, right? I think this little thing, this little agent becomes the thing that helps you figure out what to go do with it. And I say that for two other reasons, there is, from our research, both with checks we've had with people using Enterprise, AI, there is a gigantic skill gap between people who get it, who are deep in the weeds, who are like automating everything, and then others who stare at a prompt and go, I have no idea what to do, how to use this to get the most out of it. And my inkling is that this little thing, whether it's Muse, whether it's Copilot, whether it's Gemini, becomes this, I don't know, anthropomorphic concept that starts to guide you and prompt you and start to suggest, you know, I can go do this for you, I can make that were documented. for you. Watches you and says, "Oh, I see you're struggling with this. Let me help you." I think that might help this concept of this agent, it might help bridge that skill gap for those who aren't, right? Deep in the weeds here and pushing the envelope with every model because they they just want to test its limits. That's not every person out there to get the most of it. So, to your onboarding point, to your UI point, which is exactly right, right? You've got to productize this. I think the agent, this little assistant thing, is the thing that might be the catalyst for adoption and onboarding/how do I use this to go do the wonderful things that it can do for those that don't aren't immediately aware of that? Agreed, but I think it's not quite there yet. It's definitely not. It is the thing. For me, it was OpenClaw, and Opus 5, 3.5, and OpenClaw. This is what the consumer product could look like. I think this is going to spark a lot of people's imaginations, but I also think there will absolutely be similar versions from everybody else, and sooner or later, somebody will get it right. Maybe it'll be met up, but it's a step in the right direction. Yeah. I think Apple makes a play here, sooner than a year. I think Google does this too, right? And the thing I keep walking back from is if you're just going to pick, who would you think is best advantaged? And I hate to harp on Facebook, but I do think meta. I do think there's people who will hesitate to give all of their secure information, even if they're okay with it, right, to your point, to meta. I do think there's others who have higher degrees of trust in their life who they would be more willing to engage in some of these things, are Apple being one of them, right? One of the areas where security and privacy is going to play off really well for them is in a Siri agent like Muse. But I think Google's there too. I don't think Google's out of that list, and Amazon might make a go also. But there's only a handful of companies who have what I would just say the right and the right relationship plus the distribution to go and succeed at this route for consumers. I agree with most of that, but I do think there is room for some third party start up to come along and build their brand on privacy and also on some form of independence. I've maintained for a while that part of the agent's role will be an orchestrator across different AI models, so you don't have to pick which model you're using. It will optimize for that. I agree. Meta shows me that that may not be so important because the model is free, right? But it's also not quite clear how capable it is. I think if you really tasked Muse with something very, very complicated, I don't know what the limit would be. I imagine there is one, but I haven't heard it yet. If I want to have Muse go build a giant research report for me, it's pretty simple. I can't do that. I haven't tried it, but I don't think it can do that or do it well. It's not it's goal. It's goal is different. Yeah, but I think the ultimate consumer product will possibly be from a third party who and I think the way that they can monetize is to you pay them and they basically mark up someone else's frontier model, but then they optimize around as you don't have to bet. You don't end up coming up with a big bill. They use free open source when they can and they use the frontier when they have to, of course. If anybody's working on this call me because I've got the product in my head, I just give you your monetization. My point is just that there's room for a third party here. I think a hot startup, a smart team, small smart product people could become very powerful. Yeah. Well, I mean, we'll see because I think two very big lab quote unquote startups are going to try this also. Everybody can try this. And we'll see. Yeah. And we'll see. Okay. Anyway, so let's let's talk about Anthropic Plus Occamye, which was an interesting deal. And then what this means for compute demand and tie it all together. So Anthropic and Occamye. So Anthropic and Occamye announced the partnership together. I forget it's like it's a like $11 billion deal. And Anthropic is going to pay Occamye over some period of years, $11 billion. And in exchange, Occamye is giving them more. So typical deal we see nowadays in equity exchange. And for those of you who don't remember Occamye, or forgotten about them, they are a CDN. They're an edge network provider. They have servers all over the world that sort of accelerate local services like the cash movies that are always going to be watching this or neighborhood or whatever. But they also have a cloud computing business. They bought linoid a few years ago. And they have not released many details about what exactly Occamye is going to do for Anthropic. And so I've been digging into it all morning. There's a couple of ways it could go as far as I can tell. One is Occamye is going to run inference for Anthropic as a pure compute deal. Anthropic needs CPUs in particular to do inference. They probably haven't been getting a lot of those in the deals they've signed. They really need those now to do more inference. And linoid and Occamye has a big pool of probably underutilized CPUs sitting around. I also think there's another angle here which is around the network topology. The way that we access agents today can be more distributed. Right now, you know, you put in a request to cloud. It probably goes to an AWS server in Seattle or Utah or wherever. But having something more local to offload some of that and also provide some of the security for agents I think is really important. Which kind of leads me to this theory that like Cloudflare has a has a big role to play here too. I think they're the next one who has a comparable offering. But they don't they don't have a big pool of cloud CPUs. So yeah, there's a lot of ways this can go. And it was pretty interesting. It's a pretty interesting deal because what was the last time anyone talked about? I mean, I think they either used to host Netflix or they still host Netflix. I can't remember, but that was the long time. But then Netflix built its own CDN. It did. It's that's right. Yeah. Hey, maybe that's Netflix's next business. If entertainment slows down, let's start renting out compute. All right. So, but all of this like just again continues to emphasize how short we are compute. And there's been a resurgence of the CPU demand conversation because of mues, right? Because yes, while it is its own computer and running in VMs, that still takes a ton of CPUs. And so I think, you know, it's just it's interesting where and I'm working on a report for this next week. So we can talk about this more in depth. I'm going to call it the the agentic CPU, the agentic CPU guidebook is going to be. But everybody is taking a very different approach to their architectures. I think there's going to be some vendors with architectures that play better with GPUs than others, which will be very interesting to play out or accelerators, XPUs. But it is true that the the CPU here, like we need a lot more CPUs because they again, they're not just part of orchestration. It's a part of the actual token workflow from reasoning to action to review back to reasoning. Like it's a it's a very close loop. But we haven't we haven't yet seen mues come up against this, although I've heard they're ramping pretty quickly and they're starting to get up this. But what's interesting about this to me with Facebook with Meta is Meta has a lot of CPUs. Like they may very well have more not more, but a lot more CPUs than others out there. And so it's it's interesting that they can leverage that in a way that maybe maybe others won't. Maybe others might run more GPU, right? Or XPU agentic workloads here for these for these personal assistant agents. But the bottom line is like everybody you talk to in the space is continuing to to pound the table on. How much more compute a personal agent running its own personal computer, but having the ability to go work around the clock for you in the background of doing things is going to press on our infrastructure. And I think that's true. Yeah, I there was there was even before the Meta stuff came out. There was a big debate on Twitter last week about CPU demand going through the roof. And there were people on Twitter. I think you commented some of them were saying the ratio of CPUs it's going to be a 30 to one ratio of CPUs to GPUs. And it was one of those things where I was like wait a second a year ago the same guy I was saying it's going to be 16 GPUs to one CPU and now we're 30 to one in the other direction. I was like, uh, everybody needs to take a breath. Just like, just chill. But the world's going to need a lot of CPUs. Yeah, right. It's very clear. And it's, I think it's, I think it's surprising everybody. I don't think anybody saw this coming six months ago, which is kind of weird. Yeah. Maybe open AI did. They don't seem to be struggling for compute, but that's the next deal. Who knows? Yeah. Well, I mean, again, right? Everybody needs to get to get more CPUs. We just unfortunately cannot make as many CPUs or GPUs for that matter. Um, but anyway, it's, uh, I, you know, the pace of this is, is so fast. Like, I made this joke the other day. I don't know if you feel like I, I wasn't, I was not in, well, I was aware of the industry participating in the industry, but not working in the industry, um, when, you know, there was sort of the first software evolution, you know, when stuff that you couldn't do digitally took you a lot of times manually, you could all of a sudden do. And sometimes I feel like we're similarly in this environment where like it, that's, that's what the excitement level was when you saw something and you were like, holy cow, like Excel, right? I can't believe I can do this. Like, I feel like we're in one of these moments. Something new comes out and you're like, I'm going to try this. It worked. Are you kidding me? Yeah. Yeah. Yeah. Fascinating. Fascinating times. Alright, tighter episode for you all this week, but, uh, we appreciate your time and, uh, we will talk to you next week. Thank you for listening, everybody. [BLANK_AUDIO] [BLANK_AUDIO] [BLANK_AUDIO] [BLANK_AUDIO] [BLANK_AUDIO]