Go back

Unknown

57m 26s

Unknown

The energy transition and AI infrastructure are now deeply intertwined, with a shift from large-scale, centralized data centers to distributed models that leverage existing buildings and homes. This shift is driven by both technological feasibility and political pressure—many states, especially after recent elections, are imposing strict limits on large data centers, prompting a reevaluation of where and how computing is deployed. The core idea is that underutilized grid capacity, such as in commercial buildings or through EV charging networks, can be repurposed for AI compute, improving efficiency and affordability. However, challenges remain, including power quality issues from spiky inference loads and the lack of long-term, stable off-take agreements for smaller facilities. While residential models face skepticism due to cost and complexity, they offer potential for community-level savings through virtual power plants. The episode also highlights the fallout from NVIDIA’s leadership crisis, which has further eroded trust in AI infrastructure investments and raised concerns about corporate accountability. In response, utilities and industry players are pushing for more transparent, integrated, and community-focused solutions—such as combining smart thermostats, EV chargers, and batteries—that serve both consumer savings and grid stability. Ultimately, the future of AI computing may not lie in bigger, more expensive campuses, but in smarter, decentralized, and more socially acceptable models embedded in everyday infrastructure.

Transcription

9968 Words, 55408 Characters

English
Latitude Media, covering the new frontiers of the energy transition. You got around your corporate firewall. Yes. I think we're just being extra cautious, you know, all those Chinese spies out there. The power of being an executive. You, too, can jump the corporate firewall if you need. Which makes you question the firewall to begin with. In 2017, Anthropic CEO Dario Amadei, who was then a researcher at OpenAI, wrote a memo that set today's infrastructure boom in motion. Called Big Blob of Compute, it argued that the only way to build general-purpose AI was to make the models bigger. Clever, narrow models were not the answer, he argued. Rather, it was more compute, more data, and longer training runs. That research hypothesis turned into the operating philosophy of leading AI companies. And that blob grew into something physical – gigawatt-scale campuses, nuclear restarts, gas plants, big batteries, and a build-out now measured in the trillions of dollars. We've been sold a very specific story about the direction of AI infrastructure. Bigger is always better. But is it? There's no question that we need way more computing to push the frontier of general-purpose models. But large data centers face so much opposition and so many constraints, there's a debate over how that computing gets built. Should it be, can it be, far more distributed? Rather than taking up thousands of acres, could the computing live in smaller warehouses, empty office space, or even homes? This week, we'll look at emerging models for distributed computing and how they're becoming a distributed energy play. Plus, we'll dip back into some post-midterm commentary and talk about the fallout from Jensen Wang's departure from NVIDIA. That is coming right up on Open Circuit. Open Circuit is brought to you by Everyday Electric, formerly known as Renewable Energy. Ben Brown is the CEO of Everyday Electric. With six gigawatts of flexible capacity, Everyday Electric empowers customers to take part in that transition and helps the grid grow responsibly. So yes, you need to build infrastructure. But you have to increase utilization of the system we have. We have to find ways in which to drive savings, both to households that are part of virtual power plants. Because that, if you do that at scale, it actually drives savings for everyone. From batteries to thermostats to electric vehicles, homes around the country already have the energy products that could save them money while providing more capacity to the grid. Since its founding, Everyday Electric has been working toward a more affordable, more reliable system for everyone. Now that we are reaching scale, there's just these amazing moments. There's just these amazing moments in time that we can unlock 50 megawatts or 100 megawatts of resource across every major climate region, every major utility. Later in the show, Ben explains how VPPs can meet utility growth while driving down rates for everyone. To learn more, visit EverydayElectric.com. Giving Tuesday is December 1st. Time to put climate giving on your year-end checklist and apply the same due diligence you would with any investment. That's what Giving Green does. Giving Green researches the strategies. And nonprofits best positioned to unlock the next big climate breakthrough. And sends 100% of every dollar donated to the Giving Green Fund directly to them. Maximize your impact per dollar. No cuts or guesswork. Make your first gift before 2027, and it's matched up to $1,000. Go to givinggreen.earth.com. Open Circuit is brought to you by Energy Hub. Energy Hub's DERMS platform gives utilities flexible capacity they can stake their grid planning on. Coordinating batteries, EVs, thermostats. And commercial and industrial sites through a single system. Utilities unlock a partner ecosystem that covers more than 90% of connected DERs in the market and integrates seamlessly into existing infrastructure. The scale is proven. More than 230 utilities across North America trust Energy Hub to manage over 3.5 gigawatts of flexible capacity. Learn more at energyhub.com. Welcome to the show. I'm Stephen Lewis. I'm the executive editor of Latitude Media. We have another full roster today. Jigar Shah and Caroline Golan are with me as always. Jigar's the co-managing partner of Multiplier. Jigar, small data centers, distributed data centers. Is this the episode you've been waiting for? No, but I will take it. I will take it. What is the episode you were waiting for, just out of curiosity? I mean, I still am obsessed with vehicle-to-grid. I know. Yeah, you've talked about that a lot. Yeah. I was hoping you were going to say, when we're in Barbados or something like that, but no. Vehicle-to-grid, that sounds fine, too. Well, did Elon say that every Tesla was going to become a distributed data center? Yeah. I mean, he says a lot of things, though. Well, that relates to the episode that we're having. I don't know, Jigar. I'm a little disappointed because I thought we, you know, we're hanging out at Climate Week and you were so excited about talking about this topic. I'm excited about a lot of things, especially when it comes to you and Ad Hoc. Oh, thanks, Jigar. Appreciate you. That voice you hear is Jim Capsis. He's the founder and CEO of the Ad Hoc Group. He also authors the GIST newsletter, and he has been spending some time thinking about distributed data centers lately as well. Hey, Jim. Good to see you. Thanks for having me, guys. Yeah, great to have you on. And, of course, Caroline Golan is the Chief Growth and Policy Officer at NRG. You had your work cut out for you after the election. Did this week and last week just bleed together into one? Oh, yeah, it's all fine now. We're good. Cool. No, we're good. Figured it all out. Done. Yeah, rate freeze, rate freeze. Move on. Move on. We're good. All right, before we get into the big questions about distributed data centers, let's just focus on the biggest story in the news right now. CEO Jensen Huang has resigned after revelations of an affair with a direct report and a woman suspected of being a Chinese spy. Other affair allegations have come out in the last day or so. NVIDIA is seen as Jensen Huang. Jensen's been personally responsible for many of the deals. With other AI companies that help finance chip purchases and data center buildouts to the tune of around $530 billion. So if NVIDIA's stock continues to take a hit or if it starts to pull back on those deals, what are the possible downstream consequences? And I think those consequences connect to the topic today. Caroline, do you think this is going to eventually lead to some meaningful shift in the infrastructure buildout? Well, I do think NVIDIA has been a strong. Investor in what they believe is the right way to build infrastructure. And, you know, and that is to Jensen's credit. And I think to the people that he surrounded himself with, apparently they didn't give him advice in other points of his life. But I do think they have been that. And, you know, and I think both together, I've met him before and he's an incredibly charismatic man. And he's very committed to the concept that AI is. Inevitably, what is best for all, for everyone and the entire community. And he did push NVIDIA oftentimes to make investments that would never pencil. But his view was if it made the infrastructure of AI better, then in the long run, it was better for the overall narrative. And that really came from him. There are many people I think of at NVIDIA, some of which I've worked with and, you know, some of which have been there since, you know, employee number five or six. Who have. Who have carried this mantle. So, you know, with new leadership, he really ran that company like command and control. Right. So it's going to be interesting what the board does with this and what new leadership does and how they redirect. I will also tell you, I mean, anyone who was looking at that knows that Jensen, to his credit or to his own, you know, political risk, has been hand in hand with Trump. And the administration and NVIDIA is on the list that Dems are going to go after. And there are tons of questions about, you know, what NVIDIA's relationship has been and Trump's, you know, profiteering with that relationship. And someone's going to take the fall for that at a lot of companies. So it's not actually surprising to me, both for his poor judgment in his personal life, but also what's probably facing the company. Yeah. That they. Made this decision when they made it. Everybody on that board is lawyering up right now. Yeah, absolutely. As a former national security professional, I'm just horrified, you know, on that level alone, like, as to how this was allowed to happen. And also, who knew what and when did they know it on the government side? And I think, you know, we'll see what happens with hearings next year or whatnot. But there's still a lot of questions. Maybe there's not as much there as there may, you know, seem to be. But I've got to think this is a story that's going to keep unraveling over time. And so there's a lot more that we're going to learn. I absolutely agree. Yeah. Particularly on the government side. I think that's where the set of questions are targeted now. Jigar, what do you think? The interim CEO, the former CFO, is someone who. Is female. Is female. Yes, and has reportedly pushed back on some of the deals that Jensen has pushed through, again, to support data centers and chip sales. What do you think this does to NVIDIA strategically, and what effects would that have on, you know, the build-out itself? I mean, the problem I have. I have with Jensen, but also other people as well, and I certainly didn't know anything about the national security stuff or anything else, is that I think there is a feeling that they can take this global commons that we've created, like the grid, and manipulate it to their will so that they can become wealthier without any collateral damage to the other people using those global commons. And that's the thing that I always viscerally reacted to. And I don't think that he thinks any differently today than he did when I told him he was a dumbass in 2023. I don't know if you've seen all the investments they've made in energy. They're the most harebrained ideas I have ever heard of. Some of these concentrating solar projects and long-duration energy storage projects and all these technologies that you and I both know don't work and have not worked. But they were supposed to magically solve all these energy problems. The level of selfishness just never ceases to amaze me. I have said this before. None of the hyperscalers or the big chip companies thought that it was their job to fix the grid. It was their job to win the AI race. Fixing the grid became a barrier. And poor people on the grid be damned. Poor people on the grid be damned. Right? Their mindset is, I have to win this AI race or China is going to. And that's what they were going after. Fixing the grid became a hump in the Everest that they were, you know, climbing to win this race. And I think that what the approach that Jensen took was, well, there's a lot of stable capital out there to fund the things like natural gas and solar and wind. And I think that what the approach that Jensen took was, well, there's a lot of stable capital out there to fund the things like natural gas and wind. And it didn't make sense as an investment thesis. But I think he invested in some great shops and some great ideas. And I don't know how much of it was actually Jensen. But I do know that, you know, very famously, there were people on his team who said, we try to pay as much as we can to our supplier partners because we want them to succeed. And I do think he brought a different perspective to it. Was it calculated and was it informed by excellent modeling of what is best needed to lower rates and increase the resilience of the grid? Obviously not. Obviously not. But it was informed by people who thought that they, like, had a vision and were very hungry and, you know, could raise millions of dollars off a slide deck. But we should let him off the hook because he clearly didn't have the resources necessary to get smart people around him. I'm not letting him off the hook. I'm just saying, like, I'm just saying, let's not negate the good that he did just because he can't keep his pants. Zipped up. That's all I'm saying. Go ahead. I have this sort of, maybe it's overly optimistic, but I think we may be in this sort of both cultural and market moment after the election now with what's happened with Jensen of, like, a little bit of a reset of, like, you know what? Like, this is usual. This is like an exclamation point of this is as usual as over on data centers. And it's already true by heading into the election. It's very true on the other side of the election. And I think what's this exit and how sort of big a deal it is means that, like, is he key man risk not just for NVIDIA, but for, like, the market? And I've got to hope that it's not for the market because if that's the truth, then we really have built this market on a bunch of sand. It's really about one guy. I do think they are somewhat outliers in the market. Like, I was talking. I was talking to a friend about this last night. Like, let's say the same story ran at Google. There's a succession plan. There's transition plan, right? Also, I don't—everyone is so watched, and it is so corporate at Google. I don't see how all of this could have happened, right? There are so many internal checks and balances. And that is the difference between a company that was 15 years ago making, you know, video game parts and now is the biggest company in the world, right? And so, like, as hard as it is— As someone who has been in that corporate bureaucracy for quite some time, it does produce a level of stability when shit hits the fan like this, you know? And, like, NVIDIA didn't have that. It was ruled like a monarchy. Yeah. Yeah. And I'm also preoccupied with this set of really practical questions, which is what happens—NVIDIA is taking all these equity stakes in neoclouds. What happens if they draw back on that support? Are they going to continue supporting that? Massive SP Energy Ohio site for OpenAI. You know, there's a lot of large load tariffs that require collateral. And so, if NVIDIA support weekends, will customers have to post more collateral? The CFO has already answered those questions. Like, I think in general— What did she say? She basically said what we've all been saying, right? The thing with Jensen Huang, which I love, and Elon Musk, which I also love— Wait a second. Is that they create their— I love for the logic. Right. Is that they create their own distortion field, right? All of it is just one big distortion field, right? And what Jensen understood about CoreWeave and everything else was that once you start to break the distortion field, it all comes crashing down, right? Like, it's not like—you know, like, people have to believe in something they can't see to continue to invest this money, whether it's with SpaceX or whether it's with— Whether it's with CoreWeave and the neoclouds, right? What the CFO basically said was that she is not comfortable with any of these financial positions that they've taken, and she's going to be reevaluating all of them, right? And my sense is, is that if this distortion field breaks, right, then you will have the correction that Jim was suggesting. And it's not because AI is not a real thing. I think it is a real thing. But it is very clear that people are front-running their investments before you see— Before you see the level of revenue growth necessary to justify those investments, and Jensen wanted that to continue. And now we'll see whether it continues. The erosion of institutional trust in this country is going to get even worse. And so I think what is particularly interesting about this timing is that it's happening on the heels of a major populist movement in politics, a change in the guard in electeds, who ran on the sentiment of these— Oligarchs have ruined your country, right? And here falls the first. But if the Chinese spy and the sending of scrub servers to China, all of that proves out, like, I think the flame that's already been lit on the political regulatory clampdown on data centers, you know, becomes a bonfire. But I think this is that moment of, like, maybe in terms of the trust rebuilding, it's like the folks that people have— The folks that people have not been trusting, that might get worse for them. Yeah. And I think, you know, I think Jigar, leading up to this, we were talking about, you know, the election and some of the different new governors and what's happening in these states. And I think that, like—and I said this to some of the hyperscaler folks before the election. I'm like, you guys don't have a lot of time to figure things out. Like, there's a presidential race that's starting. Oh, the presidential race starts in two weeks ago. No, but it's— Yeah, yeah. The minute folks put out their positions on data centers, which are being written by Jigar right now, then, you know, then it's sort of game over. So I think that the—you've got the governors are going to have to stand up, which they're going to do, and they're going to say, here is the way forward. And they're going to have to be viewed as—whether this is how real it is and how much it's manufactured behind the scenes, the hyperscalers are going to have to be viewed to be dragged—being dragged to the table. Like, with a finger being waved at them and saying, like, if we're going to build, this is the way we're going to do it, and you're going to do it this way, or you're not going to do it at all in our state. And that—even if they were willing to do it in the first place, it has to— Yeah, you said the optics of punishment over proactive, yeah. And I think that, like, Jigar, you end up talking to a lot of the hyperscaler folks, too, and I think that they're, you know, they're people, they're emotional about this stuff. They're like, well, we think we're doing some of them. We're doing the right thing. We think we're doing all these great things. We're not getting any credit for it. I'm like, I don't care what you think, because the voter doesn't think you are, and the people in power now need to show to the people who elected them that there's a new way. And that's your only hope. And so you better get on board. And if it means that you're going to look like the bad guy, you're going to have to accept that in the short run. And I just think that's the reality of the political situation they find themselves in. And then we can talk about distributed data centers, which I think is a whole other opportunity. Yeah, let's get there. Which is also what we're going to force ourselves into. Well, and I think the utilities are in the same place, Jim. In terms of what? Yeah, no, I think. That they're going to have to look like they are taking it on the chin from the governor to earn the respect of the voter back. I think you're right. Yeah, again, and the same thing for the same issue for them. You know, I spent a lot of time with the utilities. They believe they're doing the right thing most of the time for their customers. I mean, I was at. EEI in June, and it was all, oh, why doesn't everyone understand how great we are? I'm like, that's the wrong message, guys. Yeah, I mean. You know, it's like read the room, read the room. Right, it's like read the room, 100%, read the room. And again, we can debate, you know, how much is true or not true, but it's the perception. And therefore, you've got to figure out a way forward that like is going to be acceptable so you don't lose your social license to do the things you want to do. I was wondering how long it would take us to get from Jensen Wang's affair to utilities. Here we are. I was just wondering how long it would, Jim, you're going to be on every week because when you keep saying, Jigar, I agree with you, Jigar, I agree with you. I mean, it is, it is. Those might be the three times I've ever said that, Caroline, don't worry. Earlier, we introduced you to Ben Brown, CEO of Everyday Electric. Ben has been pioneering smart home technologies and virtual power plants for 20 years. With electricity prices rising fast, those flexible resources, solar, batteries, EV chargers, smart thermostats, are more important than ever before. It's really important to understand that, yes, we are in an energy affordability crisis, but I also think that because of that, we have so much energy and so much focus going into how we can dig ourselves out of this. And a new study from the Brattle Group backs this up. It finds that residential VPPs could cut expected electricity bill increases in half over five years. There's just a massive amount of value that can be brought to homes when you increase the efficiency of the system, the utilization of the resources on the system, and not just for people that are part of those virtual power plants, but their neighbors, the communities around them. That can mean $4 to $7 billion of savings per year for Americans. Everyday Electric already coordinates the largest residential VPP in the country, and it's working with device manufacturers to make participation effortless, unlocking more savings and more flexibility for the grid. It's really important to think about that cross-orchestration and think about that whole home energy fabric, being able to think about unlocking savings across every, every component of our home. Making electricity more affordable isn't just about what we build next. It's about getting more from what we've already built. Learn more at everydayelectric.com. Giving Green researches the strategies and nonprofits best positioned to unlock the next climate breakthrough. Climate impact is difficult to quantify, but Giving Green's research suggests a donation to one of their top climate nonprofits is roughly 10 times more effective than a high-quality carbon offset. And the Giving Green Fund sends 100% of every dollar straight to them. Maximized impact per dollar. Make your first gift before 2027 and it's matched up to $1,000. Go to givinggreen.earth slash circuit to claim it. Utilities don't just hope flexible capacity performs. They need to be able to stake grid planning on it day in, day out. Energy Hub's DERMS platform is trained on over a decade of real event data from 2.6 million connected DERs. That means utilities can forecast available capacity with accuracy, optimize dispatch across any DER type or ownership model, and maintain visibility across their entire system. The platform integrates across more than 80 OEM and aggregator partners, giving utilities the reach they need as markets and regulations change. Consistent, verifiable, affordable. Energy Hub is the flexibility platform for the modern grid. Let's talk about distributed data centers now. Jigger has been beating the drum on smaller distributed data centers on this show for a while now. And in the last year, they've gained attention as a possible solution to AI infrastructure bottlenecks, a way to perhaps avoid costly grid upgrades, reduce opposition, even give homeowners a financial stake in AI, use empty offices after COVID. And this is morphing into a distributed energy story as well. This spring, Span unveiled XFRA, this 16 GPU compute node that sits inside a house and pairs with a smart panel and batteries. Span CEO, Arch Rao, made some big claims about the cost here, saying that it would be $3 million per megawatt and take six months to build versus, three to five years, and $15 million per megawatt for a 100 megawatt data center. And then we've seen a bunch of commercial activity. So Sunrun launched a pilot that pairs with AI compute nodes in the home. Span and Sunrun have partnered up on this. On the commercial building side, Perimeter came out of stealth with plans to put GPUs in office buildings. Telecom companies like Comcast and Spectrum have been doing this for a little while, putting GPUs at their network facilities. And then, of course, we've been talking about NVIDIA, which has pursued partnerships on different forms of distributed computing. So added together, maybe not that much capacity right now, but there's a lot of excitement in this space. Jim, you have been thinking about this and writing about this as well. How are you defining a distributed data center? What caught your eye about this market? Yeah, well, I mean, distributed at its simplest basis is small. Small data centers, you know, tens of megawatts or even smaller. Smaller versus the hundreds of megawatts or gigawatts campuses that we're seeing. I think most people when they're talking about distributed compute, especially when you start talking about edge computing, which means that the computing is really close to where the device is or on the device itself that is doing the calculations. That means, you know, you're really talking about inference, which are like, like you're calling your AI, you're asking it a question in a moment, and it's doing a calculation as opposed to training all of the data. So I think there are different flavors, but I think those are the ways that I'm thinking about it. In terms of what's interesting to me, I had a conversation this summer with a utility executive in a major urban, major city. And I asked this executive, hey, what's the latest with you in data centers? Because given your territory, I doubt there are big data centers being built because you don't have a lot of like land. I said, what about distributed data centers? And his response was, you know, I'm hearing a lot of rumors about people doing behind the meter data centers, but I wouldn't know what they're doing. And I was like, what do you mean? He's like, well, if it's behind the meter, like, and there are a bunch of, you know, negotiations, you know, going on, like I wouldn't know. And that struck me as like, oh shit, like, and I've been part of the number of conversations starting in the summer, and you guys have reported a latitude, like there are a lot of companies that are doing a lot of work in this area. And so, you know, there's a lot of companies that are doing a lot of work in the area. And so, you know, there's a lot of companies knocking on doors at big, you know, commercial real estate buildings downtown in, you know, the suburbs, saying, hey, you got some capacity that I could use. I'll give you a check, right? And I'm going to come and develop something at your site. And so, my first instinct was like, wait a second, are we going to repeat the same mistakes that we've made with the big data centers and go really fast and go make a buck? Because, of course, I mean, capital is going to flow where there's opportunity. And we're going to have a problem when we get too much, you know, behind the meter that we weren't expecting and the power is going to go out. So, that was my original concern. Yeah, and you compared that to the cannabis industry, right? It's probably the same sales people do. Sure, right. Grow houses came out of nowhere and suddenly utilities are saying, oh, it's true. It's a great analogy on many levels. And, yeah, the same sales people, let me tell you, let me show you something. And, you know, they found out later when they saw the load curves beginning to shift. And so, I definitely spent some time talking to the utility folks about this. I'm a little less worried now about that problem. I mean, I do think it's important that the utilities and the regulators and the developers of any site, as much as possible, this is happening out in the open and not behind closed doors. Because I do think that there's some real potential risks. But I'm less concerned that you're going to have actual, like, outages as a result. Because the way that my understanding of the distribution system and how it actually operates is that, you know, a building in terms of its capacity, it's never, if you get to the point where you're hitting that threshold, the power of the building is going to go out, but it's not going to cause an issue on the feeder. So, I'm a little worried a little less worried about that. But there are concerns about power quality, especially with inference. And so, I've talked to a few utility distribution people who have said that, like, if you have too much inference happening all at once in a given area, you could definitely have issues with power quality that could lead to, like, something going wrong on the distribution feeder. But these, by the way. Because it's so spiky, right? Right. Because it's so spiky. Exactly. It's like, search, boom! You know, and then it comes back down. Search, boom! It's not just constant, right? Like a big data center. And those spikes can be problematic if you don't have the right power conditioning. The voltage and harmonics. The voltage and harmonics and all of that. But these are. I'm told, and I don't know if you guys have heard, these are solvable problems and not, like, showstoppers, which is what I was originally worried about. I will tell you what you're going to say. Storage. That is the solvable problem. He's not wrong. Storage, right? I got it. I got it. I mean, he's not wrong. That's part of the solution. Well, Jigar, there's a lot to unpack there on how this computing interfaces with the grid. Maybe you step back and make the case as you see it for distributed computing. Like when you think about this space, what size data centers are you talking about? What workloads are they serving and why does it matter? So my definition of distributed compute is generally, you know, these data centers being put onto the distribution grid. So they're not connected at transmission voltage. They're being connected on distribution. And so generally that's 25 megawatts and below, but I mean, it could be higher, I guess. I think that the reason why this topic fascinates me so much is because of what Jim just said, right? When you think about just how much capacity is in the existing infrastructure that we have that goes underutilized. I was talking to Prologis the other day, and they were saying the number of customers that we have, they're requested 5 megawatt, 10 megawatt, 20 megawatt like interconnections, but they're only running at 25% of their total connection size because they haven't gotten Elon's robots yet, or they haven't gotten full automation yet, or they haven't put in all of the EV charging infrastructure for their heavy trucks yet, right? So now they're looking to put data centers in that extra capacity, right? So when you think about just how low our assets are, how low our asset utilization is of the wires that we paid for, it is abysmal, right? And now we are forcing the electric utilities to come to the table to just admit how low that number is and what their plan is to do something about it. And so I love this conversation because it lays everyone naked, and they're now going, "Oh, yeah, we were pulling the wool over your eyes before." And we do have the ability to actually figure out a way to accommodate double the load growth on some of these feeders by reducing peak. And we do have the ability to put in some Taurus flywheels in strategic places like Rocky Mountain Power is doing, et cetera. And I'm like, "Oh, you're admitting that modern American technology works, and you were piloting it to death for no reason? Fascinating. Tell me more about how this load growth could get accommodated faster." And I'm like, "Oh, you're admitting that modern American technology works, and you were piloting it to death for no reason? Fascinating. Tell me more about how this load growth could get accommodated faster." And how you could use modern American technology to integrate it now. Tell me more about how adding more kilowatt hours to the distribution grid would reduce rates for everybody, patty poppy, when you're on stage talking about how data centers are the only way that you can make electricity more affordable in California. I love it, and the popcorn has popped. I'll be a little bit of a contrarian on Jigar on this. He likes to take his swings against the utilities, but I think the utilities are already looking at how to actually use their grid more efficiently, for the reason that you just stated. I mean, literally, I think the joke in the utility industry about the whole idea of grid utilization is, like, we're utilities. Like, literally, utilization is what we're supposed to be doing. And so a lot of the ones I'm talking to, they welcome this conversation. They don't feel as if it's being imposed upon them. I think it's true of some. You guys had Todd Conner on the show. Well, we're passing laws in 30 states, so let me know whether they're for it or against it. I think what is definitely true is that utilities have been largely blind on their own distribution systems for a long time. I mean, up until recently. And in fact, many of them are still pretty blind, because maybe this is making one of your points, Jigar. I don't know, but, like, there are companies that have technology that they can avail themselves of, and some are, that can actually help them understand what is happening almost in real time, if not real time, down to the transformer level, instead of just using heuristics, right, engineering assumptions about, like, what the distribution system can handle. And I think the fact that they have that data, and that's being used, I mean, Excel as an example, how they're looking at and trying to find places to put batteries, right, for the filing that they made. In Minnesota, and the work with SparkFun that you guys talked about before on the show, like, that was, like, software that they availed themselves for, you know, working with a company called BridSight that we work with out of Australia that is able to do that. And that was not true two, three years ago, and it's true now. So, like, they have the tools. So I think that, like, where I will agree is that the tools are now there for utilities to know what is happening on the distribution system. They now need to deploy those tools at scale. And then they can actually find where that capacity is. I think we need to -- this is actually a really fascinating part of the conversation, but I want to take a step back and talk about how viable is this compute model. And, Caroline, I'm wondering if you -- when you were at Google, I mean, obviously a lot of workloads are shifted across geography and data centers. But if someone came to you and said, like, we are going to compile however many 20-megawatt data centers or 10-megawatt data centers, like, is that even viable for a company like Google? When I was there, it wasn't something we were looking into. And I don't want to speak for Google. I don't believe it's still a primary agenda. I think that the lens by which the hyperscalers were looking were economies of scale at the time. And this was based on having a land bank strategy and the belief that there was more capacity in the system than there probably was, right? But that hadn't really hit. And the other reason why I don't think that it hit is because we had already a cloud market infrastructure, right? And so a lot of the edge compute and neocloud world has evolved as the more micro version of what cloud was. But cloud was inference. It needed to be a certain radial proximity, largely Virginia, Ohio, to where network was and where -- and use was, right? So that wasn't a novel idea. I think we've focused on the C&I side of this. But I do think where the hyperscalers are interested more so is behind power plants that actually aren't being fully utilized because prices are soft or because it's areas where there isn't as much demand. And so going behind -- you couldn't fit 50 megawatts. You could put 50 megawatts behind a gigawatt, two gigawatt power plant, right? And so I see more of a movement there right now. And the reason is because it's a single contract, right? You're only signing a contract with that supplier, as opposed to also having to sign a contract between the supplier and the C&I or the C&I customer and the end use. And the bulk power system is already there, so, you know, in terms of the harmonics constraint, you have much more headroom. So I've seen a -- I've seen that from the hyperscaler community. It's just the world they know. I think they get a little nervous when it's their brand and when, you know, a Walmart or a big office building is like we have a contract with Google. That means that they have to manage their brand with that endpoint customer, as opposed to saying we have a contract with no name edge provider. You know, it's not as big of a risk. So I think it's also about brand management. However, what I'll tell you is that, you know, everything that just happened in Ohio and what we expect will continue to happen across the country is going to force, I think, more edge compute or smaller compute in this way. Because people are going to be doing -- companies are going to be doing the tradeoff between do I just let these chips depreciate and lose my X billion dollars I've invested here or take a risk on what might be -- I mean, I don't want to say it's spotty, but at least potential compute or training. And so I do think you'll see a shift in that. What I think is interesting is that that headroom, as Jigar sort of alluded to, that headroom is sort of the headroom that many analysts have seen as the potential for EV charging. If that gets eaten up by, you know, edge compute, what are we doing to accommodate for, you know, potential electrification? And so I think we're going to see continued electrification of the grid and continued electrification of transport. Can I break it down, Steven, where I see like a few of these different opportunities in the organization? I think Caroline definitely touched on a few. So like one, to be very explicit, like the co-location with existing generation, which you mentioned, and there are definitely some companies focused on the renewable side, right, that are doing this. There's a startup called Rune. There's a startup out of Australia, Windy Sea, that's going to have, they claim, 250 or megawatts. They're going to have 250 megawatts deployed next year and have big ambitions beyond that. And so I think that's one. There is placing them at substations where there is headroom, so utility-owned substations. I think EPRI has done some studies there and is piloting something at one or more substations, so that's interesting. There's the actual, like, the commercial buildings themselves. I'm a little less optimistic that there's many opportunities inside buildings because it turns out retrofitting a building, building to make it work for a data center that can actually handle the most sophisticated fire codes the weight of the servers yeah fire codes cool cooling like fiber all these things that you need it seems really hard not it might be some edge cases where it where it works but i think really hard i think if you can find uh maybe in suburban ex-urban parks like buildings that have parking lots and they have large interconnects and have headroom you could stick a modular data center like a prefab data center right like there are companies like armada or flexstone and others that are like in that market um crusoe etc and those can get right deployed and i'm i'm kind of interested in um going you mentioned ev charging you know we are seeing a good amount of money going into despite where we are these at this exact moment in this country although you know as one friend of mine just said the iran war might have been the best thing for climate that trump ever did uh but the uh you know if we if we start moving in the right direction on evs we we are building a good amount of uh level three chargers in this country but the utilization rates aren't particularly good uh i mean i think on average it's like 15 percent in the most used areas it's like 30 i mean that's a lot of opportunity those are really big interconnects and there's a lot of capacity there and so i've been talking to folks in that space and they're just looking at it but where it's going to get really interesting is well i think it's a big potential opportunity the way that those were set up is that with regulators and utilities is that there are um special ev rates that those facilities are on and the way that those rates were written you can't actually add something like a major additional load and still be on the ev rate so either they're going to have to make changes to the rate structure which the good news about that is that will be out in the public it'll be transparent there'll be a conversation it may take a little while um or i was talking to my team about this it may be possible to switch to opt out of the ev tariff just go on to a normal cni tariff and co-locate a data center and now you're no longer on the special tariff you're just on your normal tariff but you don't care because you're using so much power you're getting so much revenue that everyone's ending up better off in the end so i think that's an interesting i'm really intrigued about that space that we'll see how it evolves but there's gonna be some regulatory work that has to happen potentially but there seems like there's a lot of room for that and i think there's you know i think there are you know many gigawatts available capacity and only growing as we build out more and more l3 chargers and those l3 chargers are getting bigger right it's not they're like 250 kilowatt hours in europe i think now they're deploying 1000 kilowatt uh chargers we're going to see bigger chargers with larger capacity which might be good opportunity um for co-location and more utilization jigger what do you guys think about the residential model you it's uh the home battery provides backup grid services and also keeps the gpus online so it seems like you know you have multiple use cases that could come into conflict um what do you think about the actual use of gpus as part of a solar and battery system and is it a model that will actually tie enough to compute together anybody have strong opinions on it i think that there is a certain percentage of the residential model that will actually tie enough to compute but i still think that the vast majority homeowners are skeptical and just want their energy bills to be simple and be left alone so i don't think it's for everyone i don't like and i would be like to be proven wrong on that but i think my my gut is that the vast majority of people will not buy into that but i think the ones that that do is substantial maybe like 10 percent And that's a substantial amount, and you could do something with it. I question whether the interoperability of the grid is strong enough to make that work at the distribution level and what additional – as Jigar likes to say, these idiots out there didn't know how hard it was to build a power plant. They didn't know what they were doing at the hyperscale level. I question whether people know what they're doing at the aggregate distribution level. I think that's exactly right. I think – So much agreement, it's disgusting. I mean, so my client's the largest deployer of distributed data centers in the country and called Available Infrastructure. And the bigger challenge for me as a project finance guy is that what you find is for the five megawatt data centers and up, you can get three to five year offtakes really easily. So there's a ton of banks, others who want to buy that compute. When you start going towards the 250 kilowatt data centers, so we're rolling out 5,000 data centers at Crown Castle Towers right now, for instance, across the country, those data centers get their contracts from smart cities players. So think LG and IBM who control traffic lights or are providing services to Waymo or other things, right? But those are not ubiquitous. So… Yeah. Yeah. Yeah. Those have very high returns because they're selling their tokens merchant. But, you know, for folks who want long-term contracts, you have to wait for those smart city folks to win the local contract for controlling traffic lights or whatever it is before you get your long-term contract, right? And so part of the problem with residential is that you can certainly do it and it's a lot of money, you know, it's like $150,000 to put this thing in your garage. But you're getting merchant tokens, right? So it's not a long-term fixed price offtake. And so, like, part of the challenge I think we're going to have with these distributed data centers is the CTOs who are buying this compute have not gotten comfortable with data centers below five megawatts yet. Full disclosure, we work with Spam and my sense of the residential market, at least in their experience, is that… New construction is a lot easier, and being able to sell across multiple homes all at once, new cul-de-sacs. And also it's just part of the community, it's part of the kind of the deal. And I think what's interesting, at least with their putting forward, and then there are some other players that have not announced that they're doing this too, by the way, that I think is coming soon, they're going to provide free power to customers. Yeah. And so, like, you're never going to have to pay your power bill ever again. At the big data center level, we know that the hyperscalers have offered a lot of money to communities, and they're like, we don't want it. But I don't know. I think this is an opportunity, particularly in new construction. But in the short run, for retrofit, they're mostly going to CNI locations like telecommunications deployments, which is kind of, Caroline, what you said. Okay, let's wrap up with some post-election analysis. I think by now everyone is really sick of the phrase blue tsunami. Maybe we should be talking about, I don't know, the green undertow, something like that. No, no, Stephen. Oh, my. We saw a couple of big shifts on the state level. You know, people are waking up to the importance of local electricity regulation. We saw PUCs move in a clean energy-friendly direction in certain states. New crop of governors were elected with a more expansive view on leveraging DERs and influencing utility regulation. Jigar has said he's working with many of those incoming governors. So I'm just curious. You're up. We've had some time to digest the election. Any storylines emerging for you as ones to watch? Jim, what are the most consequential ones for you right now? I mean, certainly, everyone's eyes seem to be on Texas, which you guys have talked about before. And I think Governor Inos, the elect, is still putting her team together. And they're being pretty tight-lipped, from what I understand, in terms of what their intentions are. So I think there's a little bit of wait and see. There's some indications that, Jigar, I'm curious, your perspective, and Carolyn, given where you are, that it may not be as sort of as negative on the data center front as may have, you know, appeared and talked about, appeared during the election. I think that to me is a big question. In some of these moderate states, or not even moderate states, that, well, until a few weeks ago we thought of as not moderate at all, where you have governors that are going to have to govern to the middle. It's true that people don't like data centers, but it's also true that unions and trades and others like to build things and understand the economic development benefits of building data centers. So to me, it's, can you change the compact? We talked about this earlier. What's the new way forward? And can the governors in some of these states, like Texas, because it's such an important one, can they say, we're going to do it differently? Here's how we're going to do it, but then keep moving forward, you know, with certain amount of development. Because as Carolyn mentioned too earlier, if the alternative is the big canvases go overseas and all we're doing is distributed, like I'm all, I'm for distributed, that's the subject of the episode, but there's some limitations on what we can do, right, on the distributed side. So we still, I think, want to build those bigger data centers, but we just need to build them in a different way and take a deep breath and figure out what is going to be socially acceptable. So I think that, you know, Texas is interesting. I'm actually maybe the most interested in. places that where you've got the trifectas on the democratic side i think six states yeah like out of wisconsin or michigan like how where do they actually go like how anti whatever utilities anti-data centers um pro clean energy which again i would like not to see all those things kind of framed in that way um i think that's a big question for me is this i'm kind of looking at what's good and also can they get beyond 100 clean energy mandates right how creative will they get on the clean energy side rather than going back to yeah but i also think you know affordabilities is so important everywhere that everyone's gonna have to look at it in that context i think the arizona election by the way the last commission like undid their energy efficiency resource standard right just killed all efficiency funding in arizona i would like to think that this new commission's you know going to go in a slightly different direction um so my eyes are on that on arizona in that case as well jigger what do you think so i think that when you think about where the election went a lot of the voters who voted for the 457 new state legislators around the country were steve bannon and tucker carlson followers and so i think that when you think about how broad this coalition is it's not really a clean energy coalition it's more of a god damn it we kept telling you to focus on cost of living now freaking do it right and that's why i think utilities are in the crosshairs because they have not proposed anything that will get rates to inflation or lower and so it will be imposed on them i think a lot of it's going to be grid utilization a lot of it's going to be demand flexibility load flexibility a lot of the things that we've talked about i think on the data center side almost every one of these governors ran on a moratorium so there's almost nothing they can do except to provide some sort of safe harbor number but like it is impossible for governor-elect acton in ohio to come out when there's like i think it was 40 out of 41 county ballot initiatives basically are banning data centers above 25 megawatts right and so like there's no way that she can now say psych like here's another gigawatt data center right and so i just think that there's a level of understanding that everyone needs like i just think most people have not sat down i think with the election results and realize that we're in a whole new world right this is not modify your previous position slightly for this moment it's you know you have to show in visible ways that you care deeply about the affordability crisis that we sit in today and that's not solar wind that's not this and that that's just people like they don't care about ai versus china nobody cares about ai versus china especially after all of like black october and the ai the anthropic s1 like let them all burn to the ground no one cares well judging from the debate over the permitting bill i don't think progressive environmentalists have had have internalized that new world all right caroline you've had you know a week and a half almost two weeks to really think about the election you run policy for a major corporation what have you what do you what questions do you still have what have you figured out i don't know that i have any more questions and that have been than have already been articulated i think that where i am focused is actually less on the policies and regulations and more on what are the commercial structures that solve the political moment right and so i think that we as an industry have a tendency to go shop and lobby and defend our own book whether your book is renewables or your book is thermal or your book is storage or whatever it is and you can have a few trade groups that present a bridge against you know renewable general or clean general left general versus renewable general or clean general left general versus you know thermal on the left or thermal on the other side and stuff and i i don't think that's going to work anymore um so what i'm focused on and what my team is focused on right now and is being informed by the policy team is like what are the integrated commercial solutions that are going to work how do you look at up rates in the natural gas space but you know combine that with virtual power at the residential space how do you how do you make it easier to solve all those problems at once as opposed to going and you know arguing your piece of the political pie and i think the companies that go and argue their piece of the political pie and i will say that it's 99 of all companies in this space whether you are a supplier uh you know third party or utility we all just sort of go and argue our book are um i think if we do that it's going to be very unproductive and we'll end up with some very aggressive and potentially populist limiting um legislation and what we what i don't want to see is the energy community be tone deaf not present innovative commercial opportunities and we end up getting very short term fixes that look politically um astute but don't fix the market in the long run right and i think that's a big risk overall and so where i'm focused mentally and where i've sort of focused focused the growth team and focused everyone over here is like no let's let's be the company that innovates on the commercial offering and integrate something that that speaks to all parts of the problem here and you know if we can do that as an industry i think uh we could get some really great long-term change if we can't we're going to be fighting each other and we're going to get some pretty short-term aggressive um and potentially even regressive uh ships in the market absolutely urgent moment of creative thinking i think we're gonna have to close it out there caroline golin is the chief growth and policy officer of nrg thank you good to see you thank you stephen jigger shah is the co-managing partner of multiplier great to see you my friend of course and jim capsus is the founder and ceo of the ad hoc group great to see you jim thanks for joining us good to be here always fun to be with you guys that's it for the show open circuit is produced by latitude media the show is edited by me sean marquand and ann bailey you can find all of our episodes on apple spotify or wherever you get your podcast and you can get transcripts at latitude media.com check out all of our coverage there and sign up for our newsletter and of course you can get all of our episodes on latitude media's youtube channel so if you like the show make sure you hit subscribe thanks so much for being here i'm stephen lacey we'll catch you next week

Podcast Summary

Key Points:

  1. The AI infrastructure boom, driven by early research on scaling compute, has led to massive investments in large data centers, but faces growing political and environmental resistance.
  2. Distributed data centers—smaller, decentralized facilities in homes, offices, or commercial buildings—are emerging as a potential alternative to massive transmission-scale data centers, enabling greater grid utilization and local energy integration.
  3. Recent controversies surrounding NVIDIA’s leadership, including personal and national security allegations, have sparked broader questions about corporate accountability, trust, and the sustainability of AI infrastructure investments.
  4. Utilities and regulators are under pressure to rethink grid management, with new political leadership emphasizing affordability, demand flexibility, and the use of existing infrastructure, such as EV charging and virtual power plants.
  5. Residential and commercial co-location of AI compute with energy systems (like solar and batteries) offers potential cost savings and grid support, though adoption remains limited by skepticism, lack of long-term off-takes, and technical challenges.
  6. The political landscape post-election is shifting toward stricter local controls on large data centers, with many states imposing moratoriums or limits on facilities over 25 megawatts, forcing a reevaluation of infrastructure deployment models.
  7. A major trend is the move toward integrated commercial solutions—like virtual power plants—combining energy storage, EVs, and smart devices to deliver real savings and grid resilience without relying on isolated, high-cost projects.
  8. The viability of distributed computing hinges on regulatory transparency, power quality management, and the ability of utilities to deploy real-time monitoring tools to manage distributed loads safely and efficiently.

Summary:

The energy transition and AI infrastructure are now deeply intertwined, with a shift from large-scale, centralized data centers to distributed models that leverage existing buildings and homes. This shift is driven by both technological feasibility and political pressure—many states, especially after recent elections, are imposing strict limits on large data centers, prompting a reevaluation of where and how computing is deployed. The core idea is that underutilized grid capacity, such as in commercial buildings or through EV charging networks, can be repurposed for AI compute, improving efficiency and affordability.

However, challenges remain, including power quality issues from spiky inference loads and the lack of long-term, stable off-take agreements for smaller facilities. While residential models face skepticism due to cost and complexity, they offer potential for community-level savings through virtual power plants. The episode also highlights the fallout from NVIDIA’s leadership crisis, which has further eroded trust in AI infrastructure investments and raised concerns about corporate accountability.

In response, utilities and industry players are pushing for more transparent, integrated, and community-focused solutions—such as combining smart thermostats, EV chargers, and batteries—that serve both consumer savings and grid stability. Ultimately, the future of AI computing may not lie in bigger, more expensive campuses, but in smarter, decentralized, and more socially acceptable models embedded in everyday infrastructure.

FAQs

Distributed computing involves small-scale data centers, typically under 25 megawatts, located on the distribution grid. Unlike massive, centralized data centers, these are smaller, more flexible, and often placed in commercial buildings, homes, or substations, enabling closer proximity to end users and better utilization of existing infrastructure.

Homeowners can host small AI compute nodes using existing solar panels and battery systems. These setups allow homes to run inference workloads while using stored energy, creating a financial stake in AI and contributing to grid flexibility through virtual power plants.

Distributed data centers can significantly improve grid utilization by using underused capacity in existing buildings and infrastructure. This reduces peak loads, lowers electricity costs for consumers, and increases the overall efficiency of the power system without requiring massive new grid upgrades.

Inference workloads, which are common in AI applications like chatbots or image recognition, are more viable in distributed settings due to their lower compute intensity and proximity to user devices. Training workloads remain largely unsuitable due to their high compute demands and energy needs.

Post-election shifts, especially in states with anti-data center sentiments, are likely to restrict large-scale data centers. This could accelerate investment in distributed computing as a more socially acceptable and politically viable alternative to traditional mega-data centers.

VPPs aggregate flexible energy resources like batteries, EVs, and thermostats from homes to provide grid services. When combined with distributed computing, they enhance system resilience, reduce energy costs, and offer real-time load balancing across communities.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.