How Data Centers Actually Work

Hey Lauren, how you doing? I'm great. It's so nice to be back in studio with you again because our schedules were not aligning for the past few weeks. No, but the stars in the moon have aligned now and here we are once again. Here we are and I'm sure all of our listeners have just been sitting here wondering, when are Lauren and Mike getting back together? When is the band getting back together? I just went to yet another AI dinner last night. Everyone's talking about AI. Everyone is talking about whether or not we're in an AI bubble and I think a lot of this is fueled by the news that we've been seeing trickling out about all of these AI infrastructure projects. Yes, data centers. Data centers. Big warehouses stuffed with computers. Stuffed with server racks. For sure, we're going to be touching on all of that on today's show. But before we get into it, we have to welcome our guest, Wired's senior writer and climate energy expert. Molly Taft is joining us today. Hello Molly. Hello. Hello Molly. Is this your inaugural Uncanny Valley podcast? Yeah, and I'm a fan. So this is a big deal for me, frankly. I'm very excited to be here talking about my favorite topic and the thing people ask me the most these days about it parties. So. And what is that? Data centers. Data centers. I thought you were going to say they're electricity bills. Yeah, I mean, they're related, aren't they? People are fired up about it. [MUSIC PLAYING] This is Wired's Uncanny Valley, a show about the people, power, and influence of Silicon Valley. Today, we're talking about the AI infrastructure boom. In recent years, tech giants like OpenAI, Amazon, Meta, and Microsoft have invested hundreds of billions of dollars in data centers. These are large warehouses full of servers, delivering the huge amounts of computing power needed to run AI models. Recent announcements like OpenAI's new Stargate Data Center in Texas, or the AI startups deal with the chip maker AMD, have only added to the hype and the capital surrounding the expansion of these data centers. But just as the deals keep piling up, the issues around this rapid expansion are becoming more worrisome. We'll dive into how these data centers work, what effect they are already having in some communities, and what they reveal about the current state of the AI industry and our economy. I'm Michael Calorie, director of Consumer Tech and Culture. I'm Lauren Good. I'm a senior correspondent. And I'm Molly Taft, a senior writer covering energy and the environment. So I think it's safe to say that all of us, including our listeners, have heard of data centers somewhere in the news, and they have a pretty good idea of what a data center is. But let's talk about why AI companies rely on them. So to put it another way, how does a chat GPT query on my computer end up in a data center? This is an excellent question. And I think we're going to nerd out a little bit. Awesome. OK. Say you type in a chat GPT, but you want a few options for dinner recipes tonight, or what am I going to get Mike for his birthday next year. Bless you. You send your request through, and then it goes to Open AI servers. It goes through a few checkpoints first, like there's authentication. Are you a valid user? Moderation is the prompt OK. Or aligns with their guidelines. And then some load balancing, which data center should ultimately handle this. The words are the strings of text that you write are broken down into these little chunks of text called tokens. They're kind of like these puzzle pieces that artificial intelligence models can process. And at this point, the request lands on specialized hardware, in most cases, GPUs. Yeah. The next part is key GPUs, which stands for Graphic Processing Units. Are essential to AI systems. They're electronic cards that are really good at parallel processing, which basically means they can perform many calculations at once. Data centers are full of metallic rows of servers that contain these GPUs. So once you send your query in, it gets processed by those GPUs. You've probably heard some of the most famous ones in tech headlines, like Nvidia's H100s. Ah, the H100. The kids can't get enough of them. My old friend. And then I guess to bring it back full circle to our query for what's each for dinner tonight or Mike's birthday present next year. Once that query arrives at the data center and the servers, the AI model starts working. And this is called the inference time. The model is going to predict what words in the form of tokens should come next, one after another, until it builds out this full answer. And then finally, the response is sent back through the same network path to your browser or app. This is a very summarized version of what happens. But on a high level, that's why your chat to BT question or query needs to go through to these data centers. This all happens in a matter of seconds. It's pretty miraculous. Yeah. Well, I love the breakdown. It's also just, you know, for the record, I have to say that I probably would never ask chat to BT for recipe recommendations. And Lauren, I really hope that you get a good answer as to what to get me from a birthday, because I'm already very excited. You might recall that I did this last year for your birthday, and I think I still haven't gotten you anything. Well, you know, I think if there's a next question that everybody has about data centers, it's about how energy intensive they are. Because even a simple query like that, I mean, we're talking about accessing, you know, multiple different servers. You're breaking up your query into different tokens, sort of spraying them into this data center. How much impact do those computing requirements have on the environment? And Molly, I want to ask you this question because I know you've reported on this a lot. What does the energy consumption of a data center look like? And are we talking about just energy? Are we talking about water? What does it look like? So large data centers, there's a lot going on inside of them, right? They need cooling systems. They need to keep the lights on. They need to run their network equipment. That all creates a lot of energy. They don't run at the same energy level all the time. They often run in cycles based on when there's more queries. You know, they'll take a break at night when the queries are lower. And the environmental footprint of a data center also depends on what kind of energy they're hooked up to. If you're hooked up to a dirtier grid, a grid powered by fossil fuels, you're going to create more emissions, a data center that runs on solar power or wind power will have a lower environmental footprint. You know, it's funny trying to calculate the footprint of a specific data center because a lot of the information about the environmental impacts is proprietary information. So a lot of what we're working with is stuff that companies volunteer themselves. You'll see headlines, you know, meta is building a data center. They're calling Hyperion and Louisiana. It's part of their bigger build out. And that's going to be huge. It says it's going to be about five gigawatts, which is like massive. That's about half the peak power load of New York City. That's the biggest one under construction right now. But there's a lot more out there. There's also regional level approximations. And there are some places in the world that are starting to kind of get worried about how much energy data centers are consuming or are set to consume. In Ireland, right now, data centers use more than 20% of the country's electricity, which is crazy. Wow. Virginia is also facing a huge cliff and they're projected to use a lot more over the next couple of years. So as a group, there's definitely an energy trend that's going up. I mean, you mentioned that each tech company, you know, data center company is reporting their energy usage. I'm curious how far down the stack they're going. You know, for example, if they are using a certain number of GPUs and those components from GPUs have to be shipped from different parts of the world or manufactured in different parts of the world, that is also an emissions cost. So are any of them incorporating that? Yeah, I mean, the problem with climate stuff and the problem with emissions stuff is you can keep going down this rabbit hole forever. Kind of what you're talking about is gets into like emissions reporting, which is like, what is the total environmental footprint of what you're doing? And this is something that climate folks and people who like to do math on emissions are constantly trying to figure out when it comes to specific companies and many companies don't necessarily want to go down that chain. But I think when we talk about power use, we are talking about like on-site power use. We're talking about just what it's going to take to turn that thing on and to run that thing. The total footprint of these things is probably a lot bigger than we think. And when you mention people who like to do math and figure this out, I think of someone like Sasha at Lucia-Honey, who, you know, she's pretty vocal out there. She's the climate lead at hugging face. And she tends to put it eloquently, call bullshit and a lot of the numbers that these tech leaders put out there. Yeah. And Sasha's been really good at calling out, especially when you have models where it's less transparent as to how much energy is actually going into a query. There's numbers that kind of get tossed around. And Sam Altman put out a blog post this summer where he said the average query for chat GPT uses about what an oven would use in a little over one second or a high efficiency light bulb would use in a couple of minutes. It's about 0.34 watt hours of energy. But Sasha really correctly points out that these kinds of figures don't give us a lot to work on. Like when he says an average query, what does that mean? How many of those queries are happening? What are these data centers hooked up to? Are they hooked up to grids that are mostly renewables? Are they hooked up to grids that use coal? Like there are so many factors that go into the energy use of these products. And Sasha put it-- really, really well. She said that Altman pulled that figure out of his ass. That's a straight quote. And, you know, she told me like, it blows my mind that you can buy a car and know how many miles per gallon it consumes. Yet we use all these AI tools every day. We have absolutely no efficiency metrics, emissions factors, nothing. So just one number doesn't help us get to the bigger picture. And a lot of the times it may just kind of help companies procrastinate on putting out more data. So we don't have any clear indicators about how much energy each query is taking and how much energy these data centers are consuming other than it's a lot and it's a problem. And even knowing that it's a problem, a lot of the big tech companies that are investing in AI infrastructure are continuing to build, build, build. So we've talked about some of the big players and we should make sure that we mention as many of them as we can. So who are the other big names in this space? Honestly, I think the biggest ones are the folks we've already mentioned, which is OpenAI, AMD, Nvidia, and the Stargate project that's kind of an umbrella term for a bunch of different partnerships. Like the Stargate project is a $500 billion 10 gigawatt commitment between OpenAI, Softbank, Oracle, and MGX. All of these interconnected relationships between the different hyperscalers and the chip makers. What's interesting is that there has been a little bit of a shift in kind of the terminology or the way that these investments are being described and that they're being described as gigawatt investments. So they're kind of staggered investments in many cases. They're kind of based on the assumption that the man for AI will continue to scale upwards and that eventually more and more energy or compute power, I should say, is going to be needed. It's not some kind of clear cut. Well, in exchange for this amount of investment, this company is going to get a million number of GPUs. It's not basic unit reporting. Okay. So there's one thing that you said there that I want to hear more about and that's hyper scaling because it sounds like what we're already talking about is hyper scaling. You're saying that there are companies that are interested in growing even more than what we've already talked about. Well, they're all interested in growing more. Who among us, Mike? But the hyperscalers refers to this class of major tech companies or cloud service providers. So meta, Amazon, Microsoft, Google, they're all in that category. Yeah. And I think it's important to remember that these companies have so much money and they have an ability to raise capital like nobody's business. So they're able to do some really crazy stuff to build quick and to build out really, really big. And they're getting pretty creative because their goals right now are to build these things quickly and get them up and running so they can basically use this physical infrastructure to compete with each other. I think that's right, Molly. I think there's a lot of frenemy building happening right now. And I would just love to be a part of their group chats when all of these announcements are being made. Yeah. And speaking of frenemies, the other sphere of influence that these companies are operating in is the political sphere. Obviously, in order to build a giant data center somewhere, you need to have the political will to do it, which means you need buy in from the local residents, the local government, the state, the country. So what's happening in the political sphere with folks who want to build more data centers and people who oppose it, regulation, how is that playing out? That's a great question. And I think if you look at the national conversation, it's quite different than what's happening on the local level. So you know, you have Washington, you obviously have an administration that is very friendly to the idea of an American AI empire. Importantly, for the energy conversation, the way that the Trump administration has approached the support has been through support of fossil fuels. They would really like for all data centers to be powered with oil and gas, a little bit of nuclear and coal. And this works out great for those industries as well, right? If you're going to have this massive expansion of power demand, it's really cool to be in the middle of that and be the one that everyone wants to turn to for energy resourcing. And then on the other side, there has been this influx of local opposition to these data centers for a variety of reasons, be it the water use, be it fears about rising electricity rates, be it noise, and some of the really big struggles have kind of catapulted this issue to national conversation. I'm thinking about X AI in Memphis when Elon Musk wanted to get X AI open running. He installed a bunch of unpermitted gas turbines in order to get X AI working that he installed in a majority black community in Memphis that already had severe issues with air pollution and asthma. And those folks made themselves known earlier this year, there was an attempt in DC to impose a moratorium on any state regulation around AI at all. It was an incredibly broad inclusion in the big, beautiful bill that ultimately didn't succeed. But one of the people who opposed it publicly was Marjorie Taylor Greene, who actually mentioned data centers in her opposition and she compared AI to SkyNet, the fictional AI from the Terminator movie franchise. So this is getting some strange bedfellows, you know, in league with each other. I think this kind of contrast between what the administration is trying to push forward and some very powerful energy companies that stand to gain from it versus like some truly grassroots local movements and people concerned about the impacts of what these things are going to do in their communities. We're going to take a quick break right now, but when we come back, we're going to dive into why AI companies aggressive bed on scaling might come back to bite them. So stay with us. When I listen to the news, here's what I want to know. Why this story matters. Who's at the center of it and how the reporters uncovered it? And as a journalist, I want to make sure that's what you get to. I'm Ella Hayes-Addy, co-host of the podcast post reports. Every weekday, my colleagues and I at the Washington Post give you the context you need on the biggest stories, health care, tariffs, artificial intelligence. We've got you covered. Look for post reports wherever you listen to podcasts. Welcome back to Uncanny Valley. And Lauren and Molly, we just talked about why data center investments have been ramping up and what the known impacts are. So now comes another key question. Is all of this aggressive expansion even a good idea? It depends on who you ask. AI founders will typically tell you that there's nothing to worry about. And this is necessary to meet the current and future demand for AI. But the big elephant in the room is that for all of this investment, consumer spending on AI is not there yet. And some of the frontier model companies have seen a significant amount of their revenue coming from enterprise developers who are actually building for their companies and using AI for that reason. I have to imagine, and this is just my own theory, that unless we become a world where consumers essentially are developers, where consumers are using AI to the point where we're like all building things and writing lines of code based on this, that I don't see how the revenues get there. I also found this really interesting. The economist reported that AI hyperscalers, which are the cloud service providers I talked about who are the largest spenders on AI, are using some interesting accounting tricks to try to depress the reported infrastructure spending, which then of course has the effect of inflating the profits from that. So I think that's partly why there's a lot of people concerned that there could be an AI bubble. It's just it's about supply and demand. And right now we are spending so much money as a nation on supply. And just hoping that the demand continues to grow. Fingers crossed. I think when we talk about the demand for this, you have to understand that behind it, there's a lot more going on than just accurately reading the tea leaves. There was an incident in the late 90s and early 2000s where there was all of a sudden this narrative about how much energy the internet was going to use. And there were a bunch of high profile articles that made some claims about how by the 2010s the internet was going to use half of the US's electricity like the internet was posed as this huge electricity suck. And we were going to need to make more coal fired power plants to deal with it. And there's a really smart researcher named John Coomy who dug into this. And you know, he found that it was really being pushed by the industries that stood to gain from such a build out. And of course, that didn't end up happening, right? Like efficiency gains were great. The internet doesn't actually use that much energy. Like, I think we're in this weird moment where, you know, US energy use has stayed pretty stagnant. We're getting better at using what we have. And we're getting better at being more efficient. And all of a sudden, there's the possibility for a big customer of various industries, not just energy, right? And so it's hard to tell if we're in a bubble when there are so many people who stand to benefit from people believing that this is really going to happen. And to that point, I mean, when you make this kind of investment in infrastructure, it's a very fixed investment. And you're basically doing it under the assumption that the way the compute happens will remain the same for a certain period of time. And actually, what we're seeing is that some of the more computationally intensive AI models could soon potentially offer diminishing returns compared to smaller models. So for example, like a frontier model company, like OpenAI, is currently much better, smarter than a smaller model from an academic lab. But that may not always be the case. There's a lot of interesting research going on. There are new alternatives to deep learning. There are novel chip designs. There's new approaches like quantum computing. And I think what we saw with deep seek out of China, and it's remarkably or reportedly low-cost model this past January, that's already served as kind of a reality check for the AI industry. What do you think is something that we as citizens, as people who have the decision whether or not to use AI tools? And we have decisions to get involved in whether or not data centers are going to come to our state or come to our county. What can we do? I think about this all the time. You know, I'm going to assume I'm going to go out on a limb that most people listening to this are nerds and enjoy reading and learning. And one thing I'm so much fun at parties, one thing I love to tell people at parties is not even get involved, but to learn more about how their local electric utility works. Because that's, you know, if a data center comes to your area and impacts your electric bill, it's because of your utility. And utilities in the US are set up in this really, really strange way. And a lot of them are investor-owned, so still have to make a profit for themselves. But they also control how much money you're seeing on your electric bill every month. And so I think this kind of problem can feel so big and so outside your control. But the more you dig into the sort of wonky side of things, the more you will find people who are organizing around even just basic asks to utilities, to use more renewable energy, you know, to monitor kind of rate hikes. And they will be the ones that will be up on if a data center is coming to town, what it might be doing to your electric bill. So I would encourage people to read more about the fascinating topic of electric utilities in the United States. That's really great. Yeah. Molly, you'll be happy to hear that a few weeks ago I was down in Chandler, Arizona for a tour of Intel's new fabrication plant. And there were a handful of us who were, you know, considered maybe mainstream tech journalists there. But then there was one woman there from the paper in Arizona who was paying very close attention to power and utilities and water usage in the area. Yeah, she's the real one. Honestly, read her stuff. Yeah. My piece of advice, it's not related so much to energy usage or your own usage of AI. But I would just say double down on the humanities. I don't think AI is going away even if we're in a bubble, even if that bubble bursts. The way that it looks may change in the next few years. But I do not think it's going away. And I think in the future, I want to believe that what will make us stand out as our ability to think and our human relationships, our human capital, if you will, and our ability to appreciate real human generated art. And so that's what I've been doing. I mean, I just spend a lot of time when I'm not on the internet reading books written by humans and watching films, reconnecting with family and friends. So that's my, I don't know, I guess my small active resistance. That's great. Say, I'm honestly. Yeah. And I would say that you do need to understand these technologies in order to have an opinion about them. And in order to understand them, you have to engage with them. So I would encourage you to dabble a little bit, but don't use AI like a monster. So don't say thank you to the machine because we know that people typing in thank you at the end of a query or after they get an answer is like burning even more resources. So just understand the technology enough so that you understand why it's important to have an opinion about it and you help you form your opinion about it. And also like push back on AI features being crammed into things where they don't need to be, you know. So if you don't see the utility in the AI features on your phone or in your car, like figure out how to turn it off because then it's like turning off the lights when you leave the house. All right. Well, let's take another break and then we'll come back and we'll do our wired tired, our new little experiment. What does the AI revolution mean for jobs? For getting things done? Who are the people creating this technology? And what do they think? I'm Rana Elkeljubi, an AI scientist, entrepreneur, investor, and now host of the new podcast Pioneers of AI. Think of it as your guide for all things AI with the most human issues at the center. Join me every Wednesday for Pioneers of AI and don't forget to subscribe wherever you tune it. Welcome back. Lauren and Molly, we are now going to be diving into a new segment that we're calling wired tired and there's a slash between those two words. It's wired slash tired. If you're a long time wired reader, then you will know that the wired, tired format is a big part of our brand. So whatever is new and cool is wired and whatever pass a thing that's replacing is tired. And the wired and tired don't have to be related to each other, but it just rhymes better and really rings the bell if they are related. So we're going to try it here on the show and we're going to talk about what is wired and tired in our lives. Are you ready? We're all tired. I'm exhausted, personally, but thrilled about this. Well, let's start with Lauren. Lauren, tell us what is wired and tired. Okay, tired to me are extravagant coffee drinks and wired is just like go for the simple drip coffee. And what's your drip coffee? It's pumpkin spice season, everyone. And I have to say I'm seeing the signs for not only pumpkin spice, but weird twists on it and now it's like Dubai chocolate pumpkin spice lattes and where are we going with us? There's no end. So just stop it now, shut it down and just go for either a pour over or drip or maybe if you're going to get a little fancy in Americana, I love a good Americana. I put a little half and half in mine and sometimes it's sweetener. Okay, so tired, tired PSL, wired drip, be a drip. Be a drip. Yeah, that's it. All right. Thank you. Molly, your turn. Tired is looking at my phone. I don't want to do it anymore. Yes. Frankly for anything. I don't care. I've, this is a public apology to all of my friends. I haven't answered texts in months because I don't want to look at my phone. So that's, it's very tired. I'm very tired about it. Wired, new thing called books. I love to read them. And my recommendation, the book that I read earlier this year and I have not stopped thinking about is a novel called Sky Daddy by Kate Volk. It is set in the Bay Area. So it ties in and it's about a woman in love with planes and I'm not spoiling anything. It's a little not safe for works. I don't read it at work, but it's like, honestly, one of the funniest books I've read in a very long time. Nice. Wired books, tired phones, solid. Great. Mike, what's yours? My wired is hydration tablets and my tired is plain old water. Oh, I love how we both want it beverages. Yeah. Okay. I drink water at meals, but for those times in between meals, when I'm doing the thing and hydrating at my desk, I've gotten really into hydration tablets and hydration powders. So these are the things that basically turn your bottle of water into a gatorade. It gives you electrolytes. It gives you some flavor, a little bit of sugar. So they have caffeine in them now, which is amazing. Your eyes just lit up to change. Oh, yeah. Totally. So it's like, you know, you get them at like the sporting good store. You can get them on the internet or whatever. They're sold as like sports nutrition, but really they're great because they get me excited about drinking the water that's in front of me too many times. I'm like filling up my water bottle and then just forgetting about it. And then I come back, you know, like in four days into the office on a Monday and unscrew the cap and it tastes like pennies. Nobody wants that. So put a hydration tablet in your water bottle and suck it down and love it. I like noon. Those are good. And they come in handy. Those are really good. I like it. They do. Yeah. Yeah. Lots of single use plastic. Awesome. Also, liquid IV is good. Liquid IV has a firecracker popsicle flavor that is my household's favorite. We need PSL flavor electrolytes. Yes. No. Thanks for listening to Uncanny Valley. If you'd like what you heard today, make sure to follow our show and rate it on your podcast app of choice. If you'd like to get in touch with us with any questions, comments or show suggestions, write to us at [email protected]. Today's show is produced by Adriana Tapia, a Mar-Law at Macrosound, Mixed This Episode. Mark Lytte is our San Francisco Studio Engineer. Pran Bandia is our New York Studio Engineer. Kate Osborne is our executive producer. Katie Drummond is Wired's Global Editorial Director and Chris Bannon is Connie Nass' head of Global Audio. (upbeat music) I'm CNN tech reporter Claire Duffy. Claire Duffy was one of the things. I cover artificial intelligence and other new technologies for a living. And even I sometimes get overwhelmed trying to keep up with it all. So I'm starting a new show where together we can explore how to experiment with these new tools without getting played by them. It's called Terms of Service. This technology is so crazy powerful. Follow CNN's Terms of Service wherever you get your podcasts. From PRX.