How "Junk" Data Boosts Data Center Energy Costs

(upbeat music) This is Clean Energy Today, which provides interviews with clean energy leaders, innovators, businesses, and consumers shaping the clean energy transition. I'm your host, Lisa Kohn. Whether you're a consumer who hoards old photos or a company that hangs onto unusable data, you're boosting data center's energy costs. By focusing on eliminating junk data and improving efficiency, data centers can slash power consumption by 40% to 60% according to Simon Ninen, Senior Vice President, Business Strategy Hitachi Vantari. How can data centers achieve this kind of savings? Nina provides details. Can you begin by defining junk data and explain how it affects energy use? - Sure, yeah. First of all, I think, if you don't mind me setting the stage a little bit, right? The energy problem in data centers is fundamentally, I think, I would describe it as a data problem, right? It essentially, what's happening in data centers is energy is used to power where the data is stored, how the data is moved, and how the data is used, right? And so essentially, if you think of it, if data centers are needing to power servers, storage devices, and cooling systems, the more data you have, the more hardware you need to have to run for longer periods. And so this is basically the overall power consumption, carbon emissions, all of this goes up. And so this is where junk data becomes a problem. Junk data is basically data that is not useful, right? It is excess data. It is data that is beyond what organizations actually need or can derive value from. There's a very interesting, I can qualify it a little bit for you. There's an interesting Deloitte study that actually said that if, you know, for the companies that they analyzed, only 37% of data is usable once analyzed, which means more than 60% of the data being collected and stored by an organization is not useful. It is still being stored, still potentially being moved, but not actually useful for what the organization is trying to do on a day-to-day basis. - Can you explain why organizations would collect this extra data? Is there an example? - Yes, I can maybe, there's several reasons why this may actually happen, right? This, by the way, you can think of both individual practices and organizational practices. As part of day-to-day business, people may create data for any reason, right? And data is created by individual people, but data is also created by machines, right? In the world of physical AI, you're actually collecting a lot of information. There's a pretty stunning statistic that I've been quoting over the last couple years, which is self-driving cars are going to become a thing, right? They're already a thing, but it's going to become an everyday, every person kind of a concept, just in the matter of years. Every self-driving car on a single day generates four terabytes of data. Now, four terabytes is not a small amount, right? It's a pretty sizable amount of data, and that's every single car, every single day. And for regulatory reasons, you can't actually delete most of that data. You have to store it, right? You have to keep it in your records, because if someone needs to do an investigation, right? And say the car took some action at a certain point of time, some accident was caused, some driver made a mistake. The only way to prove that is to go into the system and actually find the data. So for regulatory reasons, there's actually a period of time for which the data is actually retained. So there could be compliance reasons, but there could also be just, let's just call it digital practice reasons. People don't delete photos and documents and so on for reasons of emotional attachment, or because, let's just say, it's a just in case mentality, you know? Someday you need to retrieve that data, right? Sometimes you don't organize it or manage it or delete it because there's so much information, so you're overwhelmed by the scale, and so they're just paralyzed by the thought of sorting and deleting data. You allow clutter to accumulate, right? And so for many of these reasons, both individual reasons and organization reasons, this data tends to be created and then propagated, yeah? - Ah, so this is something that maybe I do on my computer. - Sure. - And that affects data center energy use. - That is 100% correct, right? Where you are storing your data, right? Essentially, you can do it on your own system, but ultimately organizations are backing it up in their local data centers, right? If you're doing this in the cloud, what data centers are doing is they're becoming massive, massive storage centers for the data that is being created by people. Every time someone uploads or takes 100 photos on their phone, some of it is being automatically backed up, some of it is being transferred in ways that people want to store the stuff. And on a single day, right? You're creating so much data. This is why we're talking about gigabyte scale, gigawatt scale data centers, right? These are, you know, massive investments, not just in terms of size and space and energy, but just storage quantity, yeah? - Right, so that's what data hoarding is. That's what I do with my photos. - That's correct. - Really interesting. So what can be done about it? You talked about leave no trace policies. - Yeah, so leave no traces. You can think of that as being a generally good practice kind of a term, right? Just like, for example, if you and I go to a park and you want a hike, right? The leave no trace policy basically says, you don't litter, you don't leave anything behind, you take what you brought, right? And you extend that concept to how you manage energy and carbon emissions. Essentially, leave no trace in data center design means developing and operating data centers in a way that minimizes environmental impact, reduces waste, and essentially what you're doing is ensuring sustainability throughout the entire life cycle, right? So the core idea here is stewardship, environmental stewardship, and making sure that you are efficiently using resources, lowering or minimizing your carbon footprint, limiting any unnecessary data and infrastructure expansion. And some of that you can do through, you know, actually there's multiple lines of defense, right? But some of that you would do manually and thoughtfully through policy. Some of that may be automated through organizational practice, right? In a sense, I think when it comes to data, people have been designing data centers based on an assumption of massive explosion of energy needs. We're actually saying the energy needs don't have to be that much if organizations get better with data. Don't assume that the data problem is just gonna be what it is. You can actually make data a lot more efficient by reducing junk. And when you do that, then you're also able to reduce energy needs, yeah? - Now, how do you think that this could be implemented? It would it be a policy issue? - I think there's a few things that you can do, right? I mentioned the term, you know, using multiple lines of defense, right? From a data center design perspective, first of all, I'll just talk at the data center level and then I can bring it back to like the data level itself, right? On one hand, you can adopt high density, energy efficient equipment. You can use renewable energy for operations and cooling, but also when it comes to then to data, you want to regularly audit and delete any unnecessary data. You want data stewardship, right? That goes into essentially looking at the policies of what data is actually being touched. Does it need, you know, where does it need to reside? Does it need to reside in high energy consumption storage or in some backup ways, for example, where essentially it's cold, right? And, you know, it's not really actually touching energy as long as it's needed. What is the energy, what is the data that needs to be made primary versus, you know, secondary, right? And so on. These are policies that organizations can institute, right? And then regularly do checks and cleanups. In addition, I think, you know, it's a combination of how you manage your data, which is bits and bytes, zeros and ones, but also how you manage your hardware, right? E-waste recycling programs and all of these things. Essentially, these are design principles and actions that essentially have to go together, yeah? - And so this would be for data centers. So there's obviously an incentive to do that because they wouldn't use as much energy. - That is correct, exactly right. You know, and I think the linkage between data centers and the enterprises that are actually generating that data needs to be better, right? Like the, there has to be an assumption that, you know, the data center design prioritizes the efficient use of data, right? And the efficient storage and movement of data. By collaborating between data center providers and operators and the people that are actually generating the data, you can actually advise people, right? On how to actually do these things in a way that is more efficient, yeah? I'll just blow that up a little bit, right? It may be financially beneficial for data centers to store more data, right? But the meaning, the more data an organization creates and places with the data center, like they're making more money, right? You say, okay, this is a financial consideration. But if you're doing this in a responsible way, you'd actually go back to your user, to your end customer and say, I can help you reduce costs. I can help you reduce space. I can help you reduce power consumption and carbon footprint. And here's the way you can do it, which means you don't need to spend as much on me. I can make your footprint here as efficient as possible. That is a different way of working, right? And I think that would be part of what, you know, the way companies like Hitachi and Hitachi Ventara operate with that kind of a mindset, yeah? - Really interesting. Is there any kind of software that can help with this or any kind of programs? - For sure, yeah, good. Sorry, finish your question, please. - Well, you talked about storing it. I got the sense that you're talking about maybe siloing it so that the stuff that you don't really need is off to one side and is not causing an increase in energy consumption. - That is correct, yeah. And so there's many ways to do it, right? So I'll use the example of Hitachi Ventara because, you know, this is now our core business, right? First and interesting data point. Yeah, call this bragging a little bit, but I'll just state it. You know, we have Energy Star that provides ratings of different kinds of electronic appliances and they do that for our storage equipment as well. In the top 10 storage devices, and they use a metric called IOPS per watt, which is basically, think of it like miles per gallon. It's performance and the efficiency of power consumption to be able to deliver that performance. Hitachi Ventara holds the number one, number two, number four, and number seven position out of the top 10. And the reason is we have hardware that is designed to be energy efficient, but the software or the algorithms on top of that, that manage the data storage and the data movement in a way that is highly efficient. It, for example, takes into consideration usage. It takes into consideration peak usage times or peak data floor times and how you can optimize, you know, the use of energy and power to go into sleep mode or to divert into two different tracks so that, you know, you're being as efficient with data as possible, which is then translating into being as efficient with energy as possible. So that can happen in the storage device itself. But I think to the question that you're asking, there is additional software that you layer on top of that, call that intelligent data management software that can help with things like governance. And governance is basically saying what data sits where and who has access to it and how frequently is it accessed, right? But also more technical concepts, for example, like, you know, Turing, which means, you know, it's not just the location of data, but it's also saying which data is more easily accessible at short notice because it is highly used versus data that is less used and only just needs to be available in backup. It can be put in quote unquote cold storage, right? So yeah, there are software that can absolutely be able to do that intelligently, right? And take that decision making maybe even out of the hands of humans so that, you know, you can be more efficient. - Interesting, so this is hardware and software that the data centers would buy as they're building or can they add it? - Yeah, so this is really interesting, right? When you think of data center providers or data center operators, usually, you know, the work is divided between what's in the rack, right? And that is usually storage, which is my business or our business, and then you have computer, the servers, and then you have the network. But that's basically the technology in the rack. Then you have cooling and then you have energy and then you have data center operations and so on. Very often, data center providers do not really look that much in the rack itself because they are managing a big infrastructure operation kind of around the rack. They would allow, for example, a customer to rent out space or rent out a rack within the data center and say it doesn't matter what technology you want. You can choose whichever, you know, storage provider you want or the servers you want, and we're gonna allow you to rent that space in the rack, right, in the data center. Some are more prescriptive and say, we will offer you the technology, you can bring your data and put it over here. Either way, wherever the decision point sits, the companies, which means, let's talk about any end user, whether it's a financial provider or a retail provider or a manufacturing, anyone who needs the data stored can make a decision around what is the technology that I want to store my data on, right? Which data center do I want this stored at? And then work with the data center provider to make sure that the right technology is placed in the right place, right? So there's multiple points of decision making, but we, like, let's take a company like Hitachi Ventura, we would advise our customer to say, work with us because we can deliver the most efficient choice for you. And we would work with the data center to say, our storage or our technology is the most efficient so that we can co-promote or cross-promote, yeah, if that makes sense. - Right, so when you talk to data center owners about this kind of thing, is there a way to give them a feel for what their payback and energy would be and energy savings? - Yeah, I think this is a, let's just call it an emerging skill set, right? But we certainly have calculators that are able to do that, that basically say, if you compare one technology versus another, you shift workloads from one kind of rack set to another, here's the amount of savings that you can deliver both in terms of compression of data, right? And the reduction of how much storage is needed, but also then the overall power consumption and the subsequent impacts overall to the data center. So yes, we absolutely have the ability to provide that kind of information. - Is there any kind of range of payback in doing this kind of work? - Sure, yeah. Listen, I think just, again, state, you know, stats from my own company, but we're able to, we generally are able to go to a customer and say, by choosing our technology versus the average or, you know, a competitor piece, you can save anywhere between 40 to 60% in terms of power consumption or more, depending on how your data works. So it's usually something we would design with them, but we're able to make such a dramatic promise because we've been able to prove that over time, yeah. - Wow, that's really interesting. So one way to address this issue is to sell better products, right, to these data centers. Is there any kind of policy that might require them to be more energy efficient? I have an example in Oregon. There's a law here that says that data centers have to keep their energy costs separate from consumers. So that's one way that this issue is being addressed. And I'm wondering if there's other ways via policy that this could be addressed. - I think that's a good question, right? I think when you say that particular policy that you're referring to essentially is saying that data centers consume energy and they could place a tremendous amount of burden on the grid, you don't want the residential consumers in the area to have to bear the risks of maybe rolling blackouts or shutdowns or, you know, peak challenges or even seeing any of the impact on the energy bill, right? So what that means is you're creating a little bit of like an isolated energy environment for the data center and de-risking the local population. Those things can help, right? But what that, the requirement that places on the data center is essentially making themselves, let's just call it self-sustaining, which means the ability to store energy on site, right? You know, things like battery technologies, right? Or the ability to maybe even return power to the grid during low usage or the ability to, for example, even heat that is captured in the data center can be dissipated or disseminated to the local community, which can help, for example. I think you may know the example of in Sweden, Stockholm, right? There is a, in some of the Nordic countries, they actually do this actively. Energy that is captured or that is created in the data center is then sent out into the community to be able to help heat homes, right? And so there's a good, let's just call it synergy, right? Or interflate between now the data centers and the local communities. Back to your question, can policy help? Absolutely it can, yeah? - Have we seen any policies other than the ones like Oregon's, and I know there's others that say consumers can't pay for your energy usage. - I think this is coming up all over the place. You just have to assume right now that because, you know, what's happening in the last year or so is pretty unprecedented in terms of the rate or the scale of growth of these data centers, yeah? Some areas, for example, are completely tapped out in parts of the US itself, right? And now people are looking for locations where they can set up these data centers. It's coming up at the same time that that's coming up and you have backlash against that from certain parts of the community to say, we don't want this around. It is causing governments to kind of rethink what should the permitting process be? How free and how quick so that we can meet the need and we can bring these businesses into the area, but at the same time protect our communities and prevent them from any negative impact. So yeah, I think there's another dimension though. This has less to do with energy, but this is also a dimension on site selection and the setup of data centers, which is the idea of sovereignty. Sovereignty is basically the security of data, right? And I'm just going to broaden the topic a little bit. European countries, many other countries, including Canada, Japan, India, and others are trying to define now sovereignty requirements that say data centers and the data within them are now national assets. They need to be protected within the boundaries of a country. They need to protect citizenry data. They need to be protected in a way that outside influences cannot access your own citizenry data, right? I would say that is not just a data problem and a cybersecurity problem. That is also something that they're thinking now in terms of data centers being critical infrastructure. So think of it from an energy perspective. Think of it from a water and resources perspective. Countries are now beginning to think of data centers as in a very different way than they thought about before, right? So yes, policy will drive how they are set up, how much resources they can use, how companies can operate with them, et cetera. Interesting. I asked you earlier, are you seeing that some of the utility projections about data center energy use may be a little exaggerated? I would say the answer is yes. Technology evolves so fast, right? And so sometimes when we make projections, very quickly, our projections are set off a little bit. So I'll give you an example. Very early on into the Geniai revolution that we are seeing, people measured the energy usage of a chat GPT query versus the average Google search, right? And depending on how it was measured, the nature of the query, this could be anywhere between 10 times to 23 times the amount of energy used or the amount of power consumption for a chat GPT query versus an average Google query. But what has happened since then is that things have gotten a lot more efficient, right? We're able to actually see because model efficiency has grown for all of these Geniai providers. And technologies and algorithms have grown for how information is processed in parallel and so on and in a more distributed way. So even since then, I think we've seen a 33% more reduction in terms of energy consumption. So I would say that if people have been projecting a certain amount of A, footprint expansion, and B, energy consumption, some of those have gone up and some of those have gone down. I think the world is expanding a lot faster than we thought it is, right, in terms of data center footprint, but at the same time, the energy per query and per overall workload that has actually gone down because efficiency has also been gained at the same time. So is there any way to characterize how much more efficient data centers have become as a result of what you just mentioned? This is very hard data to get, unfortunately. It'll depend very much on the model, right? So for example, the chat GPT model would look very different than the anthropic model and so on, right? So that's one. But the second is not a lot of information is actually released regarding the underlying data. I think Google, for example, a couple of months ago released a report that said this is their current energy but the information was so high level that it raised more questions than it answered. So I think we've certainly over the next few months got to do a lot of work, right? Not just enterprises, hyperscalers, collocation providers in terms of being able to collect the information but also be transparent with the regulatory community and with industries to be able to say, here's the true impact of energy use, right? And here's how we can co-plan, right? Work together to design systems in the right way that we're not under-counting or over-counting how we need to cater to these, yeah? Are there any examples around the world where that's happening? I think the deployments across the world vary. I would say that there's certainly a lot more focus in Europe around these kind of deployments and that's because Europe has always been, I guess maybe one, two steps ahead as far as environmental concern and regulation is concerned, right? So their request for data collection are actually pre-intense. Intense to the extent that it may actually slow down a lot of work or cause some deterrent in terms of new data center setup. But that being said, right? I think the answer is somewhere in the middle. Somewhere in the middle between a completely deregulated approach, right? And a highly-regulated approach. We need to be able to actually collect the right amount of information so that we can plan accordingly, yeah? Right, yeah, that makes total sense. Now you mentioned that heat reuse is a good way to reduce or make the best of energy usage. Can you talk about other ways? I think carbon capture, grid modernization. Yeah, these are all good topics actually. Let me rattle through a couple of them very quickly, right? So carbon capture is basically saying, I think people know what the term means, but it helps mitigate the carbon emissions produced through high energy consumption, especially when underlying power is sourced from fossil fuels, right, like natural gas. So what you're doing is you're essentially taking these carbon capture technologies and there's a variety of them, but you're putting them in the data center so that data centers can either like directly remove or capture carbon dioxide whenever, you know, whenever from an on-site power generation scenario. So you talk about CHP systems, which is combined heat and power systems of fuel cells, right? So before this carbon is released into the atmosphere, you can actually capture the carbon and this allows the facility to operate with a much lower carbon footprint. Heat reuse, of course, we talked about, right? Which is basically you're taking the waste heat from cooling servers and you're translating that from waste into a valuable resource that can be used for neighboring buildings, for industries, for communities. So rather than letting it go unused, it's a pretty innovative approach, right? And you're able to use that to provide significant environmental, economic, and even operational benefits. - Are there any examples in the US of heat reuse? You mentioned, I think, Sweden. - Good question. Actually, I'm not aware, I should probably do a little bit more research and I'm sure there are probably proof of concepts being set up, especially in colder areas, but this is definitely something that I would say maybe in the data center areas of the country that are more advanced, you're probably going to see some of these POCs. - And so, other ways to reduce energy? - Other ways, so grid modernization, I think, you mentioned that it's a really interesting idea, right? But it's basically saying when you make your grid smarter, you're actually using data to make, to exchange information between what's happening in the data center and what's happening outside the data center, right? There's a whole bunch of things that can be happening outside, including weather changes, temperature increases, potential storms and other events, but not just that, outside use, depending on the peak time of the day, right? Or what's happening in the local community, especially if there are other industries, loads could go up or down, right? So when you have modernized grids, right? With smart data collection and metering and all of that, you're able to actually not just have the exchange of information, but the ability to actually run with large data sets, collect and project what forecast, right? And run massive simulations in terms of what energy demand is going to be, and in a very real time way respond to it, right? So that allows you to, for example, make sure that the energy used in the data center is as efficient as possible. You're only using as much as is needed and not more, and you're optimizing there. But also, depending on what's happening on the outside, the exchange of that information allows you to kind of like, you know, transfer a shift, you know, the burden of where the electricity is coming from, right? You're able to maximize downtime and so on. So overall, I think this whole dynamic concept, right? And the ability to kind of respond to events and peak periods and demand response, and I think also I mentioned on-site batteries and, you know, storage of energy, all of this helps make data centers a lot more energy efficient. Right. Are we seeing a lot of interplay between utility grids and data centers at this point? 100%. 100%. There are many utility companies that are paying very, very close attention to and actually making investments in the data center space, essentially because of that interplay. They know that this is going to fundamentally affect their business and their ability to provide. It's an opportunity and it's a risk, right? They need to continue to cater to the communities that they serve, but the ability to actually grow their energy distribution and generation footprint, right? In a way that serves the data center and serves the local community. So, yeah, I think this is a very interesting and rapidly evolving space. No energy utility is going to look the same five years from now. They're all going to change in pretty dramatic ways, both in terms of the size of them, as well as the nature of the energy sources that they use, but also most importantly, how smart they are in terms of how much data and AI modeling and automation is used to actually control the grid. Interesting. So, in terms of the interplay between utilities and data centers, so demand response is one option. Yeah, yeah, go ahead, sorry. Are we seeing that? We are, we are. In fact, Hitachi itself is a company that is very actively innovating in this space. To explain what that term means in a second, right? Demand response is basically a grid management strategy where, like I said, data centers can adjust their electricity usage in response to supply conditions, in response to price signals, in response to weather. Whenever the grid is under stress for any reason, and it could be peak demand times, it could be emergency times, data centers can lower, shift, or reschedule their operations, which may be energy intensive. So, what this does is the benefit is it helps stabilize the grid. It also helps reduce their own energy costs. How do you do this, right? You can do something that we would call load curtailment, right, which is reducing any non-critical workloads during that time, during critical periods, right? And draw less power whenever there's a peak pricing or an emergency, or you can load shift, right? Where you can move some of these loads to off-peak hours, or to even data centers in other regions, right? That are less stressed, right? You can also have, I think I mentioned this before, but you can have alternate arrangements to generate energy and store energy onsite, including your backup generators, UPS, your backup battery storage, and how they're used during high demand moments and reducing pressure on the grid while maintaining essential services, right? These are very, very powerful ways to reduce our energy. It's consumed and moved, yeah. - Really interesting. Thanks so much. I found this fascinating. I'm glad you enjoyed the conversation. As you can tell, you know, this is a very actively moving space. Hitachi, I'll throw in another plug over here, but Hitachi is a very interesting company because it is a very, very large, a well-known global industrial conglomerate. Not just do we attack the problem from a data side, right? So how is data created? How is it collected? How is it managed, stored, and used? But we also tackle it from our power generation and consumption side. So we have an energy business. We have a cooling business. We have an operations management business, a managed services business. So we're bringing multiple parts together to be able to solve this problem as a whole. And I think it generally leads to a statement that I'll make that says you have to treat these problems as holistic problems, as integrated problems. Because if you don't, you're solving for points in the process, and not really taking advantage of the fact that you can cross optimize today in a way that has not been possible before. AIX that happened, yeah. - When you talk about the holistic approach, you mean all the things that we discussed today. - Correct, correct. So I'll bear with me for a second because I'm gonna show you an example, right? In the data center, there have been typically different software to manage different pieces. You have an energy management software. You have a building management software. You have a cooling management software. You have an operational management software. And you have software to manage the technology in the rack like storage management. Now, what happens when you actually bring these together? It's not as simple as just saying, hey, one software to rule them all. I'm going to integrate across all of them. Yes, that is absolutely essential. But what we have today with AI and physical AI in particular is the ability to ingest massive amounts of data sets, right? That are collecting all this information, comparing or combining them together. And in automated ways, driving intelligence to say, here's how I can optimize across all of these dimensions at the same time. As an example, if the utilization of the technology is spiking in a way that is generating more heat, you're not only able to connect what's happening in the rack to the energy being, the heat being generated, to the power being consumed and saying, if I tweak this dimension, it can optimize this other dimension. Now, that's really, really powerful. That kind of, I call it, you know, sustainability operations or Sustops, but that ability to use AI to cross-optimize across multiple dimensions of data center management, I think is going to be the next generation of innovation. Really interesting. All right, well, thanks so much for joining me today. No, thank you. It has been a pleasure. Thanks for listening to Clean Energy Today. I'd love to hear from you. Email me at [email protected] with news tips and trends shaping the clean energy transition. I hope you'll subscribe and rate this podcast wherever you get your podcasts.