Does a single AI query use a bottle of water?

Hello, and thanks for downloading the Morales podcast with me Paul Connelly. Each week we take a closer look at the numbers in the news, and in everyday life, not to spoil anyone's fun, you understand, but instead to see if they're real, right, accurate. Now, today we're going to try and break open the robots brain and see what makes artificial intelligence tick along. Specifically, language models like ChatGPT, DeepSeek, Croc, Copylus, and the likes. Lots of you have written in wondering how much water AI systems use when you ask them simple questions. And it's one water claim in particular that has caught your attention. This idea that the equivalent of a small bottle of water is guzzled by computer processors every time you ask an AI that question or get it to write a short straight forward email. So where does the claim come from, and can we tell if it's true? Before I was studying, when people look at the environment and impacts, they primarily look at the carbon emission, which is of course very important, but I think there are other important aspects such as the water consumption. That's Dr. Shaile Ren, an associate professor at the University of California Riverside. It's his research that's the main source of the water claim. In 2023, he co-authored a study titled Making AI Less Thirsty. We want to bring a more complete picture of the environmental impacts of AI to the public. Dr. Ren didn't have full access to the inner workings of OpenAI, the creators of ChatGPT, or to the other big tech firms behind language models. So he did what academics so often do, when faced with a data door that's bolted shut. He annotates it, he calculates it, and he extrapolates it. The energy was cited from OpenAI's own paper about GPT-3, and also we cross-validated the number with a few other resources, including the paper recently published by Microsoft and in their production system. And when we look at the water efficiency, we refer to Microsoft's own disclosure. Microsoft runs and maintains open AI's data centers. So what is AI using water for? Well, when you fire a query ChatGPT's way, it's processed by a server, a computer basically, in a massive data center. All that digital processing generates heat and to stop the servers overheating, these data centers run some heavy, juicy cooling systems. They get rid of that heat by evaporating water into vapor and cooling terrors, which then carries the heat away. Some people have assumed that Shaolais figures refer to just this kind of water use, the water used by data centers that are running AI systems, but it's not just that. We include both direct water consumption for cooling down the data center facility and also for indirect water consumption that is consumed for generating electricity. The calculation also includes water that ends up being evaporated off-site, in the generation of the electricity that powers the servers, water that evaporates, for example, in the cooling towers of coal power stations. And that water is most of the water we're talking about here. 87% of us for an AI query go into an average US data center. So important context uploaded, let's get to the calculation. Using the only relevant data Dr. Ren and his colleagues could get their hands on, plus some fancy analysis, they came up with some numbers. If you have about 10 to 50 queries with a medium-sized larger language model, there will be 500 million liters of water consumption, or the water evaporated into the atmosphere. And that is for just answering the question without using reasoning or very sophisticated models. So charlotte's paper gave an estimate for how many medium-sized AI questions think maybe of an AI generated email, but 100 words long or so, it would take to evaporate 500 milliliters of water. On average, according to charlotte's calculations, if the data centers are in the US, then you can ask around 30 questions of an AI chat, but like chat GPT before using that much water. That jar, doesn't it, with a widely circulated claim that a solitary query prompt door question uses the same amount, but here is where there's a twist in this tale. A while later, journalists from the Washington Post came knocking. Charlotte's calculations were all for the slightly older version of the AI system, that's chat GPT3, but the post wanted to pin down how much water the newer version, that's chat GPT4, thanks when writing the same 100 word email. First, I provided some law data, but they did some other follow-up analysis. Chat GPT4 is a lot more powerful than the earlier version GPT3, meaning it uses a lot more processing power. The Washington Post took charlotte's calculations, applied them to the new system and weirdly the resulting number was almost exactly the same. Except this time it was for just one query, not 30. Now remember, the vast majority of that water use will be in power stations, not in data centers. And in terms of limitations, obstacles, this data deep dive was even harder than before. Because GPT4 is a completely closed source, it's not like GPT3, which we have some concrete information about, especially the model size, but for GPT4 it's totally closed. So there's naturally more uncertainties associated with our study in GPT4. So the accuracy of this extrapolation is not something we can check. Open AI responded to the Washington Post's findings, saying they are constantly working to improve efficiency. But then in June of this year, CEO of the company Sam Ultman broke his silence on the issue in a blog post, saying that the average query uses about 0.0, 0.0, 0.0, 8.5 gallons of water, roughly 1/15 of a teaspoon. With the ultimate number is the devil lies you see in the lack of detail. What for example does he mean by the average query? Well essentially my experience average doesn't mean much. That is Sasha Luciani, a computer scientist and AI energy specialist at the curiously named Hugging Face, a machine learning and data science platform. Because especially for a quote unquote general purpose tool like GPT, people are doing all sorts of things with it. I think it could be reliable if you define the boundaries better. If you really say for a text only query that requires maximum, I don't know whatever, 20 words generated, 20 tokens we say generated, then it could be that number. But it doesn't make sense to give a single number for all queries. It would have to be a range or much more granular than that. A further caveat here, experts like Sasha thinks he's probably only talking about that unsight water used to cool the processors, not the off-site of operation in power stations. So then back to our original question, does every AI query use a small bottle of water? I think that the true number is somewhere between Altman's number and the thirsty AI number. I think 500 milliliters. It's a lot to me. I think they try to make an estimate of something that is inherently unquantifiable given the amount of information that's available to us. So all told, when it comes to the tech alliance and their data were mostly in the dark, those smaller players in the AI game are starting to break ranks and to fess up. In late July, Nistral, a French company went public, saying each typical response from Le Chat, that's their version of chat GPT, uses around 45 milliliters of water per 200 to 300 word response. So quick and elementary maths then tells us that 10 of those responses is just shy of a small bottle's worth of water. Now this data is so sizzlingly hot off the press that key voices in the space like Shalai and Sasha haven't had a chance to fully fact check the findings, but early analysis suggests that between 80 to 90% of the water uses their reporting happens off sites. Remember, that's in power stations, not data centers. Watch this space. Thanks to Dr. Shelley Ren and Dr. Sasha Luciani. If you've seen a number you want us to try and make some kind of sense of an email us and more or less at BBC.co.uk. Until next time, take it handy.