The threat that AI poses to human life — with Karen Hao

[MUSIC PLAYING] This is an ABC podcast. Hey, it's Anthony Burke here. Radio National presents the home front. Housing in Australia is under pressure. The crisis is real. But could better design be part of the fix? I'm Anthony Burke, and in the home front, we explore how the great Australian dream is shifting, and what we could do to build a better future. To hear the home front, search Radio National presents on the ABC Listen app. [MUSIC PLAYING] Every time we do a topic like this, I forget. I forget to do it. I should have gotten, I don't know, a chat GPT or some other AI machine to come up with a script to begin the show. Unless I didn't. And so you get this mess. Welcome to the Mindfield. Well, that alley is my name. Scott Stevens is my co-host. Today, Scott, we finally deliver on a promise, which is something we try to do. We perhaps do it a little later than we anticipated, but there were contingencies last week. But also, because of a guest that we have for today's show, who we are very excited about, and was only available to speak to us this week. And I'm really looking forward to it partly because we-- can I confess something to you, Scott? Hello, by the way. Hey. It is amazing. The percentage of conversations are just socially and I find myself having now that end up veering off into a discussion of AI. It just keeps happening. We could be talking about something totally different. Someone will mention AI imparcing often as a punchline, and then we're in. And an hour later, we come up for air, and we don't know where we are. And I think, actually, we don't know where we are, is a more appropriate phrase than I have been anticipated, as I said it, because I think it's one of these issues that is dominating the consciousness to such an extent. I think because it might be existential, it might go to the heart of our living standards, or our careers, or our sense of self-worth even, but we don't know where we are. We cannot seem to find our feet and make sense of exactly where we're going and what the consequences of all this might be, or we know at this point, I think, is our anxiety. Yes, that's right. The other possibility, of course, is that the whole thing about existential risk is something like a smokescreen to justify certain forms of unprincipled behavior now, and to distract attention from the very real consequences that are playing out all around us as we speak, rather than further down the right path. OK, all right, I can say where this is going to be. This is something we're going to have to talk about soon. We've just got a couple of things to clear-- Oh, OK. --for our plate. Yes, we do. I know, I know. It's a deferred conversation, but it's going to be worth it. Just a couple of quick things to mention. The first is that a really important vote is going on. ABC Radio Nationals, top 100 books vote. We're in the middle of the countdown. So if you love this show, and because you're listening to it, we assume you do, then you know what leads in my thing about books. We love them. We really try to encourage reading as much as we can. And if you care about books, this is a really fun, meaningful opportunity, I think, to connect with some other people and to put in your vote for your favorite books of the 21st century, and leave the last 25 years. You can just head over to the Radio National Home page. And vote now. I'm going to go ahead and spoil what I nominated. My favorite book of the last 25 years, "Without Question." Willie, that's the one that's stuck with me more than any other. Is Jam could see as wonderful 2003 novel Elizabeth Castello. I can plot my life in terms of a before and after having read that. Wow, you've got a Scott calendar. Oh, yeah. So more than 30,000 votes have already been received since the votes opened. We really hope that you will join your name to that list. Can I say, I'd love the way that whenever you reference a film or a book, you have to give the year. You can't just say the name of the book. Have you noticed that? It's called Streetcred. Well, is it? Oh, yeah. Not sure which street you're on. Anyway, fair enough. Thank you for telling us. The other bit of housekeeping we need to do is we did mention that very soon we are going to embark upon perhaps the full-hardy expedition of a mailbag episode. So this is where we've never done this before. We will just take a bunch of questions and we will do our best to answer them throughout the course of the show. We haven't figured out what the breadth of it will be. We haven't figured out what the, I don't know, the brevity of it will be the speed of it. We don't know if we're going to do three questions or 10 or hopefully not one, because that would be a normal episode. But you get the sense of where we're going. A lot of those questions actually will be determined by you. So if you have questions that you would like us to address, they can be about all sorts of things. We did have had a bunch come through and they're spanning the widest range of things from horrific events happening overseas to, I don't know, sport. There are all sorts of things that even films. So all of that's coming in. If you want to send your question through for consideration, obviously we can't do all of them. But if you would like to, the email address is the [email protected]. I'll say that again, the [email protected]. You have until what date was it, Scott, the 12th? Close the business on the 12th year. To get your questions in, thank you to all of those who've already met us. That's great to get these coming through and see what's on your mind. And would like to see what else is on more minds as we wind our way to the 12th and look forward to that episode. All right, Scott, speaking of minds, indeed. What is it exactly? And does AI replicate one? So the context for this particular show is that a couple of weeks ago, we did a show looking specifically at AI in the context of the productivity roundtable that the government was holding and questions to do with what it might mean for employment. And then the flow on question of whether or not that necessitates or would usher in a universal basic income. And we had a whole conversation about that. As I recall, and my recollection is not always good on these things, but as I recall, we also said, this is looking at AI in a fairly functional economic way, which tends to be the way that public servants and politicians look at it. It's an economic question. But it leaves open, far grander questions about the meaning of the adoption of AI. So the question I want to raise, which I think Scott doesn't want to raise, is whether or not we are overlooking the question of AI's fundamental nature. That is, is it can we view it as and should we view it as merely an extension of technological advancement as we've always known it through human history? It's obviously a newer version. It's potentially a more powerful version than previous technological advancements, but it is nonetheless the same thing. Pretty much a tool that we will then use for our own betterment with some consequences we don't like, but on balance benefit. Or are we looking at something that, in its essence, in its underlying grammar, is so fundamentally different that it requires us to think about things in a totally radically different way? I'm happy to flesh that concern out a little more, Scott, but I understand that you either want to take issue with that or you want to flag a different sort of concern, market a different level of terrain. So maybe you do that first, and then we'll say what we end up. Sure, look, I'll just be really quick. I think there are two teleological ways of thinking about AI. One is that AI represents the telos, the ultimate goal, the endpoint of humanity's entire relationship with technology and tools. These are things that we use, these are things that we use in order to make our lives better. These are things that we use to shape the world that we want to inhabit. These are things that we've used as an extension, if you like, of our very physical and social existences within the world. But all of these tools ultimately serve human ends, human goals. They don't turn around and rule us. In that sense, they're not sort of idols that turn around exercising God-like capabilities over the point. For the point is one of mastery. One of mastery, that we remain the master of the technology. It remains us. Now, I mean, one of the things, though, that always overlooks, it seems to me, is the longstanding philosophical concern that the tools that we use almost never remain strictly external to us. Tools do something to human existence within the world. Tools we become reliant upon them, they end up affecting or infecting or cultivating or even limiting our own sense of interiority. Just because we become capable of something, it can then have the effect of limiting our ability to do other things. So it would be a very strange philosopher that would think that tools always only ever remain external to us, that they remain entirely within our power and strictly the extension of our own wills. But I think there's another teleological way of thinking about AI, which is that not so much as this the goal or the end of human technology, but this is something that we are bringing into the world. Call it an entity, something we are bringing into the world that we are, to some extent, shaping after our own image, after all, we're referring to this as a form of intelligence, a form of intelligence that's meant to some extent map onto, if not exceed, certain human functioning, calculating, planning, pursuing capabilities. But that such is the nature of the superiority of this entity, of this intelligence. Let's call it a superintelligence, that there will come a point inevitably, invariably, where that intelligence will simply shrug human constraints or say the constraints of humans' goals within the world. It will shrug those off, and it could well pursue other goals that emerge unpredictably from within the discrete confines of its own programming and training, and will decide fundamentally that it no longer needs humans. Or the human beings will end up serving goals that that entity will, in fact, have established for itself. Yes, and the threshold question there, which, to some extent, is a factual question, more than a philosophical one, is, does AI have that capacity within it? Is that something that it could evolve to doing, that is effectively becomes self-starting, some kind of independent thought machine, which can then, in the same way as a human brain, might develop its own goals and aspirations and imperatives, even ones that it doesn't fully understand itself? We have this experience as humans all the time. Why did you do that? I don't actually know. I don't know. Let me go ask my therapist. We understand that sort of opacity sometimes within human beings. It might be like that with AI, but does it have the ability to do that anyway? Or is it merely a probability machine that requires external output in order to do anything of its own motion, and thus remains entirely controllable, really, because of some conceptual difference between it and the human brain? And that's a question, really, though, I think. I don't know the answer to that. And I think it can only be left to people who know a lot more about AI, and probably people who worked on it, and people such as, I guess, who we may get to address it. Yes, can I just pick up one important point, though? There is an underlying irony here that I find really, really uncomfortable, and I feel like I just need to kind of name it up front. As soon as some of the pioneers of AI over the last decade began coming out in favor of universal basic income, as a solution to the unemployment apocalypse that artificial intelligence could well bring about, I began getting really skeptical at universal basic income itself. As soon as there are these kinds of warnings and these kinds of solutions, I keep looking over my shoulder and thinking, OK, what's the self interest that's being served here? There is something that I find really discomforting that the profits of AI are also the doomsayers of AI, that the very people who are heralding the imminent advent of something like artificial general intelligence are the very people that are saying that their risks involved and we need to treat those risks the same way that we would treat, say, nuclear apocalypse, which leads me to think well-eared. So let me just plant my flag on the ground. I think the whole superintelligence stick is BS. I think it's a marketing ploy. I think it is a form of retroactive legitimation for all manner of otherwise unprincipled or unregulated non-transparent forms of extraction, of exploitation, of colossal amounts of capital investment, of almost unregulated relationships with government, and ultimately a forms of profound injustice and exploitation in third world countries or countries within the global south. Because there could be something like a superintelligence and because that superintelligence will come about, we need to be the ones we quote unquote. We need to be the ones who get there first. Therefore, anything that is necessary to be done must be done otherwise. The likelihood of an existential threat will become a certainty of an existential threat, except that threat will not be in quote unquote our hands. My underlying concern here will lead is that this just strikes me as a version of what Shoshana Zuboff called the cult of inevitableism. This thing is going to happen. We just need to be sure that we're there first. Whereas what has, in fact, been produced so far through language learning models, through this obscene display of scaling, of simply more and more and more processing power and more and more resources, is that things that are necessary for human life and things that are fundamental to senses of human dignity and emotional well-being and other things are being sacrificed on the altar of a superintelligence that is being held out-- I mean, it's being held out as an actual existential threat. But it's being used essentially as a smoke screen to cover over the actual forms of threat and exploitation and extraction and environmental degradation that are taking place all around us. So from my point of view, I'm calling BS on the whole superintelligence deck. How do you account for the fact that some of the people who are talking about superintelligences are doing so precisely because they have a problem with it? Some. I mean, the classic example of someone like Daniel Cockatalla, who wrote quite terrifying forecast of how AI would transform the world through superintelligences. And he even said, it would be 2027 when it happened. I think he's revised that since about 2028. But he's not doing this because he wants to subscribe to a cult of inevitableism. No. He's doing this because he's saying, we need to regulate AI very seriously now. This is under the rubric of the precautionary principle. That's right. Right. I mean, this would be impossible, wouldn't it, to construe what he's saying as a kind of BS smoke screen? No, that's right. But when you put that scene, he's doing so with technological knowledge, right? He worked there. He knows how the thing develops. He sees this in its future because he has an intimate knowledge of the technology. And he says, this is what's coming. I said, no, it seems very brave declaration from someone in a position like one of us to just say that that's some kind of BS smoke screen. You could say that there are current urgent things that we should be worried about and that we're not addressing. I would even be with you on a lot of that. But to say that the super intelligence warning is just rubbish. I don't know. That presumes a level of technological knowledge that let's just say I'm skeptical you have. Sure. I'm just saying that you put those warnings in the mouths of the very people who are developing it and who are developing it using a single form, a single tool, a single mechanism, a single approach or model called scaling, which is simply more, more, whatever it takes, more of it. That's where I think we're getting into technology. On the basis-- --democratically, politically, and ethically very but on the basis that it's controllable. And what the critics are saying is, no, these super intelligencees are not controllable. Well, given the fact that they've behaved unpredictably to date-- Already, yes. Yes, that's right. OK. All right, you are listening to the mind for, by the way, on ABC Radio National. Well, they are these miners. Clearly, we're floundering, Scott, by drowning. Should we get a lifeguard to come and fit us out of this water? Yes, please. All right, go for it. Our guest is Karen Howe. She's an award-winning journalist who covers the consequences of artificial intelligence on politics and society. She was formerly contributing writer at the Atlantic where I first came across her work. She was a foreign correspondent, covering China's technology industry for the Wall Street Journal, and she was a senior editor for AI at the MIT Technology Review. But most important of all, she's the author of, what is to my mind, a masterpiece, not just of technological reporting, but also of moral reflection. It's called "Empire of AI" inside the reckless race for total domination. Karen, thank you so much for joining us on the mind field. Thank you so much for having me. Well, ladies, just said, we've worked ourselves into a bit of a-- OK, take us away. Your questions, both of you, are so interesting and so relevant. And maybe I'll just pick off where you left off, which is this genuine question of people like Daniel Cocotello and other people in this camp that really are sounding the alarms around existential risk of AI. Are they sincere or is there some kind of smokescreen that they're putting up to perpetuate actually, in fact, the very industry that they're critiquing? And it's super interesting because this is one of the most surprising things that I found when I was reporting my book is, I also went in with Scott's perspective of these people are just-- it's all just rhetorical tricks. They are just saying, look at this, pine the sky, potential existential threat, and therefore don't pay attention to all of the other things that are happening right now. And what I discovered is it's much more complex than that. There are contingents of people like Daniel Cocotello who genuinely believe that there is a real possibility that the technology is going to achieve some form of superintelligence. And so there's contingents of these people that are sounding the alarm out of what I describe as a quasi-religious belief in this happening, this potential devastation of the human race, potential devastation of the earth. And then there is another layer of political actors, people who are savvy about navigating public discourse, navigating government understanding of the technology, that leverage this genuine belief to then do what Scott is saying, which is have put up this smokescreen and try to shift people's concerns away from other things that are happening right now, like the environmental destruction, the labor impacts, and other types of impacts. So it's actually like both of you are right, that there are people that are sincerely working towards what they believe is a possible future. And there are also people that are then using that in ways that is ultimately just perpetuating the AI industry. And so what I conclude in my book is essentially that the superintelligence doomsayers actually work hand in hand inadvertently with the AI industry accelerationists. And in fact, Daniel Coketela would tell you the same thing that he would reflect that in hindsight, some of the work that him and his ideological community have really put forth and the messaging they have put forth has in fact inadvertently done exactly the opposite of what they intended. I'm just not sure that that matters all that much. Now I understand that sounds like a scandalous thing to say, but let me flesh it out. Let's say that instead of the focus being on the superintelligence and the doomsday scenario that gets painted there of the machines taking over and slaughtering us all, effectively treating us the way that we would treat any other species. Let's say instead of that, the focus was on the kinds of things Scott was talking about. It seems to me that the culture of inevitableism and the politics of it would nonetheless prevail anyway. Why do I say that? Well, let's take an example like climate change. How does the climate change debate proceed? Will it proceed certainly in this country on the basis of, well, yes, we could cut emissions and harm our economy and do all that. But in the meantime, what's China doing? In the meantime, what's India doing? In the meantime, what difference is it going to make? Now, let's leave aside the factual accuracy of any of those sort of arguments. But I'm talking more about the structure of the argument. In other words, it immediately collapses into an inevitableism anyway. All you would need to say is, look at the damage this is doing to human beings. Look at the damage this is doing to the environment, whatever, this being AI. And the answer would be, yeah, do you think China's going to stop? And if that's the case, there's an arms race where pretty much they get all the benefit of this and we get all the cost. So I guess we have no real option. The less a rival is that we must pursue it anyway. I suppose what I'm saying is, I have an inevitableism about the inevitableism that it may not. So the question then to me, the question that becomes interesting is, sure, maybe that's fatalistic, whatever. Leave that to one side. What's correct? Is the argument about superintelligence correct? Is it cogent? Is it likely to transpire or not? And then we proceed. I don't want to hold someone like Daniel Cockatillo accountable for the fact that political and commercial actors are going to spin his argument, because that's what they will do. So first of all, I do think that nothing is inevitable. But it only becomes inevitable if you believe that it's inevitable. But there are many things throughout history that people portrayed as inevitable that ultimately saw as demise, like, empires were part of the reason why I call my book, Empire of AI, because I make this argument that these companies need to be thought of as empires. And it's a very new analogy to the fact that empires portrayed themselves and their conquests as also inevitable. But then every empire in history is fallen, because at some point, people realize, wait a minute, we have agency to fight back against the empire. And so I would say, in general, philosophically, for me, there is no such thing as inevitability. The other part of your question is this idea of, well, does it matter what kind of messaging people put forward? Do the political consequences of their messaging matter if what they're saying is just true? And so then the question is, is what they're saying true? And that's where it gets quite dicey, because there's no scientific research to back anything that they're saying is true. And in these communities, like Daniel has said before that his projections are actually based on several theoretical assumptions that need to become true in order for all of his projections to unlock. And those theoretical foundations have not come to pass, which is why he's shifting from 2027 to 2028. And then he will shift to 2029 and then 2030. But yeah, there is no actual scientific consensus in the literature around the fact that we are even remotely close to developing some kind of intelligence within these systems. And in fact, there was a survey of long-standing AI researchers in the field just recently where 75% of them still think that the current available techniques will not get us to intelligence. And we might not actually ever get there at all, which is something that you rarely hear because the industry, Silicon Valley, has become so successful at dominating the narrative around what they're saying, which is right around the corner from recreating intelligence, versus what the actual science says, where that is a still a minority opinion. All right, Karen, you've actually just landed on what I think is probably my single greatest concern here. So the way that we tried to frame things at the beginning is we're talking about two different conceptions of ends or of goals, are these tools serving humanities ends? Or will humanity end up having to turn around and serve the end of something that we've created? There is a question that's right at the heart of this, though, which is, is the current model, the current mode, whereby artificial general intelligence is being pursued and is being promised? Is it capable of getting us there? So in a second, I'd really love for you to help us explain just what it is about scaling that is, to use the term we were just using before, not inevitable in the sense that this isn't the only game in town. This isn't the only possible way of framing things. But the reason that I'm so concerned, I've got to say, about scaling, is that what it involves is such a massive redistribution and redirection of certain what I would regard as fundamental goods. So I think our concerns about copyright, for instance, about the protection of human creation, of human sort of literary artistic creation. I think that is unbelievably important. But I think that because scaling requires more and more and more data-- this is the so-called data imperative-- that you find all of these things that should be valued in their own right, more or less being turned into fodder for the ends of something else. The same thing is happening with copper and lithium. The same thing is happening with land, with water, with human labor. Can the current scaling model get us there, given we haven't seen anything like the leap that we saw from, say, GPT2 to GPT3? We haven't seen anything commensurate with that leap. And yet the problem is we just need more chips, more data centers, more resources, more data sets. And eventually, the thread will break, and we'll be able to cross over to the other side. Yeah, I think maybe the best way to answer this question is actually to do a little bit of history. So the term artificial intelligence, which I think is the root of a lot of these speculations about-- I have we-reach intelligence, have we reached superintelligence-- was originally coined as a marketing term in 1956. So there were a group of scientists that came together at Dartmouth University, John McCarthy, and assistant professor at Dartmouth. He was the one that decided to name this new discipline that these scientists were embarking on artificial intelligence. And they had all these ideas of essentially trying to create a discipline based on Alan Turing's very provocative question that he asked several years earlier, Ken Machines think. But John McCarthy received actually a lot of pushback from some of his colleagues and his own mentor, saying the term artificial intelligence is very fraught with a lot of problems. It's going to confuse the public. And the central problem is we don't actually know what human intelligence is. And that was true then. That is true now. All the way until present day, there is no scientific consensus around what constitutes human intelligence. And in fact, the long history of pursuing different types of quantifications or rankings of human intelligence is very dark. Like there is very, very dark motives throughout history for people trying to do this thing. So then the question is, if you're going to call artificial general intelligence, the recreation of human intelligence in machines and yet we don't know what human intelligence is, then when do we, how do we measure when we get to artificial general intelligence? And this is part of the reason why there has been just a decades long debate about whether or not artificial general intelligence has actually been already achieved, or is it going to ever be achieved, or how do we actually achieve it? What are the signs for when we will achieve it? And essentially, there have been different two dominant camps in the discipline of AI that have emerged that have the most staying power in terms of different approaches to this so-called artificial general intelligence. One, that is rooted in the idea that human intelligence, humans are smart because we have knowledge. So we should be encoding machines with databases of knowledge and the other, which is humans are smart because we can learn. So we should be developing software that can learn from data. And what we're seeing today is essentially the learning branch is overpowering our current technical implementations of artificial intelligence. And yet in the scientific world, there's still raging debates about whether or not allowing that branch to dominate has in fact been the correct decision. And really, what some scientists would say is the reason why it dominates now isn't necessarily because it's scientifically more correct or has demonstrated more progress than the database driven or knowledge-driven branch, but primarily because it is just more suitable to corporate interests, like corporations have an extraordinary amount of data. So if they play a game whereby the rules are one based on how much data you have, then you would want to play that game. So yeah, so there's a lot of questions around is scaling actually going to lead us there, but also more fundamentally is this data-driven approach even going to lead us there? And what is there, like where are we going? No one has a way of really defining it. And that's part of the reason why it's so complicated these questions of, yeah, is intelligence even the right word to describe this entire enterprise? And are we assigning too much agency to these systems by using that kind of language? So even in the framing of the question of like, is AI going to help continue helping to achieve human goals or are humans going to start serving AI goals? Is assigning a lot of agency to this technology when I think the way that I would put it is like, the better question maybe is, are humans going to end up undermining our own goals because we are generally vulnerable to seeding agency to inanimate tools? - Well, this is the Mindfield on ABC Radio National. Well, that's Ali's, my name's Scott Stevens, my co-host. The voice you just heard belongs to a very special guest, Karen Howe, who's a award-winning technology journalist, author of Empire of AI inside the reckless race for total domination. Just as I read that out then, Karen, I was immediately arrested by this reckless race. - What do you regard as reckless about it? - I think a lot of the things that Scott has mentioned, the sheer amount of resource consumption that Silicon Valley is telling people is absolutely necessary to pour into the creation of these large-scale so-called general-purpose models. And the reason why I say so-called is because there's plenty of evidence to show that they're not actually general-purpose, like the moment you start talking to these tools in a language other than English it already starts to break down. So how general-purpose are they really? But there is just an extraordinary amount of data being put in these systems. There's an extraordinary amount of computational infrastructure being built to train these systems. And whereas in the social media era and the internet age, there was also plenty of data and plenty of computational resources being funneled into the production of that internet digital technology. We are at a completely different scale in terms of the resources required now. So meta is a really great example because they were both a social media era company and an AI era company. And in the social media era, they accumulated around four billion user accounts worth of data. And as they started to enter the generative AI race, they started having internal conversations where there were that four billion user accounts worth of data is actually puny compared to what we need to actually make a dent and become a leading player within this race. And so they were talking, and this was reported in the New York Times at the time, about potentially acquiring Simon and Schuster. They were talking about continuing to scrape every possible piece of data off of the internet. Eventually, they didn't acquire Simon and Schuster, but just downloaded all of the books from the dark web. And so that is just like one data point for the sheer order of magnitude increase in resources that we're talking about. And whereas before they had plenty of data centers that they were putting around the world, the pace at which data centers was expanding for the modern internet pre-AI roughly equated the pace at which energy efficiencies were also in increasing for data center buildouts. So most developed countries, we've seen a flat lining of energy over the last decade or even a decline in energy consumption, even as more data centers have been laid. Whereas now in the AI era, Meta is talking about building supercomputers the size of Manhattan that would have power draws the size of Manhattan. And this has single-handedly created a massive surge in energy demand in developed countries all around the world. And it is also single-handedly now reversing the climate progress that we've seen over the last decade. So all of this, I think though, there's kind of the point I was trying to make at the start. That is not going to be an arresting concern to governments around the world while the promise of hugely increased productivity is on the table. I don't think it will be any way. I mean, feel free to argue against that and point out an example where a government has walked past the promised productivity gains. Yeah, well, I mean, promised is the key word, right? Like-- Sure, sure. But governments are in that business. They have to make that assessment. And what we saw in Australia-- I don't expect you to know this, Karamot-- what we saw in Australia was we had a productivity commission that came out and put a figure on the productivity benefit. And it put it a huge number, a number that we would not-- the only way you see is with some great technological advancement. And so in that context, once you have an argument about, well, other nations are going to be using this. I just don't know how far you're going to get with an argument. As much as I'm sympathetic to it, I don't know how far you're going to get with an argument over resource consumption. Well, one of the challenges that I think governments should start thinking about is the fact that these types of data centers are starting to undermine communities, their economic opportunity, their fresh water resources, and their affordable housing. So in the UK, the development of data centers has actually caused certain jurisdictions to suspend their ability to put a ban on a more housing development, because not only do these data centers use an extraordinary amount of energy, they also end up using fresh water to cool the data centers. And it has to be fresh water because any other type of water is corrosive to the equipment and can lead to bacterial growth. And so there's not enough fresh water or electricity now to go around for housing. So these are actually present day pressing issues that are affecting people's quality of life. And in the US, we are already seeing protests breaking out all across the country, as well as in Spain, in certain parts of Latin America, in other parts of Europe as well, protests where people are incredibly angry about the idea that this technology requires them to diminish their ability to live a dignified life when they're also not actually the ones that are receiving those productivity benefits. So I do think that it will become, if not a compelling argument now, one very, very soon, because if governments do not think about a way to ensure that this technology is developed in an equitable way, then they will have protests on the streets, as we've already seen around the world. This actually gets to, again, an irony that just strikes me every time I think about it. You hold out the prospect of existential risk. We may well be bringing into being a super intelligent entity that will end up rendering the human race subservient and suck the life out of the planet. OK, I'm sure that someone's going to come along and put the necessary safeguards against it. But we are bringing something into being and the process of our pursuit of it is having this degrading effect on quality of living, well-being, mental health, resource users. I mean, one of the figures I'll confess, Karen, it stopped me in my tracks and it left me kind of dumbstruck for the better part of an hour, that on current projections by 2027, something like 6.4 trillion leaders are going to be required to cool the data centers that are projected to be built between now-- which is half the consumption of the United Kingdom, by the way. It's unbelievable to me. So there is something about holding out-- let's call them the proximate risks. The proximate implications for the way that we live in our ability to live well together. There was something about pointing out that danger that seems to have more democratic purchase than the holding up of these great existential risks that a superintelligence might represent. There's something about that I'll confess that is hope-inducing for me, rather than thinking that it suggests a kind of general state of human complacency. But I think it has to be concrete, right? I think what was so powerful about the example that Karen just gave is that it has a direct effect on people's quality of life and access to fundamental essential resources in the moment. And that's when you will get the protests. What you won't get is any kind of resistance on the basis of an abstract idea that wow, it's using a lot of energy. It's when the promised benefit becomes more abstract than the experienced home, if I can perhaps appropriately reduce it to an equation. That's the moment. That's the hinge point, right? But apart from that, I think-- because otherwise, what is being promised is a radical expansion of our quality of life. And that's where I think the superintelligence is argument. That's the work that it's doing is saying, actually, what you might be doing is ushering in a total destruction of your quality of life in ways that are just so fundamental and essential that you really need to think about this. Now, maybe people will be moved by that, maybe they won't. Maybe that's too abstract as well. But I think either way, if you're going to point to deleterious consequences in the interim, they will need to be concrete and experienced as such before they become a mobilizing force. I also want to add one other dimension to this conversation, because I do find that there is often this idea of AI could give us these extraordinary benefits. It's also giving us these extraordinary costs. And it's just a tough problem. And how do we engineer ourselves out of this fundamental trade-off? But actually, it's a false trade-off. Because there are many different types of AI technologies. And the thing about generative AI, or general purpose, quote unquote, general purpose AI systems, is that there's an abundance of evidence now that the productivity gains that these Silicon Valley companies promised are not actually panning out while the costs certainly are. So there was just a study from MIT that came out just, I think just two weeks ago, that showed that 95% of generative pilots in businesses in the US right now have been complete failures. Only 5% have succeeded in terms of any type of productivity gain. And so there's plenty of evidence now that's showing that maybe this version of AI wasn't exactly all it was tracked out to be. Whereas there's all of these other types of AI technologies, small task-specific AI models that target certain types of problems like detecting cancer earlier in an MRI scan. We've had that type of AI for a long, long time now. It does not need vast supercomputers the size of Manhattan. It actually just needs a powerful laptop to train on. We know, therefore, that the costs are very low for developing this type of AI. And the benefits are huge. We want to be able to detect cancer earlier in MRI scans. And there's also plenty of scientific studies that have shown that when you put those types of tools in the hands of well-trained radiologists, that the ability to detect the cancer earlier and more accurately increases the combination of both Dr. NAI tool outperforms Dr. or AI tool alone. So we have essentially plenty of data now to show the true cost-benefit analysis of one AI tool versus many other types of AI tools and what I would urge governments to do is this isn't the false try-dough. It's not like you use AI or you don't. It's about shifting your resources towards investing in the types of AI technologies that already are substantiated by evidence to being beneficial and not that costly and shifting away from the type of AI that is already substantiated as being hugely costly and not that beneficial. - That presumes though that the equation will remain the same, right? I mean that you won't get sudden exponential improvements once you crack the tough and not if you're not in it. - Totally, and then I would ask, I mean like to what extent do we keep pushing because at the rate at which we're going to get to those improvements, like the planet won't be around anymore. So we need a cutoff point at some point and people won't have housing and water. So yeah, this is what I find so fascinating about generative AI particularly is like under the generative AI paradigm, we have somehow suspended our reality to crave a future that may or may not ever appear. Like it might actually just be a mirage in the desert and like we should just actually move gravitate towards the AI technologies that literally already show us they have already illustrated their promise. They're it's not promise anymore. It is existing real benefits that have very little cost. This is the Mindfield on ABC Radio National, well later all these minds got stave as my co-host I guess is Karen Howe author of Empire of AI inside the reckless rights for total domination. - Karen can I ask you a question? It's going to sound a little bit crass in the way that I put it. I'm trying to simplify and make it memorable. So Christopher Hitchens' brother, Peter, said something about a decade and a half ago that I just found unbelievably funny at the time and I find it unbelievably funny ever since. He said that Stephen Frye is every stupid person's idea of what an educated person must be like. What's funny about it is there is a kind of projection of intelligence, a face of it, a sense of it and we then have a clear sense of what must be going on underneath. I've often thought that deep learning is an engineer's idea of what human intelligence must be like. The human brain must be pretty much a machine and it must learn the same way that machines learn or go machines must learn the same way that human brains do. Is it possible that what we've seen released to the public with extraordinary effect over the last three years, GPT-3, chat GPT, the whole range of suites of generative AI products that enable human interface, but that really are little more than statistical regurgitation machines. They've given a sense of virtuosity and prodigious creativity. They've given a sense of let's call it human-like aesthetic and generative capabilities such that we were kind of sold on the idea, my god, intelligence, quote unquote, is just around the corner. Could it be the case that this is pretty much as good as deep learning can give us, that we've reached something that is in many respects impressive, but that falls so fundamentally short of the promise that you were just describing before? Is there a chance that this is simply the ceiling? This is as far as this particular model of intelligence can go. - Yeah, I mean, that is definitely something that a lot of scientists wonder. I would say that what we're seeing right now isn't just deep learning at work. It's a specific neural network architecture called a transformer that is being scaled up. And even within the realm of deep learning, there's plenty of other types of techniques that have existed. And so one question is, is this just the topping out of transformers or is it the topping out of deep learning or is this the topping out of the entire AI discipline? That's a huge debate that scientists are waging right now. But so to your point of like, is it just that engineers, because they have their well-resourced and they're politically and economically powerful right now, they have just projected their own ideas of what human intelligence are into the world. And now that's kind of what we're seeing. Yeah, I think that's absolutely what's happening. And I think there's a more fundamental question here of also like, why are we pursuing AI in the first place? Like one of the things that I have started to become uncomfortable about with the general premise of AI as conceived by the original scientists as well as people that are still pushing for the School of Artificial General Intelligence is it's advancing technology purely for technology's sake. Whereas traditionally speaking, like I was trained as an engineer, traditionally speaking in engineering disciplines, you find a problem that actually needs to be solved and then you come up with a solution. Whereas Silicon Valley's model of innovation has increasingly been flipped where it's like, you just advance a technology because it's cool, it makes you rich, it makes you look like, yeah like Mark Zuckerberg and his Zuck Renaissance, his Zucka Sons. But like, then you have an issue of like product market fit where these companies are like running around like headless chickens right now trying to actually get people to use their technology at the rate that would turn it into a profitable business. And the problem is like AI as a discipline has always been conceived a little bit as advancing technology for technology's sake. Like let's just see if we are powerful enough to recreate ourselves, to reflect ourselves in the machine without a really good articulated reason of why are we actually doing that? Like ultimately we advance science technology to improve the human condition is recreating intelligence in a machine actually improving the human condition. And yeah, I would advocate for like, we should be having a serious discussion about what is that that we actually need in the world? In general, like we need technology aside. We need clean water, clean air, better education, better healthcare. We need to give people the fundamental building blocks we're having a dignified life. And then we can ask, how are the ways that AI can actually get us there? And if the evidence overwhelmingly right now shows us that a certain type of approach to AI is actually undermining all those fundamental building blocks, how do we dramatically and urgently shift towards other AI tools that fortify them instead? - So I think part of the problem is this idea that technology by definition improves human life is a very deeply held one. I think it's very deeply ingrained in society, right? And people will, in this connection, in my experience anyway, site two things. One, every time a new technology comes along, there are the doomsayers who turn out to be wrong. And you know, they, these are the same people who oppose the printing press, et cetera, et cetera. And the other one is I can give you all this. There are all kinds of inventions and the story is one of overwhelming success and flourishing, right? And you can come up with some counter examples, like the atomic bomb or perhaps television even. I might even offer the internet, but I would be a lot of it for doing so, right? This is, this is the point, the atomic bomb is probably the closest you get. But the other thing about it is, I really appreciate your model of engineering is identifying a problem and then offering a solution rather than just going into some green field and seeing what happens. But actually that's kind of the story of social media, isn't it? I mean, I don't know that social media turned up as some kind of clearly designed solution to a problem that was widely apprehended. Sometimes the creation of the thing invents the problem that is solving. It's our famous Henry Ford quote. If I asked the people what they want, they'd have asked for faster horses, right? They didn't realize, actually, that this could open up all sorts of possibilities. And given that it worked for social media in the sense that it was pursued relentlessly, it is here now, it creates all kinds of problems, but we more or less live with them and shrug. And if I gave anyone the option of pulling out the plug tomorrow, very few would take it. It seems that that logic is so deeply ingrained in us and actually so broadly accepted that, of course, it's going to apply to AI. Because the opposition you would have to AI would be, yeah, but that's just because your imagination is limited. You don't know the problems that you have, that it's going to end up solving. And the way that technology now works in this hyper complex age is not that it's a one-to-one relationship where I identify a problem and then solve it. It's that we improve things at scale and at speed in ways that are unlimited by our imagination and observations. Yeah, well, I think social media is exactly the problem that I was talking about. We're Silicon Valley. I mean, all of these-- Oh, I would fully agree with that, by the way. Yeah, exactly. Like Silicon Valley decided that its model of innovation would be to just push technologies onto people because it's cool and connecting everyone in the world. No one asked for that. No one needed that. But Mark Zuckerberg decided it was cool. So now we have social media and it's starting to create all of these consequences where we actually didn't need so many connections. No, but we still loved it. And it was done with our equity essence, right? We were the ones who created the Facebook friends and all of those things, and then couldn't get off it. Yes, yes, and no, right? You could also say the same thing about cigarettes where it's like, oh, we just-- we loved it. And therefore, we should just keep allowing people to smoke and smoke and smoke until it ends up giving them lung cancer. But it is different from the Henry Ford example in that what Henry Ford was saying is like he was saying that if you would ask people, everyone had a problem of we need to get to a place faster from A to B. But he was saying if you ask people then what the solution is to that problem, they will lack imagination in solving. And that's the innovation of engineering is you then bring the imagination to figure out how to solve the problem. But the problem was well articulated in that instance. With social media, the problem was not well articulated. No one-- I mean, maybe you could argue that people were like, oh, it would be nice to connect with long-lost childhood friends. But then of course, the solution is highly over-engineered to, OK, let's just connect you actually with everyone in the world, not just your long-lost friends. Yeah, yeah. Karen, we waited, we shifted things to get you in. And you did not disappoint. I can tell you that. We'd love to keep going for another hour. Unfortunately, we don't have that option. But thank you so much for joining us for being in the country and for being prepared to spare some precious time for us. Karen Howell, award-winning technology journalist author of Empire of AI, inside the reckless race for total domination, I guess, for this week's edition of The Mindfield, which is now at an end. And we issued, of course, with apologies to Stephen Fry, who I think did absolutely nothing but in the cold side, that he got along the way. That's it for this episode of The Mindfield. Don't forget, if you have a question for the upcoming Mailbag episode, we would love to hear from you. The 12th of September is the cut off, The Mindfield at abc.net.au. The Mindfield at abc.net.au. Look forward to reading those questions. And in the meantime, you can catch us again next week on ABC Radio National, or anytime you like, on the ABC Listen app. [MUSIC PLAYING] You've been listening to an ABC podcast. Discover more great ABC podcasts, live radio, and exclusives on the ABC Listen app.