Go back

Why AI Won't Replace Workers, But Will Crash The Economy. With Cory Doctorow

48m 51s

Why AI Won't Replace Workers, But Will Crash The Economy. With Cory Doctorow

In this interview, Cory Doctorow discusses his new book, *The Reverse Centaur's Guide to Life After AI*, arguing that the current AI boom is a speculative bubble destined to burst, with severe economic and social consequences. He explains that AI is a "reverse centaur" technology, where machines drive humans, unlike a "centaur" where humans guide machines. This occurs because bosses adopt automation to maximize throughput and profit, often at the expense of quality and worker well-being, as evidenced by high injury rates in automated warehouses. Doctorow highlights the unsustainable economics of AI, with massive investments yielding minimal revenue, and predicts that when the bubble pops, it will wipe out significant stock market value, leading to austerity and potentially fueling fascism as people lose their livelihoods and savings. He emphasizes that AI's current deployment replaces skilled workers with defective chatbots, eroding hard-won expertise and process knowledge. However, he acknowledges AI can be useful when workers drive its adoption, such as in his own typo-checking or Patrick Ball's code writing, but stresses the need for discernment. Doctorow also criticizes AI hype, dismissing apocalyptic scenarios from whistleblowers as exaggerated, and argues that the real threat is the economic collapse, not AI sentience. He calls for bursting the bubble sooner to mitigate damage, warning that companies are already unloading overvalued stock onto ordinary investors.

Transcription

8963 Words, 48867 Characters

English
And when the AI bubble pops, we're going to see a massive economic vacuum, right? 35% of the stock market vaporized. We will likely see governments defaulting to austerity, which has the well-known effect of driving support for fascism, right? When you lose the things that matter to you, your doctor, your job, your school, you become a sucker for people who show up and say the reason you've lost the things that matter is that undeserving others have taken it and you should embark on racist pogroms, which you know, we see taking place everywhere, we see austerity. So we'll see more austerity, we'll see a breakdown in our systems. And I'm absolutely delighted to welcome back journalist blogger, author and friend of the show, Corey Doctorow, a pleasure to be talking to you again, sir. Thank you very much. Pleasure to be back on. You recently appeared on a lot of really high profile programs and the comments under these videos are pretty unanimous, most people are saying, this is the voice we need right now. Additionally, I have to confess much is it's hard for a cold-hearted Eastern European like me that I've never received so much private correspondence urging me to have someone back on the pod than I did after our last conversation. So before we talk about your new book, I wanted to ask you, why do you think your analysis of the internet and AI and of big tech in general is resonating with so many folks right now? Well, I've been at this for 25 years. So I work for the Electronic Frontier Foundation. It's a digital rights group based in America, but I was there in inaugural European director and I worked on their behalf in more than 30 countries. And having done this in 30 countries for 25 years, I like to think I have some context that rather than treating all of these issues as though they occur in a vacuum like we were just hit by a meteor called AI or a meteor hit Silicon Valley 10 years ago and created in shittification, I can tell you where these things came from. I can tell you how we can get rid of them. I don't have to resort to mystical explanations like Mark Zuckerberg's the smartest person that ever lived and succeeded where Rasputin and Mesmer and MK Ultra and pick up artists failed and he built a mind control ray and that's why everyone's acting so weird because Facebook's got a mind control ray. I can ground this in material and economic phenomena. I can tell you why we have bubbles. I can tell you why my fellow progressives are wrong when they say endless growth is the ideology of a tumor. The reason they want endless growth in Silicon Valley is nothing to do with ideology. It's about material factors and I think that when you can ground the phenomena around you that are making you angry and sad and scared in real material things, that because they're real material, suggest ways that we can resolve them, that even if you're still angry and sad and upset at the end of that, at least you know what to do. But at least you're not scared in the sense that you're a prisoner of history. Maybe you're scared because now you understand the forces arrayed against you and what needs to be done to deal with them. I mean Elon Musk also has a lot of context but not even his followers are claiming that he's the voice of reason. Well, I think Elon Musk makes people follow makes people excited about him because he deals exclusively in a kind of adivism and kind of libidinal emotions. And while there's something libidinal and something out ofistic about words like insinification and the minor license to vulgarity in moments where everything seems to be going wrong, I like to think that I really respect the people who take the time to listen to me enough to assume that they have the native intelligence to follow a nuanced argument that doesn't assume that you know, Jews are importing people of color to outbreed white people and end Western civilization or any of the other absurd things that Elon Musk believes, including that USA doesn't do anything important. All right, you mentioned insinification a couple of times now. We talked about it last time and the term really took off. It's everywhere. I can only imagine how hard some Google execs jaws must be clenching when they hear this word being applied to one of their products. Did you ever, I'm just wondering, did you ever get like a sassy email or anything from any of these people? I mean, I sometimes hear from them. I mostly, what's happened is I've had a couple of high profile speaking gigs canceled because one of the sponsors objected and no one's ever told me who the sponsor was, but you know, you look at the sponsor roles and it's all the usual suspects. So I think that's how that is being felt. You know, I guess it's always a mystery to me why someone would invite me to speak at a conference. That's being sponsored by firms that I have a long track record of saying, in temperate by true things about that make people angry at those companies, you know, more power to them for inviting them. I'm grateful for it. But I think that there is this very foreseeable outcome, I think, in many cases that those sponsors are going to object eventually, you know, I think 20 years ago, 25 years ago, the sector was diverse enough. There were enough players in it that there was always a kind of balance of power within it when I look at institutions like the W3C, the World Wide Web Consortium that designed standards for the web. And I look back in those days when there were, you know, hundreds of members of almost the same size, and so the worst impulses of the members kind of died in their tracks because there would be other members who had better impulses. And I look now where when there's a group within the W3C, a very large firms that want to do something that's obviously bad for the web, they win. It's not because the W3C is weak or foolish or evil. It's because when your, you know, budget depends on a dozen of your highest paying members and they all decide that they want one thing, it's very hard to resist them. You've since written a new book, The Reverse Centaur's Guide to Life After AI, how to think about artificial intelligence before it's too late. I know you've had to explain the concept of a reverse centaur, a gazillion times already, but for anyone who might be unfamiliar still, would you be so kind as to break it down again? Yeah. So it won't surprise you to know that this is not the first time that labor and automation have come into conflict with one another. And so there's a pretty good literature explaining how different kinds of automation scenarios have played out in labor markets and for the people who rely on the outputs of labor, you know, the things that we get from workers who use machines. And one of the, I guess iron laws or truisms or cornerstones of that theory is that when workers adopt automation, they typically do so in order to improve quality. But when bosses drive automation adoption, it's typically in service to increasing throughput. Because you know, if you are paying a monthly subscription or if you've acquired an asset that's now depreciating, you want to maximize your use of it, right? That's how you maximize your profit, right? You've got an expense now. You want to, you want to make the most of it. You want to, as they say, sweat the asset. And you know, this is particularly true where you have situations where you have monopolies or cartels or do-opolis that control the market because they can make worse quality products and still sell them because they're the only game in town. And so this is what you see when, you know, the dollar general that's replaced the grocery store in your town makes everything smaller and more expensive. What are you going to do? Make the bus 10 miles to the last remaining grocery store, you know, so in those conditions you would expect things to get worse. And so this is where Centaurus and Reverse Centaurus come in. In automation theory, a centaur is someone who is assisted by a machine. The analogy here is, you know, the human head on the horse's body, the body being the body of a horse, it's tireless, it's strong, it's fast, but it doesn't have discernment or judgment. And that comes from the human head. So the human head is directing this strong, fast, enduring system, machine body. So when you're riding a bicycle, you're a centaur, right, even look like one, when you're using a spell checker, you're a centaur too, you know, there are lots of ways in which we can become Centaurus, a reverse centaur, the corollary, that's the reverse, right? That is the horse's head on the human body, that's the machine driving the human, that's the machine making decisions about the human. And of course, the machine is still stronger than you, faster than you, it has more endurance than you. There's just some part of the task that it can't do. And so you are, it's peripheral, and you are, it's helped me. And because you are slower and weaker and get tired more easily, it is going to work you at the limit of your endurance and at the limit of your capability. And that means that you're not just going to get used by the machine, you're going to get used up by the machine. And we see this in our kind of memes and tropes about automation and labor, you know, people who've seen, I love Lucy, we'll know this classic scene where Lucy and Ethel are trying to put chocolates in the chocolate box and the conveyor belt's going so fast that the chocolates are going everywhere or that classic scene in modern times where Charlie Chaplin gets sucked in the machine and ground up in the gears. You know, this is the logical consequence of a capital driven automation, an inferior product that makes the workers who produce its lives hell and it's worth it. Why the most automated warehouses we've ever built, the Amazon warehouses, also have the highest rate of injury of any warehouse in the sector, that's not a coincidence as the consequence. If you're spending seven or eight figures on automation, you're not going to run the machine slower than you need to. You will run them at the limit of the worker's capacity. Of course, if you're working at your limit for long stretches, you will eventually make a mistake. When you make that mistake in a warehouse, sometimes you get impaled on a forklift. The last part of the title of the book is even scarier than what you were just talking about. I'm not going to lie. What do you mean by before it's too late? Well, look, AI is a bubble, which is not to say that AI technology isn't useful. If labor was driving automation adoption here, if labor was in charge of AI, I'm sure we'd see lots of people finding useful things. For it, in fact, when you meet programmers who say, "Oh, I write code with AI," and it's the best code I've ever written, and then you meet other programmers who are writing code with AI, and they say, "I can't believe how bad this code is." I work in avionics, and I don't think anyone should ever get on an airplane because no one has ever produced tech debt at the scale that we're producing and now with these AI tools. The way to understand that contradiction is not that the first group is lying, or the second group is lying. It's that the first group are centaurs, and the second group are reverse centaurs. There are useful things we can do with AI, but the bubble where they've raised now over $1.4 trillion on less than $50 billion a year in revenue, that bubble demands that we fire as many workers as possible and replace them with defective chatbots, and even if we do that, we're still going to see a rupture, that bubble is going to burst. 35% of the US stock market, the seven companies, are AI companies, and they trade around the same $100 billion IOU really fast, pretending it's in all their bank accounts at once. Every generation of AI technology loses more money than the previous generation. Every new AI customer loses money for the business. Every time an AI customer touches the AI, the business loses even more money. And so here we are, replacing critical functions in our society with technology that will eventually just be switched off. We are wiping out the kind of hard-won expertise, what's called sometimes process knowledge, right? All the things that workers know that can't really be written down, that tacit knowledge that's held between workers within a firm, across different divisions, and across different firms and a sector, and up and known as supply chain. All of that's dust being wiped out because we're firing those workers, and they're becoming discouraged or they're retraining or they're retiring. And when the AI bubble pops, we're going to see a massive economic vacuum, 35% of the stock market vaporized. When you lose the things that matter to you, your doctor, your job, your school, you become a sucker for people who show up and say the reason you've lost the things that matter is that undeserving others have taken it, and you should embark on racist pogroms, which you know, we see taking place everywhere, we see austerity. So we'll see more austerity, we'll see a breakdown in our systems, and it's going to be really hard. And the sooner we burst the bubble, the better. And if we wait too long, some of these AI companies are going to unload on regular investors. Right now, most of that AI money is big dumb money. It's institutional, it's golf states, it's billionaires. And you know, those people are really itching to get you to relieve them of their soon-to-be-worthless stock. We just saw this with Elon Musk and SpaceX, where he did a sweetheart deal with the NASDAQ 100 to debut SpaceX in the index of top NASDAQ shares. That's never done. Normally, if a company debuts on the stock market, it's ineligible for inclusion and in the index until it's had several quarters of sustained profitability, because it might just be like a pump and dumb, it might be a swindle. And those indexes are held by everyday sabers. That's what most people put their retirement savings into as an index fund. And if you manage an index fund, you are legally obliged to buy stock from every company in the index. So if SpaceX is in the top 100, you have to go buy SpaceX. And what Musk did when he floated SpaceX is he didn't just get this exemption to the restriction on debuting in the index. He also restricted the shares that the company put in up on the market. So the company put almost no shares up for sale. All of the shares that were sold to fund managers who were required to buy them at any price with your retirement savings. All those shares came from insiders who were not obliged to follow the normal legal requirement of a lockout, where insiders aren't allowed to sell for the first 90 days or 180 days again, in case it's a pump and dumb. And so we see these companies already starting to unload everything on ordinary sabers to create basically bag holders. And again, that it's not just the injustice, although this is very important or the social problem of people having their retirement savings wiped out. It's also what it does to the trusted institutions and to our social cohesion. It paves the way for authoritarian movements when people lose everything. And so that's the real risk that we face right now is the collapse of our important firms and institutions because we'll replace skilled workers with AI and then the AI will be shut off when the companies run out of money. And that will make people furious because they will enter a period of austerity right after having all of their savings stolen by stock swindlers who are also insiders. And that's going to be just catastrophic. What do you mean AI is just going to be switched off because after the dot com bubble burst, the internet wasn't just like switched off. A lot of people are saying, yeah, I think they're true. So the dot coms had very different unit economics, right? So a lot of them were dumb, right? For sure. But as a sector, the web had great unit economics. Every new web customer made more money for web companies. Every time you use the web, the web made more money and every generation of the web made more money than the previous generation of the web. That's not how AI works. As I've said, AI is the opposite of that. And so while there will be, and this is actually quite important, there's going to be a ton of GPUs kicking around. It's going to be like data centers looking for buyers at fire sale prices. There's going to be skilled workers who know how to do things with AI who don't have to answer to their stupid bosses anymore. And there's going to be these open source models that have barely been optimized and that whenever anyone tries to optimize them, they get all kinds of new efficiencies out of them. That stuff will be around. But like no one's going to be keeping open AI around in its current form. Even AI is a business that sells $100 bills for a dollar each, right? And like even if you wipe out all of its capital expenditure overhang and start over again through a bankruptcy, you can't, if you continue to give away $100 bills for a dollar each, you will quickly end up exactly where you started. So you know, it might be that you can replace a worker who costs you $100,000 a year with a chatbot that costs you $1,000 a year and not really care that the outputs are half as good as the worker produced. But if the chatbot costs $120,000 a year and the outputs are half as good, that's not the same situation at all. And that's what we would be in if we were going to continue to consume AI resources at the rate we're consuming them now, but I actually had to pay for them instead of having them defraint by Gulf State sovereign wealth funds and Silicon Valley billionaires. Yeah, I'm not economist, but selling $100 bill for $1 doesn't sound like. Oh, and it gets worse than that, you know, when the companies were trying to clean up their balance sheets because they were all talking about their IPOs last month, they were like, what if we start charging $5 for these $100 bills? And like all the CEOs who bought AI for their businesses and said, everyone has to use AI and if I catch you not using AI, I'm going to fire you. They turned around, they said, these $100 bills are not worth $5. I'm sorry, like we just don't get enough use out of them to pay $5 for them. We are going to institute, you know, company wide bands on using AI, limits on using AI. You're going to have to ask your manager for more tokens if you use up the tokens that you've been given, right? So like, it's not just that they're selling $100 bills for a dollar. It's that no one will pay more than $1 for their $100 bills. It's really bad. I want to talk about the ideology behind this AI push a little bit. And I totally believe that the engineers working on these LLMs have probably read too much science fiction and are working earnestly towards creating some sort of benevolent super intelligence or whatever. However, when it comes to their bosses, the CEOs who are steering this thing, do you reckon that in their minds, this whole project is basically given what we're talking about right now, just like every capitalist's wet dream. So a campaign to simply eliminate human labor. I think that there's two things going on here. I think for sure some of these guys definitely, like, would like to live in a world without people. I think there's a kind of solipsism that goes with being a billionaire. You know, you can't really inflict the kind of pain at scale that you have to in order to make a billion dollars or hundreds of billions of dollars. If you really believe other people feel the way you feel, are as real as you are. You know, and that's true of this class when they're face-to-face in individual interactions. So, you know, the Epstein class just couldn't have believed that the children who were on that island are as real as their own children. But it's also true, especially when they deal with people as a statistical abstraction. You know, Jeff Bezos and his army of drivers and warehouse workers who don't even get pea breaks, right? Like, he would never stipulate to those indignities, right? Like, he gets a catheter fitted when he goes up on his spaceship, right? He doesn't just like pee in a bottle, right? Like, so even under the most extreme circumstances, like there's lots of toilets on his yacht, right? The idea that you don't get a pea break is something for other people and not for him. And so, you know, you get Elon Musk calling everyone who disagrees with them and NPC and so on. So, I think there's a lot of these people who just don't think other people are as real as they are, or at least, you know, everyday people. And certainly, like, if you run a social media business, I think it's probably really bad. Like Mark Zuckerberg deals with people entirely as statistical abstractions. You know, he's just like twisting dials in the back end of Facebook to see if he can events, human behaviors at scale, right? So that this is really a disindividuated way of confronting people. But then you have on top of that, you have what Milton Keynes called the beauty contest. So Milton Keynes said that making money by investing is not a matter of picking the most beautiful person in the beauty contest, the company that's going to make the most money. Making money as an investor is about picking the person the other judges will think is most beautiful because you are trying to pick the stock that other people are going to want to buy irrespective of whether you think that stock is good. So, you know, there's people who invest in, you know, the supplements that Andrew Tate and, you know, those other chud podcasters, Joe Rogan, and so on, that they fall. Oh, yeah. Yeah. There's people who invest in those in those supplements who don't believe that the supplements work, right? They know that it's just a way of getting extremely expensive urine. But they also know that there's like an infinite army of vulnerable, desperate young man who will pay everything they have for the supplements. You don't have to think the supplements will work to invest in the supplements. You don't have to think that, you know, Gwyneth Paltrow has a jade egg that will make your vagina better to think that women will buy the jade egg and risk their lives by putting it in their bodies, right? And so I think by the same token, there's a lot of people who think, you know, bosses are really easy cells for a sales pitch about how you can replace workers with software because like workers are a pain in the ass, they demand moral consideration, and even more so, they provoke these kind of ego shattering confrontations where like you have an idea, you want to do a thing. And then you have to go out and talk to a worker who knows how that thing works. And they tell you that your idea is like stupid or illegal or impossible or it's going to kill people. And you know, eventually you start to have this suspicion that while you're telling yourself you're in the driver's seat, you know, the fact is that if you didn't show off for work, everything would be fine. And if they didn't show off for work, everything would stop. And maybe you're in the back seat with a Fisher Price Steering Wheel. And so AI is like really the potential of like wiring the steering wheel directly into the drivetrain, right? You have an idea and it just gets shit out by the AI. And so, you know, you don't have to believe that AI can do that to believe that bosses will buy it if you promise them that. And you know, you don't have to be able, you don't have to be fast enough to outrun the berry. You just have to be fast enough to outrun everyone else. So long as you get out of the AI position before everyone realizes that the AI can be useful for workers who want to augment their work, but not particularly useful for replacing workers and you get out ahead, then it doesn't matter that it was a bad investment. It doesn't matter that the company failed. You got out okay, right? There's plenty of people who made a lot of money on NFTs. NFTs were are and always will be stupid and a bunch of people made a lot of money for NFTs. I mean, it's bizarre how they're not even trying to come up with a sensible justification for all this. Are they just a half-baked version of a trust me, bro? They're saying we have to throw everything into their company so they get to AGI as fast as possible. And then out of the goodness of their hearts, they will reap the benefits on the rest of humanity. Call me Mr. cynical, but when in the history of the planet that did ever happen, that the guy at the top and usually it was a guy said, you know what, now that I robbed all of you blind, I think I'm going to divide the pie fairly, let everyone have a nice piece. Or maybe I'm, you know, just not as well-reader my history as I thought it was. Well, you know, there's a connection between this and some of the more esoteric forms of billionaire mysticism. So, you know, the Sambankman freed and a bunch of his cronies practiced a thing called effective altruism. Yeah. It has its roots and some pretty benign stuff like trying to empirically determine whether you save more lives investing in like anti-malaria research or mosquito nets, right? But it quickly became this kind of hyper-utilitarian, self-paradying venture where you earn to give, so you take the most destructive, harmful job you can that pays the most to earn as much money as possible, no matter how much pain you inflict on people around you. In the name of earning so much money that you can give it to causes that will make people's lives better. But of course, the way to improve the most lives is not to help people who are alive today, but rather to invest all your money in improving the lives of 10 to the 53 artificial humans who will come into existence on Venus in 10,000 years. Because if you make even the smallest improvement to the lives of 10 to the 53 people, it will swamp all the benefit you could possibly confer on a mere 8 billion people who are alive today, right? This is a very weird idea, right? And like it is, you know, as they say, I think, again, this is, I can't remember if this is Keynes or Galberth, but you know, it's the endless pursuit of man's oldest ideological project to find a superior moral justification for selfishness, right? So much of this, you know, kind of cash is out there, but I think that it's worse than the idea that they're just hand waving and saying, trust me, bro, and like using the shitty trope from science fiction novels where computers get complicated enough and then they wake up, which is like what science fiction writers do because we're lazy and we just want to get straight to the futuristic parable, we don't want to come up with a bunch of ways in which computers can become self-aware, we just say like, oh, you add enough transistors and now it's a brain, right? It's worse than that. When you read accounts of the NIH researchers who pleaded with the Doge boys to not cancel their long running cancel research, cancel their long running cancer research, sorry, it's hard to say. But what the Doge kids said was we don't need cancer research because general AI is around the corner and when general AI arrives, it will cure cancer. So it's worse than that. It's not just that they're making false promises. It's that they're discontinuing the actual promising things. If you want a more immediate version of this, every time an AI bro turns on a gas turbine and is asked about the climate impact, they say don't worry when general AI comes along it'll solve the climate emergency, right? And our job now is to outrun the second law of thermodynamics, right? Before we melt the ice caps, we have to make the general AI that tells us what to do once the ice caps melt, right? It's an incredible gamble they are making with the planet and their basis for asserting that a general AI is around the corners is deeply foolish. They're basically saying that if you teach enough words to the word guessing program, it will wake up, which is like saying that if you breed horses to run fast enough, one of them will give birth to a locomotive, right? Like it doesn't matter how fast the horse runs, it's not a train and it doesn't matter how well the program guesses words, it's not conscious. Consciousness is not just guessing words. Okay, since we're here, let's talk about the state of this technology today because for somebody like me, usually uses it as some sort of like an expanded Google search. Sure. I honestly have no idea if this thing is like the philosopher's stone or just a really confident parrot. And the discourse at large is really not helping me. For example, when I opened my ex yesterday, God helped me. My feed was suddenly full of folks saying that AI has cracked some old mathematical problem in apparently. It's true. It's not bullshit. On the other hand, immediately after that, I watched a video on the work of Jennifer Daudna, she's a scientist who I want to have on the podcast someday. She's a pioneer in gene editing and she said that chat bots are basically useless when it comes to innovation and that she thinks it's going to stay that way for a while. So which of the two is it? I'm confused. You know, one of the things that AI, and this is the first generation of AI we've ever had. AI was coined at a conference in the 1950s and we've had many different kinds of AI emerge. And you know, while they're all radically different and while AI has been a marketing term for each of them, what they all share in common is that they were all a bid to replicate and possibly even replace some function that we think of as an ately human. And what each generation of AI has done is showed us that while there is a unique human way to do certain things that there are other ways to do it too, right? The way that humans play chess is not the way that AI plays chess. And while they may look the same from the outside, they're doing something different. And you can achieve the state of a chess game with AI, like a successful chess game, a victorious chess game, in a way that's different from the way people do it, which is interesting, super interesting, right? I mean, for a long time, you look at the history of human flight, and for a long time, we were kind of becald in those doldrums of trying to mimic birds. We just built ornithoctors, which are basically planes that flap their wings, and they never worked. And we had to invent a totally different way of flying to actually leave the ground. There's a famous quote about thinking machines, where a computer scientist was asked, can machines think, and they answered, can submarines swim? Summarines don't swim, right? But they move through the water. They do some of the things we do with thinking. I don't know if you're married, but I'm married. And I often can predict what my wife is going to say. And my wife's auto-complete, I can often predict what my wife is going to say, because it knows the corpus of her language, right? She sends a lot of text messages, and so it knows something about this statistical, just likely statistical distribution of the replies to certain messages. But when my wife says something that isn't what I predict, I can events a pretty good theory of what's going on to make her say something that is out of character. And the AI is at sea. And that is the difference between understanding and extrapolating, right? Understanding involves a theory. And the promise of big data, which AI is the latest generation of, you know, 15 years ago, we were calling this big data, and now we're calling this AI, was theory-free inference, right? The idea behind big data was that you could observe by crunching population scale data that these molecules taken in this combination to people who had these lifestyle factors and these pathologies produced a cure or produced some kind of improvement in their state without ever having to know why, right? That this was theory-free inference. And so long as it works, you know, be nice to know the theory, but so long as it works, you don't necessarily have to know the theory. And indeed, there's a lot of medicines we use that are theory-free in the sense that we've observed that they work. We've we've double-blind tested them. We've checked for side effects and so on. And we use them even though we don't understand at all or completely how they work. And AI is this idea that we can just effectively give up on having a theoretical basis for the world and switch to these theory-free inferences. And the problem is that it's very hard, especially when you're talking about unusual circumstances to unwind false positives and detect them when you don't have a theory because you don't really know how to test the theory, right? A theory-free inference is by definition not falsifiable, right? You can't say, if this weren't true, I would do, then then you would expect X or Y to happen under these circumstances, right? Because you don't have a theory, you don't have a way to know why it's true or why it's not true. And so as a way of knowing it's quite limited, which isn't to say it's useless, you know, there are lots of theory-free things that we have discovered with AI that are interesting. And if scientists were being told, here is a tool, you have all the budget you've ever had before, you have all the freedom you've ever had before. And here's a tool that does some new multivariant analysis that lets you do stuff that you might otherwise have had to hire a programmer for or a statistician, or that you might have had to spend several days designing a custom stats, crunching software for or algorithms for. But this is the thing that we'll do it for you. It will make some mistakes. So you're going to have to double check it. You have to be careful about how you use it. I bet we would have scientists doing all kinds of great things. But that's not how it's working, right? What you've got is the doge situation. Fire scientists replace them with software, make the survivors do more with less and expect the same outputs. And I think that is very frightening. You know, I'll give you an example from my life. So I have an extremely treatable form of cancer. And this means that I'm spending a lot of time with radiologists. And in fact, about three weeks ago now, a radiologist told me that I am cancer-free, which was very good news. And one of the things that we hear a lot about is that AI might be able to spot solid mass tumors that radiologists can't spot. And if there was a sales call at my hospital, which is the Kaiser clinic in Los Angeles, next to the church of Scientology, because that's how we roll in LA. And if there was like some AI salesman who like, you know, deaked out the guys from Scientology trying to get them to come in and take a personality test and went up to the CEO's office and said, look, here's the deal. Right now, you employ 10 radiologists. They do 100 x-rays a day a piece. They cost you $300,000 each. You're spending $3 million a year on radiology and processing a thousand x-rays a day. I want you to spend an extra million dollars on software. And that's going to sit on the shoulder of those radiologists. And once for twice a day, it's going to tap them on the shoulder and it's going to say, take another look at that x-ray. And they're going to do 98 x-rays a day instead of 100. So you probably, in addition to the million bucks for the software, going to have to hire a part-time radiologist to pick up the slack, which you're going to save people's lives. I'd be very happy. That as a cancer patient, I love the sound of that. That's not how you turn a trillion dollars in annual expenditure on $50 billion in revenue into a profit. The way that you turn $50 billion in revenue on a trillion dollars expenditure into profit is you have the AI salesman. Deek out the Scientologists trying to give them the personality test. Go into the CEO's office and say, right now, you spend $3 million on 10 radiologists. Fire nine of them. Save $2.7 million a year. Split that between you and Sam Altman. Make the remaining radiologists mark the AI's homework. They sign their name to the bottom of every radiology report. And when someone dies, you blame them. They are the accountability sink. Right? That's that's the thing to worry about. Now, radiologists right now have a 10th labor power. When I talk to radiologists about how they're using AI right now, it sounds more like the first scenario than the second one. But the point of AI, the reason that investors think that you can sell AI to people like hospital administrators is that it will erode labor power so you can fire nine of the radiologists, right? Like that's the that's the reason it's been capitalized the way it's been capitalized. That is the politics of this artifact as it exists in the contingencies that produced it. And while we could produce a different set of contingencies and different set of politics, that is the politics of this version of AI that we're living with right now. That all makes sense. But to play devil's advocate just for a second. But gets me rattled are some of these whistleblower. So ex employees who are not warning against the dangers of AI a couple of days ago, I checked out an interview with Daniel Kokotalo who was a researcher open AI and apparently refused to sign a non disclosure agreement for two million dollars, which would ensure that he keeps his mouth shut about their sinister affairs or something. He then created AI 2027, which I'm heard everybody has heard of. It's a detailed scenario of how things might go haywire with AI soon. I mean, listening to the guy talk, he just seemed genuinely frightened by what he thinks it's about to come. And that gets me worried. Yeah, look, you know, I don't know this guy. I've seen I've seen AI 2027. I don't think much of it. And you know, I've also seen the rebuttals from people who don't work for AI companies, but do work in AI like, you know, research, the research team at Princeton who maintain the AI is a normal technology website who wrote a very detailed critique of AI 2027. And I got to say like, we have a history in Silicon Valley, in tech, of guys like going into the bathroom, shining a flashlight under their chin, looking in the mirror and going, and then getting really scared, right? We had it 10 years ago with a bunch of people who used to work at Facebook insisting that they had been trained to be evil dopamine hacking wizards. And that the reason your grandpa was a QAnon is that they were so good at hacking his dopamine. And not that your grandpa was a racist forever, that you kind of, you know, kept his mouth shut about it. And then what Facebook did was like, trap him in a room full of a bunch of other races who laughed every time he said something racist. So now he can't shut the fuck up about it, right? Like, that's like a much more parsimonious explanation. It doesn't require any wizards. And you know, there is something I think quite exciting about having been an evil dopamine hacking wizard, because now you can promise that you're a good dopamine hacking wizard. And the operative word here is wizard, right? Like how cool is it to have been a wizard? And I'm not saying these guys don't think they're wizards. I think they might believe they're wizards. I just don't think they're their wizards. I mean, on another note, this thing is being crammed into every single product, whether it makes sense or not. At the same time, apart from a couple of like overzealous LinkedIn boomers, I don't know a living soul that would be excited about it. Nobody's happy. Half the time, I want to take a shower after I've used one of these chat bots. And yet it seems we're not really putting up much of a fight against this really brutally forced adoption. - I don't know. I think, you know, you see a lot of workers who are getting pissed off about it. You see a lot of creative workers who are getting pissed off about the data. Anti-data center movement is gigantic. You know, there's a lot of people are very angry about it. I do want to say though, I know lots of people who use AI who I think it was quite credible and who say that they got great uses out of it. People use it in VFX, not to like render final images, but to like automate parts of the process. The best programmer I know is a guy called Patrick Ball, who founded an NGO called the Human Rights Data Analysis Group, HRDAG.org. And HRDAG, they specialize in using fragmentary evidence from war crimes, civil wars, genocides. And they use rigorous statistical methods to build out a picture of what aspects are probably missing from the data by interpolating the data. And then they testify at human rights tribunals, truth and reconciliation. They worked on my loss of it, and they worked in Rios Monte and East Timor, South Africa, the Colombian Civil War. Patrick's, as I say, one of the best programmers I know, brilliant statistician, brilliant programmer, he tells me he's using a whole bunch of AI agents to write the best code he's ever written, right? But Patrick also has more discernment than any programmer I've ever met. So, you know, if I was ever gonna trust a programmer to know which parts of the job to automate and how to double check to make sure that when you automated it, you hadn't accidentally produced a bunch of tech debt. Patrick's the guy I would trust, you know? And so, again, this is one of those things where you have labor-driven adoption. Look, I use a chatbot running on my computer, a local open-source model, to check my blog posts for typos in the morning, 'cause I publish my blog on all these different platforms, I publish it as threads on all the different social media platforms, I publish it on Tumblr and Medium, and I put it on a newsletter, and I put it on the web. And when I find a typo, my readers really want me to know about it, they're very invested in my knowing about my typos. And so, I have to go fix it, or I'll just get like a million emails. And once I've published it to 10 places, fixing just like a period that's gone astray is like 15 minutes work. And so, if there's six typos, that's like an hour and a half that I lose, right? So, I drop it into this local chatbot, I type, find typos, and find all kinds of typos, right? It's just like, and many of them aren't typos. And I ignore those because I have discernment. I've been telling search engines that they were, or not search engines, spell checkers that they were wrong for 30 years. I've published more than 30 books with major publishers. I've been telling copy editors and proofreaders that they were wrong for 25 years, right? And I'm perfectly capable of telling a chatbot. It's wrong too. The difference, of course, is the chatbot never learns, whereas the proofer, when you say, that's not a typo, that's intentional, doesn't send you back to manuscript next time, with those errors in it, it's not perfect by any means, but like, I was never gonna hire a proofer to read my blog post in the morning in the 10 minutes after I write it before I publish it. It's just ridiculous. So, yeah, I'm very happy to have it. It's fine. It is absolutely fine. It is a normal technology. My first word processor was a program printed in a magazine that I bought at the corner store and took home and typed into my Apple 2 Plus. I was 10 years old, it was 1981. I've used a lot of word processor since. A lot of them have had new features. A lot of those new features I thought were stupid, and when other people use them, I think it made their writing worse. Some of them were really useful, and I use them all the time. I've never seen a feature for a word processor where I said, that's so cool. We should bet the world's economy on it. And by the way, we should probably also put all the writers in a wood chipper now that it exists, right? And like, that's the problem, right? Is we are treating what cash is out to plugins for common software that are sometimes useful and sometimes not. And when workers drive the automation, can produce things that make your life materially better. And we're treating them as genies and gods and demons as the end of the world or as the beginning of a new one. And it's not that. If AI destroys the world, it's going to be because the bubble is burst. It's not going to be because the software was so cool that it woke up and turned us into paper clips. - Maybe this is a math thought that's completely divorced from reality, but is it possible we just get bored by it all? I mean, how many more AI generated videos of Tom Cruise fighting a large Pokemon on some Chinese roof? Can I watch on X before I just want to retreat to a pre-historic cave forever? I mean, I can't be the only person that's already tired of these smartest chat bots. - Come on. - Come on, run. So this is maybe a good place to end the conversation on. I do think that a lot of that AI stuff is getting washed really quick. And this has always been one of my critiques about a certain kind of technological doom saying that even where you have a thing that works really well at the start, right? Someone's found a little cognitive blind spot. Someone's found a thing that makes people curious or catches their attention that unless there's some substance to it, that it very quickly becomes something that you're in nerd too, right? You know, do you remember when up where the headlines were invented? And it was like 11 facts about socks. You'll never believe number three. And I click through like half a dozen of those headlines. And then I realized that like number three just wasn't that interesting and never would be. And I never clicked through one of those headlines again. Now, that's not to say that that form of headline in the hands of someone who's a good writer who knows how to infuse it with actual communication, who's creative, who's smart, and who has something to say, 'cause you know, the problem with those upper the headlines is they were always on articles that were stupid, right? The best thing about them was the headline. But if you have something smart to say, there are probably aspects of those techniques that will work to capture people's attention and hold it because you're writing about something real and substantive. And I believe that we will have AI generated text and images and so on, not in vast quantities, not fucking emails, routine business emails that have been inflated three bullet points into five paragraphs of, you know, Florida bullshit. But like we will have AI generated material in our lives that will be being used in the background, whatever, you know, to help you get stuff done and to, you know, make things better. And some of it will be stupid and some of it will be fine. And most of the people who are making it now will have moved on to something else because they have nothing to say. They're just fiddling around with a new web toy. And in that regard, AI is a normal technology 'cause I've just described the trajectory of many technologies that we've had in the last couple of decades. That's how it went with Buzzfeed quizzes, right? It's how it's gonna go with other web toys too. - Corey, always a pleasure. Thank you so much to chat with you. - Thank you so much for listening. Please subscribe to our YouTube, follow and rate on Spotify and Apple podcasts. You would help me a bunch with that. Also feel free to support me on Patreon. This is a one-person podcast. I do everything by my little self. It's a lot of work, especially with no team, no ads and no real marketing. But you can help by hopping on Patreon and finding the smart cookies podcast on there and giving me our money, of course. Super hard for small creators to get through so your support would mean a lot. And then last but certainly not least, I have to thank my existing patrons without you. There is no smart cookies podcast, which makes you not only absolute legends, but also in effect actual producers of this show. So eternal thanks. Go, go, go to Lenox. Do me hold this. Charlie Balbeck. You guys are the best. I'm deeply touched. So thank you. - Hey everybody, ladies, luck here. And we're celebrating America's 250th birthday. Now all summer long, I'm gonna be celebrating by playing at spinquest.com, which is an American-owned social casino. It obviously features over a thousand slot games and live blackjack, live crabs, live bubble crabs. Head on over to spinquest.com. Get yourself a $30 coin pack for just 10 bucks. - Spinquist is a free to play social casino. Boydware prohibited, visit spinquest.com for more details.

Podcast Summary

Key Points:

  1. Cory Doctorow argues that AI is an economic bubble, with over $1.4 trillion invested against less than $50 billion in annual revenue, and predicts a market crash that could vaporize 35% of the US stock market.
  2. He introduces the concept of "reverse centaurs"—machines driving humans—where automation adopted by bosses prioritizes throughput over quality, exploiting workers and leading to inferior products and high injury rates, as seen in Amazon warehouses.
  3. Doctorow warns that the AI bubble's collapse will trigger austerity, erode social systems, and fuel authoritarianism, as people lose jobs, savings, and trust in institutions.
  4. He contrasts "centaur" use of AI (workers using tools to improve quality) with "reverse centaur" use (bosses replacing workers), emphasizing that AI's current deployment is driven by capital, not labor interests.
  5. He critiques AI hype, noting that while AI can assist in tasks like code writing or typo checking, it lacks understanding and is often used to fire workers and cut costs, risking critical expertise and process knowledge.
  6. Doctorow dismisses doomsday AI scenarios from whistleblowers, likening them to past tech panic, and argues the real danger is the bubble bursting, not AI becoming sentient.

Summary:

In this interview, Cory Doctorow discusses his new book, *The Reverse Centaur's Guide to Life After AI*, arguing that the current AI boom is a speculative bubble destined to burst, with severe economic and social consequences. He explains that AI is a "reverse centaur" technology, where machines drive humans, unlike a "centaur" where humans guide machines. This occurs because bosses adopt automation to maximize throughput and profit, often at the expense of quality and worker well-being, as evidenced by high injury rates in automated warehouses.

Doctorow highlights the unsustainable economics of AI, with massive investments yielding minimal revenue, and predicts that when the bubble pops, it will wipe out significant stock market value, leading to austerity and potentially fueling fascism as people lose their livelihoods and savings. He emphasizes that AI's current deployment replaces skilled workers with defective chatbots, eroding hard-won expertise and process knowledge. However, he acknowledges AI can be useful when workers drive its adoption, such as in his own typo-checking or Patrick Ball's code writing, but stresses the need for discernment.

Doctorow also criticizes AI hype, dismissing apocalyptic scenarios from whistleblowers as exaggerated, and argues that the real threat is the economic collapse, not AI sentience. He calls for bursting the bubble sooner to mitigate damage, warning that companies are already unloading overvalued stock onto ordinary investors.

FAQs

A reverse centaur is when a machine drives a human, with the machine making decisions and the human being used up by it. Unlike a centaur, where a human directs a machine, a reverse centaur works the human at their limit, often leading to harm and inferior outcomes.

The AI bubble will burst because AI companies are spending over $1.4 trillion on less than $50 billion in annual revenue, with each generation of AI losing more money. When it pops, it could vaporize 35% of the US stock market, leading to economic collapse and austerity.

Replacing skilled workers with AI erodes hard-won expertise and process knowledge, and when the AI bubble bursts, critical systems will fail. This loss of jobs and savings can drive support for authoritarian movements and fascism.

When workers drive AI adoption, they use it to improve quality, like a centaur. When bosses drive it, they prioritize increasing throughput, often leading to worse products and worker exploitation, as seen in Amazon warehouses with high injury rates.

AI relies on theory-free inference, meaning it finds patterns without understanding underlying causes. This makes it hard to detect false positives, especially in unusual circumstances, limiting its usefulness in fields like scientific innovation.

Doctorow dismisses AI 2027 and similar warnings as 'flashlight under the chin' scares, comparing them to past tech panic narratives. He argues they lack material grounding and are often self-aggrandizing, unlike his own analysis based on economic factors.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.