Why AI Won't Replace Workers, But Will Crash The Economy. With Cory Doctorow
48m 51s
In this interview, Cory Doctorow discusses his new book, *The Reverse Centaur's Guide to Life After AI*, arguing that the current AI boom is a speculative bubble destined to burst, with severe economic and social consequences. He explains that AI is a "reverse centaur" technology, where machines drive humans, unlike a "centaur" where humans guide machines. This occurs because bosses adopt automation to maximize throughput and profit, often at the expense of quality and worker well-being, as evidenced by high injury rates in automated warehouses. Doctorow highlights the unsustainable economics of AI, with massive investments yielding minimal revenue, and predicts that when the bubble pops, it will wipe out significant stock market value, leading to austerity and potentially fueling fascism as people lose their livelihoods and savings. He emphasizes that AI's current deployment replaces skilled workers with defective chatbots, eroding hard-won expertise and process knowledge. However, he acknowledges AI can be useful when workers drive its adoption, such as in his own typo-checking or Patrick Ball's code writing, but stresses the need for discernment. Doctorow also criticizes AI hype, dismissing apocalyptic scenarios from whistleblowers as exaggerated, and argues that the real threat is the economic collapse, not AI sentience. He calls for bursting the bubble sooner to mitigate damage, warning that companies are already unloading overvalued stock onto ordinary investors.
And when the AI bubble pops, we're going to see a massive economic vacuum, right?
35% of the stock market vaporized.
We will likely see governments defaulting to austerity, which has the well-known effect
of driving support for fascism, right?
When you lose the things that matter to you, your doctor, your job, your school, you become
a sucker for people who show up and say the reason you've lost the things that matter
is that undeserving others have taken it and you should embark on racist pogroms, which
you know, we see taking place everywhere, we see austerity.
So we'll see more austerity, we'll see a breakdown in our systems.
And I'm absolutely delighted to welcome back journalist blogger, author and friend of
the show, Corey Doctorow, a pleasure to be talking to you again, sir.
Thank you very much.
Pleasure to be back on.
You recently appeared on a lot of really high profile programs and the comments under
these videos are pretty unanimous, most people are saying, this is the voice we need right
now.
Additionally, I have to confess much is it's hard for a cold-hearted Eastern European
like me that I've never received so much private correspondence urging me to have someone
back on the pod than I did after our last conversation.
So before we talk about your new book, I wanted to ask you, why do you think your analysis
of the internet and AI and of big tech in general is resonating with so many folks right now?
Well, I've been at this for 25 years.
So I work for the Electronic Frontier Foundation.
It's a digital rights group based in America, but I was there in inaugural European director
and I worked on their behalf in more than 30 countries.
And having done this in 30 countries for 25 years, I like to think I have some context
that rather than treating all of these issues as though they occur in a vacuum like we
were just hit by a meteor called AI or a meteor hit Silicon Valley 10 years ago and created
in shittification, I can tell you where these things came from.
I can tell you how we can get rid of them.
I don't have to resort to mystical explanations like Mark Zuckerberg's the smartest person
that ever lived and succeeded where Rasputin and Mesmer and MK Ultra and pick up artists
failed and he built a mind control ray and that's why everyone's acting so weird because
Facebook's got a mind control ray.
I can ground this in material and economic phenomena.
I can tell you why we have bubbles.
I can tell you why my fellow progressives are wrong when they say endless growth is the
ideology of a tumor.
The reason they want endless growth in Silicon Valley is nothing to do with ideology.
It's about material factors and I think that when you can ground the phenomena around
you that are making you angry and sad and scared in real material things, that because
they're real material, suggest ways that we can resolve them, that even if you're still
angry and sad and upset at the end of that, at least you know what to do.
But at least you're not scared in the sense that you're a prisoner of history.
Maybe you're scared because now you understand the forces arrayed against you and what needs
to be done to deal with them.
I mean Elon Musk also has a lot of context but not even his followers are claiming that
he's the voice of reason.
Well, I think Elon Musk makes people follow makes people excited about him because he deals
exclusively in a kind of adivism and kind of libidinal emotions.
And while there's something libidinal and something out ofistic about words like insinification
and the minor license to vulgarity in moments where everything seems to be going wrong, I like
to think that I really respect the people who take the time to listen to me enough to
assume that they have the native intelligence to follow a nuanced argument that doesn't
assume that you know, Jews are importing people of color to outbreed white people and
end Western civilization or any of the other absurd things that Elon Musk believes, including
that USA doesn't do anything important.
All right, you mentioned insinification a couple of times now.
We talked about it last time and the term really took off.
It's everywhere.
I can only imagine how hard some Google execs jaws must be clenching when they hear this
word being applied to one of their products.
Did you ever, I'm just wondering, did you ever get like a sassy email or anything from
any of these people?
I mean, I sometimes hear from them.
I mostly, what's happened is I've had a couple of high profile speaking gigs canceled
because one of the sponsors objected and no one's ever told me who the sponsor was, but
you know, you look at the sponsor roles and it's all the usual suspects.
So I think that's how that is being felt.
You know, I guess it's always a mystery to me why someone would invite me to speak at
a conference.
That's being sponsored by firms that I have a long track record of saying, in temperate
by true things about that make people angry at those companies, you know, more power to
them for inviting them.
I'm grateful for it.
But I think that there is this very foreseeable outcome, I think, in many cases that those
sponsors are going to object eventually, you know, I think 20 years ago, 25 years ago,
the sector was diverse enough.
There were enough players in it that there was always a kind of balance of power within
it when I look at institutions like the W3C, the World Wide Web Consortium that designed
standards for the web.
And I look back in those days when there were, you know, hundreds of members of almost
the same size, and so the worst impulses of the members kind of died in their tracks
because there would be other members who had better impulses.
And I look now where when there's a group within the W3C, a very large firms that want to
do something that's obviously bad for the web, they win.
It's not because the W3C is weak or foolish or evil.
It's because when your, you know, budget depends on a dozen of your highest paying members
and they all decide that they want one thing, it's very hard to resist them.
You've since written a new book, The Reverse Centaur's Guide to Life After AI, how to think
about artificial intelligence before it's too late.
I know you've had to explain the concept of a reverse centaur, a gazillion times already,
but for anyone who might be unfamiliar still, would you be so kind as to break it down again?
Yeah.
So it won't surprise you to know that this is not the first time that labor and automation
have come into conflict with one another.
And so there's a pretty good literature explaining how different kinds of automation scenarios
have played out in labor markets and for the people who rely on the outputs of labor,
you know, the things that we get from workers who use machines.
And one of the, I guess iron laws or truisms or cornerstones of that theory is that when
workers adopt automation, they typically do so in order to improve quality.
But when bosses drive automation adoption, it's typically in service to increasing throughput.
Because you know, if you are paying a monthly subscription or if you've acquired an asset
that's now depreciating, you want to maximize your use of it, right?
That's how you maximize your profit, right?
You've got an expense now.
You want to, you want to make the most of it.
You want to, as they say, sweat the asset.
And you know, this is particularly true where you have situations where you have monopolies
or cartels or do-opolis that control the market because they can make worse quality products
and still sell them because they're the only game in town.
And so this is what you see when, you know, the dollar general that's replaced the grocery
store in your town makes everything smaller and more expensive.
What are you going to do?
Make the bus 10 miles to the last remaining grocery store, you know, so in those conditions
you would expect things to get worse.
And so this is where Centaurus and Reverse Centaurus come in.
In automation theory, a centaur is someone who is assisted by a machine.
The analogy here is, you know, the human head on the horse's body, the body being the
body of a horse, it's tireless, it's strong, it's fast, but it doesn't have discernment
or judgment.
And that comes from the human head.
So the human head is directing this strong, fast, enduring system, machine body.
So when you're riding a bicycle, you're a centaur, right, even look like one, when you're
using a spell checker, you're a centaur too, you know, there are lots of ways in which
we can become Centaurus, a reverse centaur, the corollary, that's the reverse, right?
That is the horse's head on the human body, that's the machine driving the human, that's
the machine making decisions about the human.
And of course, the machine is still stronger than you, faster than you, it has more endurance
than you.
There's just some part of the task that it can't do.
And so you are, it's peripheral, and you are, it's helped me.
And because you are slower and weaker and get tired more easily, it is going to work
you at the limit of your endurance and at the limit of your capability.
And that means that you're not just going to get used by the machine, you're going to
get used up by the machine.
And we see this in our kind of memes and tropes about automation and labor, you know, people
who've seen, I love Lucy, we'll know this classic scene where Lucy and Ethel are trying
to put chocolates in the chocolate box and the conveyor belt's going so fast that the
chocolates are going everywhere or that classic scene in modern times where Charlie Chaplin
gets sucked in the machine and ground up in the gears.
You know, this is the logical consequence of a capital driven automation, an inferior
product that makes the workers who produce its lives hell and it's worth it.
Why the most automated warehouses we've ever built, the Amazon warehouses, also have
the highest rate of injury of any warehouse in the sector, that's not a coincidence as
the consequence.
If you're spending seven or eight figures on automation, you're not going to run the
machine slower than you need to.
You will run them at the limit of the worker's capacity.
Of course, if you're working at your limit for long stretches, you will eventually make
a mistake.
When you make that mistake in a warehouse, sometimes you get impaled on a forklift.
The last part of the title of the book is even scarier than what you were just talking
about.
I'm not going to lie.
What do you mean by before it's too late?
Well, look, AI is a bubble, which is not to say that AI technology isn't useful.
If labor was driving automation adoption here, if labor was in charge of AI, I'm sure we'd
see lots of people finding useful things.
For it, in fact, when you meet programmers who say, "Oh, I write code with AI," and it's
the best code I've ever written, and then you meet other programmers who are writing code
with AI, and they say, "I can't believe how bad this code is."
I work in avionics, and I don't think anyone should ever get on an airplane because no one
has ever produced tech debt at the scale that we're producing and now with these AI tools.
The way to understand that contradiction is not that the first group is lying, or the
second group is lying.
It's that the first group are centaurs, and the second group are reverse centaurs.
There are useful things we can do with AI, but the bubble where they've raised now over
$1.4 trillion on less than $50 billion a year in revenue, that bubble demands that we fire
as many workers as possible and replace them with defective chatbots, and even if we do
that, we're still going to see a rupture, that bubble is going to burst.
35% of the US stock market, the seven companies, are AI companies, and they trade around the
same $100 billion IOU really fast, pretending it's in all their bank accounts at once.
Every generation of AI technology loses more money than the previous generation.
Every new AI customer loses money for the business.
Every time an AI customer touches the AI, the business loses even more money.
And so here we are, replacing critical functions in our society with technology that will eventually
just be switched off.
We are wiping out the kind of hard-won expertise, what's called sometimes process knowledge,
right?
All the things that workers know that can't really be written down, that tacit knowledge
that's held between workers within a firm, across different divisions, and across different
firms and a sector, and up and known as supply chain.
All of that's dust being wiped out because we're firing those workers, and they're becoming
discouraged or they're retraining or they're retiring.
And when the AI bubble pops, we're going to see a massive economic vacuum, 35% of the
stock market vaporized.
When you lose the things that matter to you, your doctor, your job, your school, you become
a sucker for people who show up and say the reason you've lost the things that matter
is that undeserving others have taken it, and you should embark on racist pogroms, which
you know, we see taking place everywhere, we see austerity.
So we'll see more austerity, we'll see a breakdown in our systems, and it's going to be really
hard.
And the sooner we burst the bubble, the better.
And if we wait too long, some of these AI companies are going to unload on regular
investors.
Right now, most of that AI money is big dumb money.
It's institutional, it's golf states, it's billionaires.
And you know, those people are really itching to get you to relieve them of their soon-to-be-worthless
stock.
We just saw this with Elon Musk and SpaceX, where he did a sweetheart deal with the NASDAQ
100 to debut SpaceX in the index of top NASDAQ shares.
That's never done.
Normally, if a company debuts on the stock market, it's ineligible for inclusion and in
the index until it's had several quarters of sustained profitability, because it might
just be like a pump and dumb, it might be a swindle.
And those indexes are held by everyday sabers.
That's what most people put their retirement savings into as an index fund.
And if you manage an index fund, you are legally obliged to buy stock from every company
in the index.
So if SpaceX is in the top 100, you have to go buy SpaceX.
And what Musk did when he floated SpaceX is he didn't just get this exemption to the
restriction on debuting in the index.
He also restricted the shares that the company put in up on the market.
So the company put almost no shares up for sale.
All of the shares that were sold to fund managers who were required to buy them at any price
with your retirement savings.
All those shares came from insiders who were not obliged to follow the normal legal requirement
of a lockout, where insiders aren't allowed to sell for the first 90 days or 180 days
again, in case it's a pump and dumb.
And so we see these companies already starting to unload everything on ordinary sabers to
create basically bag holders.
And again, that it's not just the injustice, although this is very important or the social
problem of people having their retirement savings wiped out.
It's also what it does to the trusted institutions and to our social cohesion.
It paves the way for authoritarian movements when people lose everything.
And so that's the real risk that we face right now is the collapse of our important firms
and institutions because we'll replace skilled workers with AI and then the AI will be shut
off when the companies run out of money.
And that will make people furious because they will enter a period of austerity right
after having all of their savings stolen by stock swindlers who are also insiders.
And that's going to be just catastrophic.
What do you mean AI is just going to be switched off because after the dot com bubble burst,
the internet wasn't just like switched off.
A lot of people are saying, yeah, I think they're true.
So the dot coms had very different unit economics, right? So a lot of them were dumb, right?
For sure.
But as a sector, the web had great unit economics.
Every new web customer made more money for web companies.
Every time you use the web, the web made more money and every generation of the web made
more money than the previous generation of the web.
That's not how AI works.
As I've said, AI is the opposite of that.
And so while there will be, and this is actually quite important, there's going to be a ton
of GPUs kicking around. It's going to be like data centers looking for buyers at fire sale
prices.
There's going to be skilled workers who know how to do things with AI who don't have to
answer to their stupid bosses anymore.
And there's going to be these open source models that have barely been optimized and that
whenever anyone tries to optimize them, they get all kinds of new efficiencies out of them.
That stuff will be around.
But like no one's going to be keeping open AI around in its current form.
Even AI is a business that sells $100 bills for a dollar each, right?
And like even if you wipe out all of its capital expenditure overhang and start over again
through a bankruptcy, you can't, if you continue to give away $100 bills for a dollar each,
you will quickly end up exactly where you started.
So you know, it might be that you can replace a worker who costs you $100,000 a year with
a chatbot that costs you $1,000 a year and not really care that the outputs are half
as good as the worker produced.
But if the chatbot costs $120,000 a year and the outputs are half as good, that's not
the same situation at all.
And that's what we would be in if we were going to continue to consume AI resources at
the rate we're consuming them now, but I actually had to pay for them instead of having
them defraint by Gulf State sovereign wealth funds and Silicon Valley billionaires.
Yeah, I'm not economist, but selling $100 bill for $1 doesn't sound like.
Oh, and it gets worse than that, you know, when the companies were trying to clean up their
balance sheets because they were all talking about their IPOs last month, they were like,
what if we start charging $5 for these $100 bills?
And like all the CEOs who bought AI for their businesses and said, everyone has to use
AI and if I catch you not using AI, I'm going to fire you.
They turned around, they said, these $100 bills are not worth $5.
I'm sorry, like we just don't get enough use out of them to pay $5 for them.
We are going to institute, you know, company wide bands on using AI, limits on using
AI.
You're going to have to ask your manager for more tokens if you use up the tokens that
you've been given, right?
So like, it's not just that they're selling $100 bills for a dollar.
It's that no one will pay more than $1 for their $100 bills.
It's really bad.
I want to talk about the ideology behind this AI push a little bit.
And I totally believe that the engineers working on these LLMs have probably read too much
science fiction and are working earnestly towards creating some sort of benevolent super
intelligence or whatever.
However, when it comes to their bosses, the CEOs who are steering this thing, do you reckon
that in their minds, this whole project is basically given what we're talking about
right now, just like every capitalist's wet dream.
So a campaign to simply eliminate human labor.
I think that there's two things going on here.
I think for sure some of these guys definitely, like, would like to live in a world without
people.
I think there's a kind of solipsism that goes with being a billionaire.
You know, you can't really inflict the kind of pain at scale that you have to in order
to make a billion dollars or hundreds of billions of dollars.
If you really believe other people feel the way you feel, are as real as you are.
You know, and that's true of this class when they're face-to-face in individual interactions.
So, you know, the Epstein class just couldn't have believed that the children who were on
that island are as real as their own children.
But it's also true, especially when they deal with people as a statistical abstraction.
You know, Jeff Bezos and his army of drivers and warehouse workers who don't even get
pea breaks, right?
Like, he would never stipulate to those indignities, right?
Like, he gets a catheter fitted when he goes up on his spaceship, right?
He doesn't just like pee in a bottle, right?
Like, so even under the most extreme circumstances, like there's lots of toilets on his yacht, right?
The idea that you don't get a pea break is something for other people and not for him.
And so, you know, you get Elon Musk calling everyone who disagrees with them and NPC and
so on.
So, I think there's a lot of these people who just don't think other people are as real
as they are, or at least, you know, everyday people.
And certainly, like, if you run a social media business, I think it's probably really bad.
Like Mark Zuckerberg deals with people entirely as statistical abstractions.
You know, he's just like twisting dials in the back end of Facebook to see if he can
events, human behaviors at scale, right?
So that this is really a disindividuated way of confronting people.
But then you have on top of that, you have what Milton Keynes called the beauty contest.
So Milton Keynes said that making money by investing is not a matter of picking the most
beautiful person in the beauty contest, the company that's going to make the most money.
Making money as an investor is about picking the person the other judges will think is
most beautiful because you are trying to pick the stock that other people are going to
want to buy irrespective of whether you think that stock is good.
So, you know, there's people who invest in, you know, the supplements that Andrew Tate and,
you know, those other chud podcasters, Joe Rogan, and so on, that they fall.
Oh, yeah.
Yeah.
There's people who invest in those in those supplements who don't believe that the supplements
work, right?
They know that it's just a way of getting extremely expensive urine.
But they also know that there's like an infinite army of vulnerable, desperate young man who
will pay everything they have for the supplements.
You don't have to think the supplements will work to invest in the supplements.
You don't have to think that, you know, Gwyneth Paltrow has a jade egg that will make your
vagina better to think that women will buy the jade egg and risk their lives by putting
it in their bodies, right?
And so I think by the same token, there's a lot of people who think, you know, bosses are
really easy cells for a sales pitch about how you can replace workers with software because
like workers are a pain in the ass, they demand moral consideration, and even more so,
they provoke these kind of ego shattering confrontations where like you have an idea,
you want to do a thing.
And then you have to go out and talk to a worker who knows how that thing works.
And they tell you that your idea is like stupid or illegal or impossible or it's going
to kill people.
And you know, eventually you start to have this suspicion that while you're telling yourself
you're in the driver's seat, you know, the fact is that if you didn't show off for
work, everything would be fine.
And if they didn't show off for work, everything would stop.
And maybe you're in the back seat with a Fisher Price Steering Wheel.
And so AI is like really the potential of like wiring the steering wheel directly into
the drivetrain, right?
You have an idea and it just gets shit out by the AI.
And so, you know, you don't have to believe that AI can do that to believe that bosses
will buy it if you promise them that.
And you know, you don't have to be able, you don't have to be fast enough to outrun the
berry.
You just have to be fast enough to outrun everyone else.
So long as you get out of the AI position before everyone realizes that the AI can be
useful for workers who want to augment their work, but not particularly useful for replacing
workers and you get out ahead, then it doesn't matter that it was a bad investment.
It doesn't matter that the company failed.
You got out okay, right?
There's plenty of people who made a lot of money on NFTs.
NFTs were are and always will be stupid and a bunch of people made a lot of money for
NFTs.
I mean, it's bizarre how they're not even trying to come up with a sensible justification
for all this.
Are they just a half-baked version of a trust me, bro?
They're saying we have to throw everything into their company so they get to AGI as fast
as possible.
And then out of the goodness of their hearts, they will reap the benefits on the rest
of humanity.
Call me Mr. cynical, but when in the history of the planet that did ever happen, that
the guy at the top and usually it was a guy said, you know what, now that I robbed all
of you blind, I think I'm going to divide the pie fairly, let everyone have a nice piece.
Or maybe I'm, you know, just not as well-reader my history as I thought it was.
Well, you know, there's a connection between this and some of the more esoteric forms of
billionaire mysticism.
So, you know, the Sambankman freed and a bunch of his cronies practiced a thing called
effective altruism.
Yeah.
It has its roots and some pretty benign stuff like trying to empirically determine whether
you save more lives investing in like anti-malaria research or mosquito nets, right?
But it quickly became this kind of hyper-utilitarian, self-paradying venture where you earn to give,
so you take the most destructive, harmful job you can that pays the most to earn as much
money as possible, no matter how much pain you inflict on people around you.
In the name of earning so much money that you can give it to causes that will make people's
lives better.
But of course, the way to improve the most lives is not to help people who are alive today,
but rather to invest all your money in improving the lives of 10 to the 53 artificial humans
who will come into existence on Venus in 10,000 years.
Because if you make even the smallest improvement to the lives of 10 to the 53 people, it will
swamp all the benefit you could possibly confer on a mere 8 billion people who are alive today,
right?
This is a very weird idea, right?
And like it is, you know, as they say, I think, again, this is, I can't remember if this
is Keynes or Galberth, but you know, it's the endless pursuit of man's oldest ideological
project to find a superior moral justification for selfishness, right?
So much of this, you know, kind of cash is out there, but I think that it's worse than
the idea that they're just hand waving and saying, trust me, bro, and like using the
shitty trope from science fiction novels where computers get complicated enough and then
they wake up, which is like what science fiction writers do because we're lazy and we just
want to get straight to the futuristic parable, we don't want to come up with a bunch of ways
in which computers can become self-aware, we just say like, oh, you add enough transistors
and now it's a brain, right?
It's worse than that.
When you read accounts of the NIH researchers who pleaded with the Doge boys to not cancel
their long running cancel research, cancel their long running cancer research, sorry, it's
hard to say.
But what the Doge kids said was we don't need cancer research because general AI is
around the corner and when general AI arrives, it will cure cancer.
So it's worse than that.
It's not just that they're making false promises.
It's that they're discontinuing the actual promising things.
If you want a more immediate version of this, every time an AI bro turns on a gas turbine
and is asked about the climate impact, they say don't worry when general AI comes along
it'll solve the climate emergency, right?
And our job now is to outrun the second law of thermodynamics, right?
Before we melt the ice caps, we have to make the general AI that tells us what to do
once the ice caps melt, right?
It's an incredible gamble they are making with the planet and their basis for asserting
that a general AI is around the corners is deeply foolish.
They're basically saying that if you teach enough words to the word guessing program,
it will wake up, which is like saying that if you breed horses to run fast enough, one
of them will give birth to a locomotive, right?
Like it doesn't matter how fast the horse runs, it's not a train and it doesn't matter
how well the program guesses words, it's not conscious.
Consciousness is not just guessing words.
Okay, since we're here, let's talk about the state of this technology today because for
somebody like me, usually uses it as some sort of like an expanded Google search.
Sure.
I honestly have no idea if this thing is like the philosopher's stone or just a really
confident parrot.
And the discourse at large is really not helping me.
For example, when I opened my ex yesterday, God helped me.
My feed was suddenly full of folks saying that AI has cracked some old mathematical problem
in apparently.
It's true.
It's not bullshit.
On the other hand, immediately after that, I watched a video on the work of Jennifer
Daudna, she's a scientist who I want to have on the podcast someday.
She's a pioneer in gene editing and she said that chat bots are basically useless when
it comes to innovation and that she thinks it's going to stay that way for a while.
So which of the two is it?
I'm confused.
You know, one of the things that AI, and this is the first generation of AI we've ever
had.
AI was coined at a conference in the 1950s and we've had many different kinds of AI emerge.
And you know, while they're all radically different and while AI has been a marketing
term for each of them, what they all share in common is that they were all a bid to replicate
and possibly even replace some function that we think of as an ately human.
And what each generation of AI has done is showed us that while there is a unique
human way to do certain things that there are other ways to do it too, right? The way that humans play
chess is not the way that AI plays chess. And while they may look the same from the outside,
they're doing something different. And you can achieve the state of a chess game with AI,
like a successful chess game, a victorious chess game, in a way that's different from the way
people do it, which is interesting, super interesting, right? I mean, for a long time, you look at the
history of human flight, and for a long time, we were kind of becald in those doldrums of
trying to mimic birds. We just built ornithoctors, which are basically planes that flap their wings,
and they never worked. And we had to invent a totally different way of flying to actually leave
the ground. There's a famous quote about thinking machines, where a computer scientist was asked,
can machines think, and they answered, can submarines swim? Summarines don't swim, right? But
they move through the water. They do some of the things we do with thinking. I don't know if you're
married, but I'm married. And I often can predict what my wife is going to say. And my wife's
auto-complete, I can often predict what my wife is going to say, because it knows the corpus of
her language, right? She sends a lot of text messages, and so it knows something about this
statistical, just likely statistical distribution of the replies to certain messages. But
when my wife says something that isn't what I predict, I can events a pretty good theory
of what's going on to make her say something that is out of character. And the AI is at sea.
And that is the difference between understanding and extrapolating, right? Understanding
involves a theory. And the promise of big data, which AI is the latest generation of, you know,
15 years ago, we were calling this big data, and now we're calling this AI, was theory-free
inference, right? The idea behind big data was that you could observe by crunching population
scale data that these molecules taken in this combination to people who had these lifestyle
factors and these pathologies produced a cure or produced some kind of improvement in their
state without ever having to know why, right? That this was theory-free inference. And so long
as it works, you know, be nice to know the theory, but so long as it works, you don't necessarily
have to know the theory. And indeed, there's a lot of medicines we use that are theory-free in
the sense that we've observed that they work. We've we've double-blind tested them. We've checked
for side effects and so on. And we use them even though we don't understand at all or completely
how they work. And AI is this idea that we can just effectively give up on having a theoretical
basis for the world and switch to these theory-free inferences. And the problem is that it's very
hard, especially when you're talking about unusual circumstances to unwind false positives and
detect them when you don't have a theory because you don't really know how to test the theory,
right? A theory-free inference is by definition not falsifiable, right? You can't say,
if this weren't true, I would do, then then you would expect X or Y to happen under these
circumstances, right? Because you don't have a theory, you don't have a way to know why it's true
or why it's not true. And so as a way of knowing it's quite limited, which isn't to say it's useless,
you know, there are lots of theory-free things that we have discovered with AI that are interesting.
And if scientists were being told, here is a tool, you have all the budget you've ever had before,
you have all the freedom you've ever had before. And here's a tool that does some new multivariant
analysis that lets you do stuff that you might otherwise have had to hire a programmer for
or a statistician, or that you might have had to spend several days designing a custom stats,
crunching software for or algorithms for. But this is the thing that we'll do it for you. It will
make some mistakes. So you're going to have to double check it. You have to be careful about how you
use it. I bet we would have scientists doing all kinds of great things. But that's not how it's
working, right? What you've got is the doge situation. Fire scientists replace them with software,
make the survivors do more with less and expect the same outputs. And I think that is very frightening.
You know, I'll give you an example from my life. So I have an extremely treatable form of cancer.
And this means that I'm spending a lot of time with radiologists. And in fact, about three
weeks ago now, a radiologist told me that I am cancer-free, which was very good news. And one of the
things that we hear a lot about is that AI might be able to spot solid mass tumors that radiologists
can't spot. And if there was a sales call at my hospital, which is the Kaiser clinic in Los Angeles,
next to the church of Scientology, because that's how we roll in LA. And if there was like some AI
salesman who like, you know, deaked out the guys from Scientology trying to get them to come in
and take a personality test and went up to the CEO's office and said, look, here's the deal.
Right now, you employ 10 radiologists. They do 100 x-rays a day a piece. They cost you
$300,000 each. You're spending $3 million a year on radiology and processing a thousand x-rays a day.
I want you to spend an extra million dollars on software. And that's going to sit on the shoulder of
those radiologists. And once for twice a day, it's going to tap them on the shoulder and it's going
to say, take another look at that x-ray. And they're going to do 98 x-rays a day instead of 100.
So you probably, in addition to the million bucks for the software, going to have to hire a part-time
radiologist to pick up the slack, which you're going to save people's lives. I'd be very happy.
That as a cancer patient, I love the sound of that. That's not how you turn a trillion dollars
in annual expenditure on $50 billion in revenue into a profit. The way that you turn $50 billion
in revenue on a trillion dollars expenditure into profit is you have the AI salesman.
Deek out the Scientologists trying to give them the personality test. Go into the CEO's office
and say, right now, you spend $3 million on 10 radiologists. Fire nine of them. Save $2.7 million
a year. Split that between you and Sam Altman. Make the remaining radiologists mark the AI's homework.
They sign their name to the bottom of every radiology report. And when someone dies, you blame them.
They are the accountability sink. Right? That's that's the thing to worry about. Now,
radiologists right now have a 10th labor power. When I talk to radiologists about how they're
using AI right now, it sounds more like the first scenario than the second one. But the point of AI,
the reason that investors think that you can sell AI to people like hospital administrators
is that it will erode labor power so you can fire nine of the radiologists, right? Like that's the
that's the reason it's been capitalized the way it's been capitalized. That is the politics of this
artifact as it exists in the contingencies that produced it. And while we could produce a different
set of contingencies and different set of politics, that is the politics of this version of AI that
we're living with right now. That all makes sense. But to play devil's advocate just for a second.
But gets me rattled are some of these whistleblower. So ex employees who are not warning against
the dangers of AI a couple of days ago, I checked out an interview with Daniel Kokotalo who was a
researcher open AI and apparently refused to sign a non disclosure agreement for two million
dollars, which would ensure that he keeps his mouth shut about their sinister affairs or something.
He then created AI 2027, which I'm heard everybody has heard of. It's a detailed scenario of how
things might go haywire with AI soon. I mean, listening to the guy talk, he just seemed
genuinely frightened by what he thinks it's about to come. And that gets me worried.
Yeah, look, you know, I don't know this guy. I've seen I've seen AI 2027. I don't think much of it.
And you know, I've also seen the rebuttals from people who don't work for AI companies, but do work
in AI like, you know, research, the research team at Princeton who maintain the AI is a normal
technology website who wrote a very detailed critique of AI 2027. And I got to say like,
we have a history in Silicon Valley, in tech, of guys like going into the bathroom,
shining a flashlight under their chin, looking in the mirror and going, and then getting
really scared, right? We had it 10 years ago with a bunch of people who used to work at Facebook
insisting that they had been trained to be evil dopamine hacking wizards. And that the reason
your grandpa was a QAnon is that they were so good at hacking his dopamine. And not that your
grandpa was a racist forever, that you kind of, you know, kept his mouth shut about it. And
then what Facebook did was like, trap him in a room full of a bunch of other races who laughed
every time he said something racist. So now he can't shut the fuck up about it, right? Like,
that's like a much more parsimonious explanation. It doesn't require any wizards. And you know,
there is something I think quite exciting about having been an evil dopamine hacking wizard,
because now you can promise that you're a good dopamine hacking wizard. And the operative word
here is wizard, right? Like how cool is it to have been a wizard? And I'm not saying these guys
don't think they're wizards. I think they might believe they're wizards. I just don't think they're
their wizards.
I mean, on another note,
this thing is being crammed into every single product,
whether it makes sense or not.
At the same time,
apart from a couple of like overzealous LinkedIn boomers,
I don't know a living soul that would be excited about it.
Nobody's happy.
Half the time,
I want to take a shower after I've used one of these chat bots.
And yet it seems we're not really putting up
much of a fight against this
really brutally forced adoption.
- I don't know.
I think, you know, you see a lot of workers
who are getting pissed off about it.
You see a lot of creative workers
who are getting pissed off about the data.
Anti-data center movement is gigantic.
You know, there's a lot of people are very angry about it.
I do want to say though,
I know lots of people who use AI
who I think it was quite credible
and who say that they got great uses out of it.
People use it in VFX,
not to like render final images,
but to like automate parts of the process.
The best programmer I know is a guy called Patrick Ball,
who founded an NGO called the Human Rights Data Analysis Group,
HRDAG.org.
And HRDAG, they specialize in using fragmentary evidence
from war crimes, civil wars, genocides.
And they use rigorous statistical methods
to build out a picture of what aspects
are probably missing from the data
by interpolating the data.
And then they testify at human rights tribunals,
truth and reconciliation.
They worked on my loss of it,
and they worked in Rios Monte and East Timor, South Africa,
the Colombian Civil War.
Patrick's, as I say, one of the best programmers
I know, brilliant statistician, brilliant programmer,
he tells me he's using a whole bunch of AI agents
to write the best code he's ever written, right?
But Patrick also has more discernment
than any programmer I've ever met.
So, you know, if I was ever gonna trust a programmer
to know which parts of the job to automate
and how to double check to make sure
that when you automated it,
you hadn't accidentally produced a bunch of tech debt.
Patrick's the guy I would trust, you know?
And so, again, this is one of those things
where you have labor-driven adoption.
Look, I use a chatbot running on my computer,
a local open-source model,
to check my blog posts for typos in the morning,
'cause I publish my blog on all these different platforms,
I publish it as threads on all the different social media
platforms, I publish it on Tumblr and Medium,
and I put it on a newsletter, and I put it on the web.
And when I find a typo,
my readers really want me to know about it,
they're very invested in my knowing about my typos.
And so, I have to go fix it,
or I'll just get like a million emails.
And once I've published it to 10 places,
fixing just like a period that's gone astray
is like 15 minutes work.
And so, if there's six typos,
that's like an hour and a half that I lose, right?
So, I drop it into this local chatbot, I type,
find typos, and find all kinds of typos, right?
It's just like, and many of them aren't typos.
And I ignore those because I have discernment.
I've been telling search engines that they were,
or not search engines, spell checkers
that they were wrong for 30 years.
I've published more than 30 books with major publishers.
I've been telling copy editors and proofreaders
that they were wrong for 25 years, right?
And I'm perfectly capable of telling a chatbot.
It's wrong too.
The difference, of course, is the chatbot never learns,
whereas the proofer, when you say,
that's not a typo, that's intentional,
doesn't send you back to manuscript next time,
with those errors in it, it's not perfect by any means,
but like, I was never gonna hire a proofer
to read my blog post in the morning
in the 10 minutes after I write it before I publish it.
It's just ridiculous.
So, yeah, I'm very happy to have it.
It's fine.
It is absolutely fine.
It is a normal technology.
My first word processor was a program printed in a magazine
that I bought at the corner store
and took home and typed into my Apple 2 Plus.
I was 10 years old, it was 1981.
I've used a lot of word processor since.
A lot of them have had new features.
A lot of those new features I thought were stupid,
and when other people use them,
I think it made their writing worse.
Some of them were really useful,
and I use them all the time.
I've never seen a feature for a word processor
where I said, that's so cool.
We should bet the world's economy on it.
And by the way, we should probably also put all the writers
in a wood chipper now that it exists, right?
And like, that's the problem, right?
Is we are treating what cash is out to plugins
for common software that are sometimes useful
and sometimes not.
And when workers drive the automation,
can produce things that make your life materially better.
And we're treating them as genies and gods and demons
as the end of the world or as the beginning of a new one.
And it's not that.
If AI destroys the world,
it's going to be because the bubble is burst.
It's not going to be because the software was so cool
that it woke up and turned us into paper clips.
- Maybe this is a math thought
that's completely divorced from reality,
but is it possible we just get bored by it all?
I mean, how many more AI generated videos
of Tom Cruise fighting a large Pokemon on some Chinese roof?
Can I watch on X before I just want to retreat
to a pre-historic cave forever?
I mean, I can't be the only person
that's already tired of these smartest chat bots.
- Come on.
- Come on, run.
So this is maybe a good place to end the conversation on.
I do think that a lot of that AI stuff
is getting washed really quick.
And this has always been one of my critiques
about a certain kind of technological doom saying
that even where you have a thing
that works really well at the start, right?
Someone's found a little cognitive blind spot.
Someone's found a thing that makes people curious
or catches their attention
that unless there's some substance to it,
that it very quickly becomes something that you're in nerd too, right?
You know, do you remember when up where the headlines were invented?
And it was like 11 facts about socks.
You'll never believe number three.
And I click through like half a dozen of those headlines.
And then I realized that like number three
just wasn't that interesting and never would be.
And I never clicked through one of those headlines again.
Now, that's not to say that that form of headline
in the hands of someone who's a good writer
who knows how to infuse it with actual communication,
who's creative, who's smart,
and who has something to say,
'cause you know, the problem with those upper the headlines
is they were always on articles that were stupid, right?
The best thing about them was the headline.
But if you have something smart to say,
there are probably aspects of those techniques
that will work to capture people's attention and hold it
because you're writing about something real and substantive.
And I believe that we will have AI generated text
and images and so on, not in vast quantities,
not fucking emails, routine business emails
that have been inflated three bullet points
into five paragraphs of, you know,
Florida bullshit.
But like we will have AI generated material in our lives
that will be being used in the background, whatever,
you know, to help you get stuff done
and to, you know, make things better.
And some of it will be stupid
and some of it will be fine.
And most of the people who are making it now
will have moved on to something else
because they have nothing to say.
They're just fiddling around with a new web toy.
And in that regard, AI is a normal technology
'cause I've just described the trajectory of many technologies
that we've had in the last couple of decades.
That's how it went with Buzzfeed quizzes, right?
It's how it's gonna go with other web toys too.
- Corey, always a pleasure.
Thank you so much to chat with you.
- Thank you so much for listening.
Please subscribe to our YouTube,
follow and rate on Spotify and Apple podcasts.
You would help me a bunch with that.
Also feel free to support me on Patreon.
This is a one-person podcast.
I do everything by my little self.
It's a lot of work, especially with no team,
no ads and no real marketing.
But you can help by hopping on Patreon
and finding the smart cookies podcast on there
and giving me our money, of course.
Super hard for small creators to get through
so your support would mean a lot.
And then last but certainly not least,
I have to thank my existing patrons
without you.
There is no smart cookies podcast,
which makes you not only absolute legends,
but also in effect actual producers of this show.
So eternal thanks.
Go, go, go to Lenox.
Do me hold this. Charlie Balbeck. You guys are the best.
I'm deeply touched.
So thank you.
- Hey everybody, ladies, luck here.
And we're celebrating America's 250th birthday.
Now all summer long, I'm gonna be celebrating
by playing at spinquest.com,
which is an American-owned social casino.
It obviously features over a thousand slot games
and live blackjack, live crabs, live bubble crabs.
Head on over to spinquest.com.
Get yourself a $30 coin pack for just 10 bucks.
- Spinquist is a free to play social casino.
Boydware prohibited, visit spinquest.com for more details.
Podcast Summary
Key Points:
Cory Doctorow argues that AI is an economic bubble, with over $1.4 trillion invested against less than $50 billion in annual revenue, and predicts a market crash that could vaporize 35% of the US stock market.
He introduces the concept of "reverse centaurs"—machines driving humans—where automation adopted by bosses prioritizes throughput over quality, exploiting workers and leading to inferior products and high injury rates, as seen in Amazon warehouses.
Doctorow warns that the AI bubble's collapse will trigger austerity, erode social systems, and fuel authoritarianism, as people lose jobs, savings, and trust in institutions.
He contrasts "centaur" use of AI (workers using tools to improve quality) with "reverse centaur" use (bosses replacing workers), emphasizing that AI's current deployment is driven by capital, not labor interests.
He critiques AI hype, noting that while AI can assist in tasks like code writing or typo checking, it lacks understanding and is often used to fire workers and cut costs, risking critical expertise and process knowledge.
Doctorow dismisses doomsday AI scenarios from whistleblowers, likening them to past tech panic, and argues the real danger is the bubble bursting, not AI becoming sentient.
Summary:
In this interview, Cory Doctorow discusses his new book, *The Reverse Centaur's Guide to Life After AI*, arguing that the current AI boom is a speculative bubble destined to burst, with severe economic and social consequences. He explains that AI is a "reverse centaur" technology, where machines drive humans, unlike a "centaur" where humans guide machines. This occurs because bosses adopt automation to maximize throughput and profit, often at the expense of quality and worker well-being, as evidenced by high injury rates in automated warehouses.
Doctorow highlights the unsustainable economics of AI, with massive investments yielding minimal revenue, and predicts that when the bubble pops, it will wipe out significant stock market value, leading to austerity and potentially fueling fascism as people lose their livelihoods and savings. He emphasizes that AI's current deployment replaces skilled workers with defective chatbots, eroding hard-won expertise and process knowledge. However, he acknowledges AI can be useful when workers drive its adoption, such as in his own typo-checking or Patrick Ball's code writing, but stresses the need for discernment.
Doctorow also criticizes AI hype, dismissing apocalyptic scenarios from whistleblowers as exaggerated, and argues that the real threat is the economic collapse, not AI sentience. He calls for bursting the bubble sooner to mitigate damage, warning that companies are already unloading overvalued stock onto ordinary investors.
FAQs
A reverse centaur is when a machine drives a human, with the machine making decisions and the human being used up by it. Unlike a centaur, where a human directs a machine, a reverse centaur works the human at their limit, often leading to harm and inferior outcomes.
The AI bubble will burst because AI companies are spending over $1.4 trillion on less than $50 billion in annual revenue, with each generation of AI losing more money. When it pops, it could vaporize 35% of the US stock market, leading to economic collapse and austerity.
Replacing skilled workers with AI erodes hard-won expertise and process knowledge, and when the AI bubble bursts, critical systems will fail. This loss of jobs and savings can drive support for authoritarian movements and fascism.
When workers drive AI adoption, they use it to improve quality, like a centaur. When bosses drive it, they prioritize increasing throughput, often leading to worse products and worker exploitation, as seen in Amazon warehouses with high injury rates.
AI relies on theory-free inference, meaning it finds patterns without understanding underlying causes. This makes it hard to detect false positives, especially in unusual circumstances, limiting its usefulness in fields like scientific innovation.
Doctorow dismisses AI 2027 and similar warnings as 'flashlight under the chin' scares, comparing them to past tech panic narratives. He argues they lack material grounding and are often self-aggrandizing, unlike his own analysis based on economic factors.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.