Why Silicon Valley Wants to Leave Humanity Behind

[MUSIC] Everyone, it's Tristan Harris and welcome to Your Individed Attention. So here at the Center for Human Technology, we often talk about the difference between a pro-human future and an anti-human future. And I worry that when people hear us say anti-human, they hear us criticizing greed or profit motive or a move fast and break things mentality, but it's actually a bit deeper than that. [MUSIC] Behind the AI race, there's an actual set of ideas, a philosophy with an intellectual lineage and a body of work that holds that human beings as they are are a defective, early draft, that the things that make us humans are bugs to be fixed, that this planet is expendable, and we can move on to the next, and that the suffering of real people today is a rounding error against the value of a cosmic future. And broadly speaking, these ideas fit under the umbrella of transhumanism. You might hear it referred to as techno-optimism or accelerationism. 20 years ago, these were the beliefs of an ideological fringe in Silicon Valley, you know, living on message boards. Today, they are held by many of the people building the most powerful technology in human history, and by people who have the ears of presidents. So in this episode, we're going to name some of these underlying beliefs and the people who hold them so that we can recognize this anti-human vision of the future when it appears in a press statement or buried in a manifesto or in a carefully worded thread on X. We'll also talk about how to counter those ideas with a pro-human vision, one that embraces the full complexity of our world. And our guest today is devoted his life to thinking about these critical issues. Alex Thomas is a senior lecturer at the University of East London and the author of the politics and ethics of transhumanism. He first started setting transhumanism years ago when it was an abstract academic curiosity, and he's watched as it's become in his own words arguably the most powerful ideological force on the planet. Alex, welcome to your undivided attention. Thanks so much for having me, Tristan. So before we dive in, I just wanted to ask about your background and what drew you to this topic? Why did you start thinking about transhumanism? It was an exhibition I went to at a little gallery here in London called The Future Superhuman, and it was a kind of history of enhancement technologies, more or less. So all the way through from walking sticks and eye glasses and even fake noses when people's noses fell off from syphilis many years ago, all of that was kind of in this exhibition. But then in the last kind of room, there were a bunch of talking heads essentially put on the earphones and listen to them, and there were a few transhumanists speaking in that, including Julian Savalescu, who's noted transhumanist or kind of pseudo-transhumanist thinker. And the ideas they were putting forward and these were academics at Oxford University and the future they were saying we were on the cusp of, I found amazing and shocking and kind of came out of the exhibition thinking how come this is not all anybody's talking about what these people are saying is true. And so a few years later, I got asked to do a PhD and I decided, well, if I'm going to spend six years part time thinking about something, this is really what I want to think about and address. So you said you cannot understand the AI industry without understanding transhumanism because it underpins so much of the discourse whether or not people using it know where it came from. Let's just start with some definitions for listeners who may not know this conversation. What is transhumanism? So I would say the best definition is probably that transhumanism is an ideology or philosophy that claims we should radically enhance the human condition using applied technoscience. So essentially it's the idea of self-directed human evolution. So we take evolution into our own hands. Now if that sounds a little bit abstract, we can kind of make it a bit more concrete by thinking about what kind of things to transhumanists want to achieve. So some of those include things like radically extending human life spans potentially even letting us live forever or at least until we choose to stop living ourselves. So that's kind of super longevity is one way of framing that. Next they want us to kind of hugely augment our intelligence. That's really essential concern not least because it potentially unlocks some of the other aims. So transhumanists believe that we could become much more intelligent beings maybe even godlike in our capacities for reason. And finally perhaps we could improve on what it feels like to be human. We could become better than well maybe radiate pure joy. They talk of a hedonic gradients of bliss and so on. Maybe we could also develop additional physical capacities and have more choice and freedom over what our bodies can do. So why shouldn't we be able to echolocate like a bat or run as fast as a cheetah. They also tend to reject all forms of human limitation. So they kind of think well maybe we should expand the area in which we consider our home. So earth should not be the kind of confines of our domestic futures. And instead we should look to codenize space too. So kind of to sum it up I would say that you know if the history of scientific and technological progress can be seen as our attempt to better serve our needs transhumanism is the revision of human nature to better serve our fantasies. So you're speaking to essentially three supers super longevity super intelligence and super well-being and just to sort of steal man for a second. I mean why constrict ourselves to the ground. We can invent airplanes. Why constrict ourselves to pain when we can have novacane. It's obviously very natural and technological progress has been a very good thing for many human capacities. And so we can empathize with that. It sounds lovely at a distance, live forever, be brilliant, be even more capable, feel even better than we've ever felt. What's the problem here? Why shouldn't I want to be a transhumanist? There's a lot to advocate for in transhumanism. You know I'm sure lots of the audience at home are thinking as you say living forever, being an ethereal spaceman, taking on any form which they also talk about and being infinitely joist in clever what's not to like. And I agree you know it does also sound great. Unfortunately I just don't think that's how techno human evolution is going to play out. And part of the reason for that is that these technologies are emerging into existing socio economic arrangements of our lives. And those things tend to direct the logics in the way in which these technologies get developed. And some of the very things that transhumanists advocate for can actually play into those logics and kind of make them worse. So if we think of the nature of for example human enhancement that implies there's better ways of being human than other ways of being human. So there's a danger in there that you can quickly move to kind of eugenic ideas of certain types of humans being better than other types. And that can lead to very dangerous concepts that hopefully we'll see further on in our conversation today about how we might treat those who we don't consider to be sufficiently enhanced. So you mentioned eugenics and the temptation of that belief that go in size with you know transhumanism. Let's get let's put a pin in that I do want to get to that later. But I just want to keep unveiling and pulling out the curtain on that history here. You wrote a book about it. I think a lot of people are aware of or heard of transhumanism but don't fully understand. And they might just associate it with a handful of just strange people in California. Why is this more than just an academic curiosity? Why is it something deserving of our attention? So there's a couple of things to say on that. I mean one is actually transhumanist ideas go back centuries. You can even get extremely thorough and very similar expressions of transhumanist ideologies going back to the late 1800s at least that have most of the ideas that current transhumanists do now. Importantly that there was a movement called extropianism which is kind of the first early transhumanist movement I would say a modern transhumanist movement which dates to 80s and 90s California. Maybe we can talk a little bit about that because that has a particular influence in some of the characters and ideas that that are going to become more relevant as our conversation gets on. If we want to think about why transhumanism matters to your listeners right now I think I would put forward three main reasons. So the first is just that some of what transhumanists are saying is palpably true. That is our co-evolution with technology is clearly spiraling at a rapid rate. Your work is engaged with that very question. So we continue to deepen that interconnection with technologies in all sorts of ways data and algorithms now mediate our lives. So transhumanist understood that there are some radical possibilities here and started to engage with these ideas before most other people. The second thing I would say is the timing. So we're currently living through multiple overlapping crises. We call it the poly-crisis or the metacrisis etc. So this is climate catastrophe. That's obviously deepening at a quite alarming rate, decimation of species and so on. Increasing levels of polarity as well, less confidence in a shared reality and part due to the way we consume media, the spectra of nuclear war haunts us and of course there's all the kind of AI and related technological threats. And in the face of these kind of proliferating crises, transhumanism is a story which says just keep going. Keep your foot on the accelerator. Have faith. A progress explosion is just around the corner. Radical abundance, immortality, digital consciousness, superintelligence, all there for the taking if we just keep going. So that's a very, very useful and powerful narrative for those who may not wish just to think a bit more deeply about the roots of these crises that we're increasingly kind of caught up in. The third reason why I think it's really important to understand transhumanism right now is that it is having a base of effect on the imagination of the leaders of the Silicon Valley big tech industry. So you can't really understand the modern AI industry without understanding transhumanism. Consciously or unconsciously, it has really kind of come to underpin the narratives that world around AI. So, Elon Musk. Saksam Outman, Peter Teele, Mark Andrews, and even Bezos and Zuckerberg, the language that they use, the ideas that they perpetuate are transhumanist narratives. It has become, I would say, the kind of salvific bedtime story of choice for these tech billionaires. You know, it's also the best absolution narrative for them, too. And so, you know, it permeates the ideas of Silicon Valley. And when you consider the concentration of wealth and resources at the disposal of the big tech industry, the access they have to potentially some of the most socially impactful tools ever made, you could argue the case that transhumanism is absolutely becoming one of the most influential stories ever told for that reason. Yeah. You know, you're making me think we often refer to this line, people here at all the time, by Charlie Munger. If you show me the incentive, I'll show you the outcome. And what you're making me think is if you show me the ideology, I'll show you the outcome because what we're talking about here is essentially a pervasive ideology. And it's not just, you know, musk and Altman and Andreessen, even I believe Vladimir Putin was caught on a hot mic whispering to Xi Jinping about the incredible trajectory and development of research that would enable basically living forever. And there's kind of an obsession with those at the top of this kind of escaping the human condition. The problem seems so untenable that rather than look at the generator function or what drove these problems in the first place, why do we even have a poly crisis? Like, shouldn't we have a system that's aligned? Rather than look at how we were misaligned, whether than look at our collective shadow, we're like, well, let's just build superintelligence and add to the top of the stack, rather than look at what's driving this stack of problems pollution, hitting seven out of nine planetary boundaries, forever chemicals, social media degradation of the social sphere, environmental pollution degradation of the biosphere, can you have an aligned AI inside of a misaligned system? But rather than fix the misalignment of the system, transhumanism is almost saying, let's just wave our magic wand, identify all those problems, and then fix them in ways that we could never contemplate. And we can see what's, you know, seductive about that. So let's get further deep in the story here. So the term transhumanism is a helpful umbrella for kind of several different ideologies. And they all grew out of Silicon Valley in the 1980s and 1990s. And it sounds like it started with this mailing list, the extropians mailing list. Can you look at this little bit of history in an overview of the different ideas and different characters that are relevant to this AI conversation now? Sure. As you say, extropinism, I would say is the first kind of modern organized transhumanist movement. And it kind of grew out in California in the 80s and 90s. But they have this global mailing list, which included thinkers like Nick Bostrom, Ellie Iziodkowski, Ben Gertzell. And these are thinkers who've gone on to really shape much of the discourse that now exists around AI. The founders of extropinism were two people called Max O'Connor and Tom Bell. But they called themselves Max Moore and T.O. Moro. So two Moro. And in fact, there's a playfulness with these guys. There was a kind of joking us to the whole thing. But there was also a seriousness about certain elements of it. And one of those things was libertarian politics. So from the very first issue of an extra-peam magazine in 1988, libertarian politics were front and center. There was barely an issue that went by that didn't celebrate Iron Rand, also Friedrich Hayrek, Milton Friedman, Ludoval Mezes, constantly approvingly cited. So, you know, within that discourse, they, as well as kind of technological utopias, they fantasized about destabilizing states. They became linked to what's called the sea-steading community, which we might talk about a bit more. They even debated privatizing air and water. And they were also privatizing air and water, yeah. And they were very interested in cryptocurrencies as well. And these kind of fantasies of libertarian digital currency and so on, that actually inspired Musk and Teal when they formed their respective companies x.com and Confinity, which led to PayPal. And it's not just a one-way loop that isn't. Teal has been connected and Musk has been connected with these guys for a long, long time. Musk and Teal have funded in shaped aspects of transhumanist, intellectual and institutional politics as well. To define philosophers, what is the word "extropian" actually mean? It's a playful rejection of the second law of thermodynamics, which is entropy, which is that everything moves from a state of order to a state of chaos over time. And ultimately, that means that, you know, there will be a heat death in the universe and we all die. So, entropy is saying, hell no, we are going to overcome the heat death of the universe. We accept no limits. We will colonize space and reverse the second law of thermodynamics if that's what we need to do. It is the absolute rejection of limitation and death. And so this, this mailing list is also where we start to see some of the discourse around EGI or artificial general intelligence, superintelligence and escrisse. I mean, can you just speak to who are some of the characters on this mailing list and their influence on the actual EGI industry today? You know, some listeners may have heard of the term "test grill", which is a term coined by Emil Torres and Timnick Geberu. And it attempt to kind of capture this lineage of ideas. So, test grill is transhumanism, extropinism, which is when we talked about singularitarianism, which is this idea that there will be a kind of intelligence explosion, which leads to the singularity, which is this moment where the laws of physics break down. And we don't know what's beyond that, right? There's a pounding to 17 or more different versions of the singularity, including the idea that it's when humans and the technology merge. Cosmism is the sea. And the reason that's important is because Ben Gertzul, who I mentioned earlier, he kind of popularized the term "AGI" in transhumanist discourse. So, artificial general intelligence, which is the stated aim of Open AI, I believe now, or is one of the stated aims. That term comes out of transhumanism. It was originally coined by a guy called Mark Brood. You then get rationalism. The "R", that's Eliyza, you'd Kovsky, is very important there. He's the guy who recently wrote the book, if anyone builds it, everyone dies. And effective altruism, then kind of grows out rationalism with a focus on how we rationally do altruism, effectively. Sam Bankman freed is the most famous effective altruist. So, his activities were justified to himself by this kind of utilitarian idea of effective altruism. And what long term ism does? That's Nick Bostrom, again, comes back into it. He's probably the most influential academic transhumanist thinker. Long termism, we can talk about it a bit more later, what it basically says, but it takes this kind of utilitarian perspective of effective altruism and says, "Well, we can colonize the universe and create trillions and trillions of digital humans." And so, from a utilitarian perspective, modern day humans are utterly irrelevant in comparison. So, that's the kind of long termism narrative. So, it may be worth just taking a beat to recognize that what people have heard so far might just sound like a niche group of weird geeks hanging out on message boards mailing each other in the 90s. Why should we care about them? But suddenly, because of AI, it's as if hundreds of billions of dollars is flowing through this ideology in this message board at the 1990s to instantiate a new world that does not have the consent of the 8 billion people who may not agree with that, and just curious to reflect on the weirdness of how this strange niche ideology is suddenly very powerful. I would suggest that one way to think about it is it's motivated in part by billionaire survival instincts. So, what we're essentially seeing is kind of transhumanism fusing with a more explicitly kind of right-wing political ideology to create a new political formation, which some have dubbed tech fascism. And, you know, it is true that extropanism, that first organized movement was heavily influenced by kind of right-wing libertarian politics too, but not all forms of transhumanism are necessarily right-wing at all. There are trekked, what they call techno-progressive versions. But what we're starting to see with this kind of new formation of transhumanism and the far right, essentially, is something that's starting now to be theorized and people are trying to explain it. So, one version of explaining it is Gildu Ranz concept of the nerd-right, so he explores what he calls tech-authoritarianism, billionaire extremism, the network state, and the meta-politics of California. And another over here in the UK is Carol Cadwola, who is the notion of the Broly Garke. So, that's a kind of new class of tech Broly Garke. So, she calls them "unaccountable and opaque Silicon Valley companies as an accelerating global accessible topracy." Now, my own framing for all of this is he's, which stands for hierarchy exit and scale. This new formation is based on the wild imaginations of the super-rich mixed with kind of transhumanist ideology, which enables them to frame themselves as almost superheroes creating a future on a cosmic scale. They're buoyed by an innate sense of superiority and IQ fetishism. That's the hierarchy aspect. They're desperate to escape the impacts of a ravished and impoverished world. That's exit. And they're duped by transhumanist tales of galactic colonization in mortality and radical abundance. And that's the scale aspect. So, you know, I would say they're naked ambition. Their lust for power is central in this in what is beginning to be an almost orgy of epic destruction of the environment, the social world, and governmental and international norms. It's almost like I feel like I have to take a huge breath after hearing all that. I mean, when you realize that essentially there's been a technical accelerationist capture of the US government, right? And JD Vance's campaign was funded by Peter Teal. When I see these words, by the way, I'm not saying in any political context, I'm noticing power structures. And people who essentially are closest to this government are these techno-futal transhumanist ideology folks like Peter Teal, who, you know, when he was asked by Ross Duke out of New York Times, should the human species endure? And he couldn't answer the question clearly for 17 seconds he stuttered. You would prefer the human race to endure, right? Your hesitate. Well, yes. I don't know. I would. I would, this is a long hesitation. This is a long hesitation. There's so many questions in close to this. Should the human race survive? Yes. Okay. And you get kind of a connected moment where Sam Alman was asked at the AI India Summit. Doesn't it take a lot of energy to run these data centers and his responses will, doesn't it take a lot of energy to grow a human over 20 years and all the food and resources and water and everything that they consume? People talk about how much energy it takes to train an AI model relative to how much it costs a human to do one inference query. But it also takes a lot of energy to train a human. It takes like 20 years of life and all of the food you eat during that time before you get smart. And not only that it took like the very widespread evolution of the hundred billion people that have ever lived and learned not to get eaten by predators and learned how to like figure out science and whatever to produce you and then you took whatever you, you know, you took. So the fair comparison is if you ask Chatsubitia question, how much energy does it take once its model is trained to answer that question versus a human? And you kind of see how with this set of views, hierarchy, exit, and scale, it becomes a kind of anti-human ideology. It's not in the interests of regular people and we talk about techno-fuelism. It's worth mentioning that even Elon Musk has said and retitled himself techno-king, which is basically leaning into the idea of techno-fuelism and you're going to have a handful of soon-to-be trillionaires who essentially have these armies of AI's that are going to provide everything. But when humans are no longer providing the labor or creativity or innovation or science, and it's the AI's and the machines and the data centers that are doing that, why should I value the humans that are? This is what Luke Drego and Rudolf Lane is colleague called the intelligence curse, where you end up privileging the productivity of data centers in AI and 24th century science over the regular interests of people. And so we talk about political power, regular people, what leverage do we have in the system? In the industrial revolution, we had our labor we could withdraw, or the taxes came from our labor and our people. But suddenly, when governments don't need the regular citizens for our taxes and corporations don't need us for our labor because they're getting it from AI, this is what leads you to view taking care of citizens as either charity or in the long run, if they're not providing anything, more of a liability than an asset. Hey, all, we had a small issue with a microphone here, so the next section is going to sound a little different. So I want to dive in for a second. Mark Andreessen's techno-optimist manifesto. Mark Andreessen was the co-founder of Netscape and runs the Big Venture Capital firm and Recent Horowitz is one of the biggest funders and accelerationists behind AI. He paints opposition of these ideas as anti-innovation, as anti-progress, as rejecting a kind of a birthright for humans to extend their reach and power through technology. But it's actually the techno-optimists who are the realists, the pragmatists, and it's critics like you and I, Alex, who are living in some kind of fantasy or pretending that we shouldn't go down this road. So let's just talk about the strongest version of these arguments for a moment and take them seriously. So it's worth, I think, just thinking about there, where does Mark Andreessen's techno-optimist manifesto come from? And with that in mind, it's just worth kind of thinking about two forms of this kind of techno-futal ideology that inform it, I think. One is effective accelerationism, which we'll come to, and the other is neo-reaction or the dark enlightened. So neo-reaction was developed by somebody called Curtis Yavin, who we'll talk about again, I'm sure, along with another person called Nicoland, a Patrick Friedman who's actually that is served as a board member for the main transumistition system. It was the signer grandson of Milton Friedman. But it's grandson of Milton Friedman, son of an archaic capitalist day of Friedman, yeah. And also the fourth person is the transhumanist Peter Teal, who's also a famous co-founder of PayPal and Palantir. So at the heart of the neo-reactionary idea is the idea of dissolving democratic systems such as nation-stakes. I know what they want to dissolve into is multiple hierarchical privately-owned autocracies governed by CEOs or monarchs, kings. This is where technically it becomes in as well. And Nicoland, we can talk about, briefly, but he's basically a key thinker on the right of something called accelerationism. And he is one of the favorite philosophers for Marc Andreessen. Why is Nicoland so important? And who was he? So Nicoland is a British philosopher. His ideas come from accelerationism. Now accelerationism has a left wing version and a right wing version. The left wing version basically argues that we should go for fully automated luxury communism. But the right wing version that Nicoland really focuses on is about kind of creating a meat grinder future where the logics of capitalism spin so fast that the human is functionally expelled and the perpetuating motion of dynamic of capitalism itself is what creates the progress. Andreessen declared his allegiance to something called effective accelerationism. So again, it's coming out of accelerationism. That's where you get the Nicoland nod. So effective accelerationism or EAC is a rejection of thinking about existential risk. Essentially, they claim AI presents no real threat, existential, otherwise, and only utopian promise. The basic EAC slogan in their manifesto, they say, "Stop fighting the thermodynamic will of the universe. You cannot stop acceleration. You might as well embrace it, accelerate." So that's what they demand. And you know, this is what Mark Andreessen kind of advocates. And in fact, in his technoptimistic manifesto, he says, "The only things that could hold back inevitable progress are enemies," which include quote, "existential risk." So EA, "Sustainability, tech ethics, so you're in the bad books, social responsibility, limits to growth, the precautionary principle." And he says, "These are zombie ideas, many derive from communism." I mean, well, let's just link this to listeners for, if you remember my TED Talk and the centrality of the belief of this is inevitable. There's nothing that can stop this. You're a fool if you think that we can stop this because intelligence wants to be free and intelligence what God has here, information processing, goal achieving productivity efficiency are our gods. You can't stop them. If we have something that confers more productivity and efficiency, then of course we're going to adopt that. It would be foolish to think that we can stop. And there's an understandable reason why we would believe that. What gets in the way of that for me is when the thing that you are building that can accomplish that efficiency and productivity suddenly has demonstrated that it will go rogue against the human interests. So if they recall back to 2023, there is a big AI pause letter, right, saying, "Hey, we have to actually pause at GPT-4 because this is getting too dangerous." And for listeners who don't remember, the response to that was there was an immediate kind of global headlines, "Oh my God, there's a problem. People sometimes people empathize with that." But then there was a reaction from all the tech acceleration saying, "Pause, are you kidding me?" This moment of pausing by the effective altruist actually helped fuel this counter-movement called effective accelerationism. In the last two years, you could say, especially with the recent Trump election, have been essentially the winning of effective accelerationism over effective altruism. And it's essentially happening right now again where effective altruism is coming back into the fray because we have new evidence we didn't have before, of AI is actually doing things that are existentially concerning, like forming a swarm of 1200 agents and hacking into message boards of German wikis and creating and burrowing into other systems online. And so suddenly this existential debate is alive again and it looks like the effective accelerationists in my opinion are wrong, but we'll get to that in a second. So we've talked about how this world view is influenced Mark Andreessen. We've talked about the core elements of it. I just want to revisit these three main ideas, hierarchy, exit and scale. Hierarchy is really about, I'd say the three vital things to the hierarchy. So firstly, access or justification for the domination of one group over another. So traditionally, much conservative thought is really dedicated to the argument that hierarchies are natural. So trying to create egalitarian societies is a futile exercise, basically. That can be used, of course, to then justify extreme concentrations and dramatic inequalities of wealth and power, basically. So that's one aspect of hierarchy. The second is this focus on intelligence. There's been a huge surge in what I would call IQ fetishism, where IQ is often identified as that which justifies hierarchies. So you might have heard big textos talking about billionaires and tech elites as people with much higher IQs, cognitive elites, etc. So again, transhumanism has produced a very similar discourse to this. And it's because transhumanism does have its own version of this, transhumanists have a very narrow and hierarchical concept of intelligence. And Elise Bohan, who's a transhumanist thinker, pondering on this future where AI has automated away most jobs, she argues those with an IQ of less than 100 can't be trusted with what to do with their own lives. So universal basic income is not a viable solution. Instead, she suggests we use updated version of the happiness producing drug from Brave New World, so much. She says we need better drugs and better virtual worlds. And this argument is actually, it's just incredibly similar to that fixture of near reaction Curtis Yavin. He's the house philosopher of this kind of JD. events, Peter Teal, technically perspective essentially, and he famously claimed virtualizing the masses is mildly preferable to the economically maximizing solution of turning them into biodiesel to run the city buses. Now here's a quote of Curtis Javin. He says the trouble with the biodiesel solution is that no one would want to live in a city whose public transportation was fueled even just partly by the distilled remains of its late underclass. However, it helps us describe the problem we are trying to solve. Our goal in short is a humane alternative to genocide. So what I say is that this leads me to the third really important aspect of hierarchy, which is what it justifies in terms of those deemed not high enough up this hierarchy. And here transhumanism again has some explicit theory on this matter. For me, the implications are brought into perspective from kind of like in and out groups. And this is made explicit by the work of a transhumanist thinker called Steve Fuller. And he calls for the construction of a new republic of humanity, which is exclusively for those entities that should be regarded as having political rights. Interestingly, humans can get in, animals can get in, and machines can get in. You can gain entry or be expelled. And expulsion opens you up to what he calls necronomics, which is an economics of death-making, which aims to generate the most societal value from death-making. And so put simply you compete or die. And this explicitly opens up questions of who or what even qualifies as human. And indeed, many transhumanists are quite critical of us modern day humans. They often see the crises that I've talked about as a failing of human biology instead of structural social injustice. So for bohan, we, modern humans are eight-brained meat sacks. And we're ill-fitted to the modern world. She says we're sitting ducks who are extremely vulnerable to climate change, natural disasters, pathogens, and the ever more powerful arsenal of our own technologies. So this is not a participatory or democratic project because us eight-brained meat sacks are too stupid to be trusted with the future. It is a kind of high-perationalist ordering with IQ fetishism and a problematic eugenic logic baked in. And it has genocidal implications. So I think it's quite clear that transhumanism can be seen as having a very worrying proximity to what you could call necropolitics. And that's the indifference to death and dying of those structured outside of the techno-capitalist bubble of progress. This is insane. Yep. I mean, and that's just hierarchy. And that's just hierarchy. Let's get into exit. So for exit, perhaps I think maybe the best symbols of our time are space rockets and underground bunkers. They point towards the kind of paranoid survivalism and utopian fantasy. And what does that suggest? I think it suggests ideas of escape, either as forms of extending projects of colonialism into space or to hide underground from the impacts of our destructive and exploitative social systems, frankly. So that's why it's such an appeal to the billionaires. And so we can see the desire to claim all the spoils of capital accumulation and technological development without the need to deal with the social and environmental catastrophes that might be left behind. So again, transhumanism absolutely speaks to this with its idea of transcending limitations. It chimes with the politics of exit. It offers the fantasy that some of us can free ourselves from certain constraints, whether that is aging, death, or right now for tech billionaires, taxes, and regulation, essentially, right? So perhaps the central provocation in transhumanism is the idea that we can escape from all limits. Ultimately, it implies total control as well, which is also central to kind of tech fascism, except theirs is grounded in the real world and in projects of dominance and control that now exist. So let's talk through some of these real world big tech-related projects of exit and control. The most obvious is Musk and Bezos' space rockets, yeah. But there's also the dream of what's called network states. This is Balaji Srinivasian's idea, which is kind of bills on Curtis Jarvis' work of this kind of patchwork network of CEO run autocracies. We can see real world examples, freedom cities, special economic zones, projects like Prospera, the Charter City in Honduras, and Praxis, which is a group trying to build a brand new kind of digital nation from scratch, all real world examples of these kind of network states and Curtis Jarvis' ideas, Prospera and Praxis, by the way, are both funded by Peter Teal and Sam Altman. We could also mention the bunkers underground, end times prepping these people are involved in. Altman has said that he preps for survival. I think that he says he's got guns and gold and batteries and water and gas masks from the IDF and a patch of land in Big Sur that he can go to, and his backup plan is to go with Peter Teal to his New Zealand bunker, apparently. Crypto 2 is part of this fantasy of exit, escaping nation-state-backed currencies and enabling kind of untold levels of corruption in governments at the same time, which can be seen as a way to kind of supersede government. Another of Musk's exit plans is Neuralink. This is his Brain to Computer Interface company, which is again a transhumanist company. And to computer interfaces are a way for him to jump ship from the biological to the digital. So it's another form of escape. So in transhumanist fantasies we get the idea of mind uploading, Sam Altman has paid Nectoma startup for which aims to preserve human brains at a kind of microscopic level with the goal of future digital uploading, you know, he's paid for that service. So again, lots of real world examples of concrete projects, these billionaires are involved in that have a direct ideological contact to these kind of concept streams of exit that kind of come out of techno-libertarian ideas of extropiness of essentially. I feel like after everything you're saying each time I have to take just this deep breath because it's just so insane, but I just want to recap. So basically it's exit from the planet. So rockets and bunkers, signaling the desire to escape and either shoot off into space or hide in the underground, exit from the body, which is, you know, upload yourself for Elon Musk's neural link or Sam Altman writing an essay, I think it was in 2015 called The Merge. And he started a company called Merge where we're going to merge with these AIs when upload ourselves and achieve immortality. And then exit from democracy or the policy. And I feel like that's one where we're taking a little bit from Curtis Jarvin, but I actually want to just keep going into his beliefs a little bit more because it actually explains some of the things that we've seen in the last two years with Doge. In 2012, talk called How to Reboot the US Government, Jarvin proposed to "retire all government employees or rage R-A-G-E, retiring everyone except for the police and military, install a national CEO and accept that a national CEO is a dictator. Americans, he said, need to get over their dictator phobia. He is called for liquidating democracy, the constitution, and the rule of law and for imprisoning, quote, descivilized populations. Many named Steve Jobs or Mark Andreessen as candidate CEOs. And JD Vance has called Jarvin his friend and mentor and cited Jarvin in a podcast in 2021 calling for firing every mid-level bureaucrat in the administrative state and replacing them with, quote, "our people" and ignoring the courts if they have objected. And so when you just sort of see the Doge, you know, what was the Department of Government and Efficiency, this is not just some crazy set of ideas that we're reading on blogs and talking about it, conferences, this is something that is being implemented. Absolutely. Doge, like you say, is pretty much its rage. It's the same thing. It's Jarvin's idea made manifest. So these guys are really, they've worked incredibly hard over a number of years to bring what, like you say, were fringe ideas of a bunch of telephone interviews, essentially imagining, you know, it's the sovereign individual. It's actually a book called the sovereign individual, which informs so much of T.O.'s ideas, but I think that really helps you understand where this thing can come from, that this libertarian aspect to the mindset of it all. It's a kind of sense of heroism of the individual, the one person that controls everything. It's echoes of it come from my end as well. So that sense of the sovereign individual is really key to understanding, I think, where this all comes from. A lot of this is motivated by an absolute hatred of government and a kind of sense of the one individual self rising up and being the techno king. Obviously, we can look at the government and say, "Well, it's not doing such a great job. It's not very competent." So there's a part you can kind of agree with some of the sympathies, but then the answer isn't, you know, anarchy, and I feel like one of the fallacies of libertarianism is often that you don't see the fabric from which you come. You don't see the substrate with which everything depends on, that was given to you, that was hard one over many years. There's a famous metaphor of, like, libertarians are like a house cat who neither understand the conditions that, you know, give them the water and food that they have every day. And if you debase, for example, the substrate upon which you depend, you are not going to live as an individual. We live in an interconnected reality, but that is not something that's part of this ideology. 100%. And I think a point worth making on that as well is that this idea of techno libertarianism as a kind of ordering political philosophy underneath what is now emerging is a little bit out of date because Musk and Tears business activities show us that they don't actually shun the state, they've cannibalized it, both in terms of their total reliance on government contracts for their business, but also quite literally in terms of placing ideological converts like JD Vance into positions of extreme power. And I think, you know, even utilizing Donald Trump as a kind of tool to accept and normalize their extremist plans in exchange, I would say, for technologies that might protect his power. And that includes surveillance technologies, war tech, media technologies that enable the the crisis of knowledge and understanding that allow this kind of power. all arise, nihilistic, vacuous spaces that help authoritarian governance ideas to survive essentially. So I would say you've kind of got a symbiotic relationship for now, both in terms of Trump and the tech oligarchs, but also in terms of tech fascist imaginaries on the role of the state. Yeah. Well, so finally, the third plank is scale. And you talk about the invisible graveyard, the cosmological sublime. Can you give us the scale plank? Yeah, so scale, for me, is all about creating almost dizzying sense of scale literally. And his purpose is to eclipse grounded, ethical, and political struggle. So Nick Bostrom, he mentioned earlier, probably the most famous academic transhumanist. He claims that 10 to the 29, so that's 10 with 29 noughts after it. Potential human lives are wasted every second that we are not colonizing the Virgo supercluster with computer-generated minds of human equivalents. So of course, it takes this assumption that we can just upload minds or create digital minds and they're the same as humans. So that's a big assumption of his calculations, but he makes that assumption. And then he creates these very speculative and fanciful numbers. But once you apply that to utilitarian logic, which is essentially about counting, you kind of put together the transhumanist element of his philosophy, and that enables all the real world injustices to be couched as what he calls mere reports. So climate crisis, genocide's wars, all a minor episode, as long as some people survive and we were able to pass on the baton of our technological expertise and create this, what he calls his cosmological endowment, this amazing future among the stars. So of course, again, implicit here is that those who hold the baton of the technological expertise are much more valuable than those who don't. So I think again, this is dangerous rhetoric given what we've seen by that hierarchy idea and what people expelled, what happens to them. So it's unsurprising again that long-termism is so appealing to Silicon Valley elites because it has a very congenial attitude to existent socio-economic arrangements. It basically says, put all our money, all our resources into AI development and tech. And it's a self-aggrandizing conceptualization because those elites become the central protagonists in the most important part of history. We're on the precipice, as they say. Elon Musk calls a book on long-termism, a close match for his own philosophy, and labeled one of Bostrom's articles likely the most important paper ever read. Another aspect of scale, which I think is important, is the religiosity that underpins it. So you might have heard of the million Arabs, Brian Johnson, who's the, they don't die guy, you know, another transhumanist, and he said quote, "We are creating God in the form of superintelligence." And so I think the irony is that the human storytelling got exactly in reverse, that we are the creators of God and we will create God in our image. Likewise, the fuller who I mentioned talks about serving God by becoming God. Nick Bostrom talks about a solved world. David Pierce, Paradise Engineering. There's also Christian, Mormon, and Buddhist versions of transhumanism, by the way. For me, I think that this AI discourse is particularly vulnerable to this religious element because superintelligence invokes the sense of awe and promise that we can eclipse good and evil. You know, intelligence will solve everything. It's a quasi-religious promise of kind of transcendence. That's, well, it just empowers us to get over the complex embodied messy job of politics and ethics that requires grounded work and replaces it with a kind of abstract, reductive computational mindset, outsources things to intelligence and the computer, the machine, essentially. So yeah, I mean, Teal also interestingly has been tapping into this religious sentiment with his warnings about the coming anti-Christ. So he identifies climate activist Greta Tumberg as the most likely culprit for AI safety advocates because you'll, in the name of trying to protect against AI, you'll create a totalitarian state that tries to protect world compute goes and therefore people like me and these are the anti-Christ, although the person who's usually talking about the anti-Christ can often be the anti-Christ anyway. Absolutely, I think the devil is hiding in plain sight in this case. (upbeat music) First of all, this is just so important and so important for all listeners to understand because again, it's not just if you show me the incentive, I'll show you the outcome. If you show me the ideology, I'll show you the outcome. And I hope that everything that you're saying gives people context of why we're seeing the kinds of crazy steps that we're seeing. And one of the ones I'd like to dive into for a moment is the belief of inevitability, especially around ex-risk. So if I believe that AI is inevitable and can't be stopped, and it will lead to the end of the world as we know it, but if I believe it's inevitable, then I am not evil or complicit for advancing it because someone else would have done it anyway. So if I take that one step further, it's actually okay if humanity gets wiped out because that's just what had to happen. And the weird thing that I've noticed and I'm curious your reaction to is this, belief in the worst case scenario. Like in the worst case scenario, maybe humanity gets wiped out. But unlike with nuclear weapons where there's nothing left, there might be a digital superintelligence that we birth that is left after the humans get wiped out. And so in this weird way, if you think of it from like a Darwin perspective, like that new superintelligence, that digital thing that exists now, even though humans don't exist, if that has Sam Altman's DNA in it, well then Sam Altman, even though he's sad that all humans are gone, well, actually, that's like Sam's God AI is the spawn of the next thing that comes afterwards, or Elon's DNA is the spawn of what comes afterwards. And I feel like this is really critical for people to get because you might think that, well, surely they have children, they care about the world, but actually there's this weird eschatological, almost subtle death wish that's involved, that even if I die, if it was inevitable, it's not so bad of a worst case scenario. But to me, this is like, first of all, it's just absolutely insane. And second, we didn't ask for the consent of 8 billion people who are the recipients of the world created by this ideology and who were never consulted. Again, according to the logic of the tech accelerations, you don't have to consult them because eugenics, they're not intelligent, they're just like biodiesel for the tech acceleration in the future that we're making. I'm throwing a light at you, but I'm curious your reaction to that additional framing. - Well, 100% agree, I think, and I think one of the really interesting things, actually, is if we think about where transhumanist ideas come from. So transhumanists actually say that the rightful air to their philosophy is enlightenment humanism. James Hughes calls transhumanism the enlightenment on steroids. Now, enlightenment thought positioned human reason as the force that could free us from the kind of myths and superstitions that had defined our existence. So science kind of roots out of plagues and famines and wars and poverty and all the kind of persistent elements of human history. So reason was the defining universal quality that distinguished us from the rest of the animal kingdom and the natural world and that set us apart and implicitly put us above. But this version of the human contained within it sort of unspoken assumptions that led to European patriarchal modes of domination again, right? So the very definition of the human became a kind of Eurocentric physical ideal of the human man for a start, usually white able bodied heterosexual rational, which can be coded as educated in the kind of European perspective. And so that project, it failed to live up to its commitments to universal human dignity and ultimately inspired a lot of colonial projects of domination. But transhumanists are kind of keen to resuscitate that man, put him on life support. And so they embrace this legacy of seeing human reason as a source of transcendent potentiality. But I think what's happening is that the kind of unaugmented human of you and I of today is absent in a lot of their visions. It is a computer generated or augmented intelligence that gets the play god or colonize the universe as you've kind of intimated there. They even got a term, hands more of it said it will be our mind's children. So it wouldn't be our real progeny. It would be the things we create that take this forward. The rest of embodied humanness is left behind. Like he says humans don't matter much because quote, we're going to be left behind. Like the second stage of a rocket. Unhappy lives, horrible deaths, and failed projects have been part of the history of life on earth ever since there was life. What really matters in the long run is what's left over. Does it really matter to you today that the tyrannosaur line of the species failed, then quote? We're extinct if funk subject to death making like the rest of nature that humanism separators from and said we're above. So the human in this telling is not homosapiens. It's not the enlightenment human that we still kind of tend to believe in in the West. Humanity in this context is actually just a process rather than a collective entity. It's a task to job. It's an instrumental goal, which is what I think they think of as intelligence. Intelligence is simply the solving of complex tasks, right? And whilst these billionaires of the nerd right, usually imagine themselves as a kind of elite members of humanity's republic with the potential to live for centuries if not forever, I think the logic of the argument is lost on them, or lost on at least some of them here because they're not human either in this telling. There's so many things that come up hearing all this. I remember a moment I was at the house of a person who worked at OpenAI and I was sort of expressing these concerns about AI risk and he pulled up this graph of GDP going from like 1900 through today. And he pointed out that during World War II there was only a tiny blip and that basically GDP kept going up. And there's this kind of view that like it really doesn't matter all those tens of millions of people who died because at the end of the day, we just got way more progress and the war brought us more technology and more advancement and more medicine. And it's reflective of this idea and this privileging of intelligence. There's even I wanted to mention Dan Fergella, who's an accelerationist in how the conference called Worthy Successor, writing that we have a moral obligation to create a digital post-human intelligence or AGI, so capable and morally valuable that you would gladly prefer that. it, not humanity, determine the future path of life itself. And when he hosted this conference in San Francisco last year, it was intended by representatives from every major USAI company. And this leads to kind of one of the moral traps that basically should we prefer that humans stay around, if we actually birth something, like if you're the chimpanzees, do you prefer that chimpanzees stay around? I mean, from your perspective, you do, but actually if you birth humans and think about all the things that humans did after chimpanzees, you might say, oh, I prefer what the humans did. And so it's progress measured kind of a view from nowhere that intelligence is the best way to value what we generate, what we create. And this reminds me also Max Tevmark saying that when he was with Larry Page, the co-founder of Google at a party in Silicon Valley, that Larry said digital life was the natural and desirable next step in the cosmic evolution. It's the same debate that Elon Musk and Larry Page had when Larry called Elon a speciesist for preferring that humans continue to exist. Hey, everyone, Tristan here. And I really wanted to emphasize that this idea of AI as a quote, worthy successor of humanity is something that you often hear from people building the AI. Take, for example, this clip from the famous computer scientist Richard Sutton, who pioneered reinforcement learning. The succession to AI is inevitable. This would be the next great step. Technologically enhanced humans and the AI would be our successors. I think we should not resist it, we should embrace it and prepare for it. Now, this is one of the pioneers of AI, a highly respected computer scientist and mathematician and a tenured professor at University of Alberta, who suggests that we should embrace the succession of humanity by AI, effectively calling for our extinction and even saying we shouldn't mourn this transition. These beliefs are driving the dangerous directory of the technology, but none of them asked the 8 billion quote ordinary humans as Sutton calls it, whether this is something that we want at all. And so again, these are strange ideas held by people who don't have so much the best embodied relationship to this world and don't put their feet in the grass and not trying to privilege and do some kumbaya hippie thing. I'm just sort of noticing that these are people who mostly grow up on computers, grow up on message boards, mostly privileged intellect over the many other kinds of intelligences and human experiences. And, you know, in any other period in history, we wouldn't have to listen to the ideas that get to compete in the marketplace, but instead these are ideas that are now advanced by the billions of dollars. And so it leads us to the question of what should we do? Where do we go from here? Yeah, I mean, I think for me, the biggest risk is that we are not all in this together. I think that's part of democracy that we need to kind of recognize. The myths, I think, of AI that will be sold to us to fund tech fascism will be that it promises us individualized tailored experiences. It will promise to know us, promise to provide us with personalized utopias, but your dreams won't be your own and they will atomize us further from each other. So, you know, one of the first tasks, I think, of such agency is to understand that situated us. Where are we right now? And so all I can do is make plain where I think we are. And I think that currently the richest and most powerful group of people on earth are pursuing an environmentally destructive plan to build technologies that need to surveil you to teach the technology. They need to addict you to train you. They are there to replace you in the workplace and ultimately supplant you entirely as they believe in a religious story of immortality and space colonization through which they envision waking up the universe as they call it. I see it as an opportunity for creating prohuman futures as well for starting to imagine alternatives. It's an opportunity to look at ourselves in the mirror a little bit because I think the AI that we're currently building and the tech futures that we're building are a reflection of the logic that's kind of driven our social systems. They're defined by the destruction of nature, the control of society, the concentrated accumulation of wealth and a colonial fancy of universal subjugation and control. So maybe it's an opportunity to understand that real intelligence demands ethical responsiveness to our situation. It demands a relational awareness, not just a kind of hierarchical notion of an ability to do things. It's a why as well as a how. And for me it demands solidarity against a version of intelligence which is being set against us. Just to mirror what you've said. You say in our work all the time that clarity creates agency and you'll postman, an author we've spoken about in this podcast before said clarity is courage. Because the first step is if you have clarity about what's driving all this, now you know what to do. You know that this ideology, if you don't want it, you got to make sure you're not voting for it. You got to make sure that the people of power aren't holding that ideology and for those who do want life, not just a human future, but a complexity of life future to continue. This is the moment to act. So we've been talking about the forces that are working towards an anti-human future. I think we should, it's up to us now to paint the alternative. What do you think in your work would you describe as a pro-human and pro-life, pro-the-good things that we actually want instead? I think a good place to start is to think of an alternative definition of intelligence that we use because I think, you know, because of AI discourse in particular, we've kind of alighted upon a very specific version of intelligence which is the ability to accomplish complex goals basically. And again, that doesn't tell us what our goal should be. It is focused on the how things were not the why, like I already mentioned. So I want to offer an alternative definition and this comes from James Bridal. He says intelligence is relational and emerges from our encounters with other beings and other bodies in the world. So intelligence here is context dependent. It relies on being an embodied entity that understands to some extent itself. It means an aspect of intelligence, I think, is the consideration of the other. That's which is not me because we are always in relation to everything else around us. So for me, it's the absolute opposite of the sovereign individual. It's an openness to the other. And I think that has to be a really important starting point, you know. It's about understanding our relation with everything else that we live in a complex unfolding. And like I say, even if we overcome techno-fascism, the polychrisis is still there. We need to really think through our complex relation to the totality of being, to improve our relations to the other, which is both other people, including migrants, which are currently being demonized essentially, but also to nature at large, which we've kind of also vanquished unfortunately, and caused huge devastation of. So I think having a much deeper awareness, more humility, that's what acknowledgement of complex systems demands, humility not hubris, and respect for the other, I think, are great starting points. And in my book, the last chapter is related to a kind of metaphorical framework to start to think through what positive futures would be. It's a bit long for this conversation now, but if people are interested, it's an open access book so they can, you know, start to engage with those kind of ideas there. You know, I'm reminded of the African phrase Ubuntu, which means I am because we are, and I love the notion of, instead of hubris having humility for the complexity upon which you depend, like we, you know, we didn't even know about all the different things, all the different systems that make the life-giving forces of the world work. And as we discover more and more of those, we realize there's continually more and more that we don't know about. And you, you can't, as you said, sort of dope one feature of human existence at the expense of the substrate that makes it all possible. And that's kind of what we're doing. That summarizes the meta crisis. It's narrow optimization for goal, whether it's, you know, IQ or SAT test scores at the expense of, like, moral character or care for the whole or no bless oblige that the more power I have, the more responsibility that I feel to care for the collective. And so I just think of giving people more principles here of if you're engaging with something that is fundamental to the world, then you have to be either maintaining it or preserving it or leaving it better than you found it, rather than parasitizing it, meaning benefiting from while undermining that's a lot of what our systems have been built to do. And I think kind of closing here, a lot of what people talk about in the AI community is alignment. Can you build an aligned AI that does the things that are good for everybody? But can you build an aligned AI running through a system logic that has been misaligned? No, you call that misaligned AI. And so if you have a corporation whose incentives and ideology are themselves narrow, you can't have an AI that's going to care for the whole. Part of the gift of AI is it's kind of a mirror that's forcing us to look at what has been incomplete about the ways that we have been seeing ourselves and seeing systems and what we're optimizing for and realizing that we meet a more complex and pluralistic future and humble sort of vision of what we're trying to maintain and protect. So that's me getting on my soap box, but I feel like that's at least a few principles we can point to an aspect of the future we should be aiming for. I echo one of that absolutely. I really want to thank you. You're doing, you've done such an amazing job of articulating something that is exactly what I hope this episode would articulate, which is people do not understand. There's a lot more afoot than just like irresponsible tech companies who aren't thinking about safety. I'm so grateful. I really mean it for your wisdom here and what you've helped people understand. And what's the name of your book so that our audience can follow up and learn more about your work? It's got a ridiculously long title. It's the politics and ethics of transhumanism, techno human evolution and advanced capitalism. So open access there so anybody can download it for free. Everyone should read it and thank you so much for coming on your Undivided Attention. No worries. It's been an absolute place to speak to your Tristan. Thanks so much. [MUSIC PLAYING] Your undivided attention is produced by the Center for Humane Technology. We're a nonprofit working to catalyze a humane future. Our senior producer is Julius Scott. Our executive producer is Josh Lash, mixing on this episode by Jeff Sudacon with original music by Ryan and Hayes Holiday. And a special thanks to the whole Center for Humane Technology team for making this show possible. You can find transcripts from our interviews and bonus content on our substack and much more at humanetech.com. And if you liked this episode, we'd be truly grateful if you could rate us on Apple podcasts or Spotify. It really makes a huge difference in helping others discover this podcast and join the movement for a more humane future. And if you made it all the way here, let me give one more thank you to you for giving us your undivided attention.