What Does Effective Altruism Really Believe?

Hey, everybody excited to share that we'll be having another plain English live show right in my own backyard here in Washington, DC later this year. I'll be live at the Howard Theater in DC on Tuesday, December 8th, and I hope you'll join me. Tickets go on sale this coming Thursday, October 29th at 10 a.m. Eastern time. The ringer dot com slash events will have all the details very soon. That's the ringer dot com slash events again, plain English will be live at the Howard Theater on December 8th, and you can grab tickets starting Thursday, see you there. On September 16th, 2026, the Trump Department of Defense, aka Department of War, posted that the United States will never be an effective altruist country. This came with an image of Trump pumping his fist with the tagline, Americanism, not effective altruism. Two weeks later, the cover of the economist declared that effective altruism was the 21st century's biggest idea. What is effective altruism? I am in a strangely unique position to begin to answer this question. In 2013, I was living in New York, and my roommates left to start a company in Africa. Some subletters moved in. Her names were Will and Amanda McCaskill, now Amanda Askel. She is currently the lead philosopher at Anthropic, and he is known as one of the founders of the effective altruist movement. Over late night whiskies, I got a front row seat at the emergence of a philosophy that has asked people to think harder about the consequences of our actions. What began in the early 2010s as a nerdy way to rethink the most cost-effective charity has instead become a lusher philosophy to rethink just about everything. Our consumer habits, our career choices, our technology, and our approach to the most existential issues of our time. Today's guest is Will McCaskill. We talk about the birth of effective altruism, its history, its insights, and its blind spots. Its high-profile challenges, including the conviction of Sam Bankman Freed, a major funder of the movement, and what effective altruism says for this moment in history. I'm Derek Thompson. This is Plain English. This episode of Plain English is presented by Comcast Business. Did you know Comcast Business detected 79.3 billion cyber threats last year? That's why they're introducing Security Edge Preferred, its big business cyber security designed for small business. Get started with Comcast Business Internet for $50 a month for one year on a qualifying Internet package and find out how to get one month of Security Edge Preferred on them, plus ask about a $500 prepaid card. Limited time offer, restriction supply, requires 300 megabits per second Internet Security Edge, Security Edge Preferred, and Wireless Connect. Call today. Will McCaskill. Welcome to the show. Thanks so much for having me on. Well, you're considered one of the founders of effective altruism, so you're probably a good person to ask, where did effective altruism come from, and when did it come from? So I'll tell you about at least my part of the story, which for me began really in Oxford, UK, in 2009, where the intellectual underpinnings really came from Peter Singer and series of arguments he'd made for the idea that if you live in a rich country, because you can help people in poor countries so cheaply at such little sacrifice, you were modally obligated to donate a large fraction, maybe even the majority of your income to those non-ploughits that could help the poorest people in the world. And I had been very influenced in that by that idea, but had done very little until I met another graduate student in Oxford at the time, Toby Ward. And he had spent years thinking about this, taking it very seriously. He had made a pledge to give everything he earns above a fairly low baseline of about 20,000 pounds per year, Oxford 2009 adjusted. And he had also been doing a lot of work in a side of things that I had not appreciated, but I think became the distinctive aspect of effective altruism, which was thinking about not just how much should you give, but also where you give. And what are the charities that do the most tell other people with every pound or dollar they receive? Someone listening could think that you're basically just applying philosophies, otherwise known as utilitarianism or consequentialism to philanthropy. Why does effective altruism have a name at all? And why is it effective altruism? So I think there's a couple of things there. The first is that all ethical perspectives should care about helping others. And in fact, almost all do. And as long as you care about helping others, whether you're a virtue ethicist or a libertarian or any other model perspective, well, then you should care if you can help more people rather than fewer or help by a larger amount rather than less. And surely that's actually important. But the striking thing in at least academic ethics over the last 50 years was that even though there'd been a lot of discussion of producing as arguments in this area, there'd actually been very little work done to really work them through and figure out, OK, well, if you do want to help other people by as much as possible, what should that involve? And so effective altruism was almost called positive ethics. So academic ethicists normally talk about thou shalt not. And the just debate kind of does that apply to abortion? Does that apply to animals? But there was very little on the kind of thou shalt, actually, if you want to help others by as much as you can, what in fact should you do? And so I think the distinctive part of effective altruism was really taking that aspect of things seriously and really trying to work through as thoroughly as we could the effectiveness question. The question of just, you have limited time, you have limited money, but you want to have as much positive impacts as you can with that, with those scarce resources, what should you do? Four years later, in 2013, we were roommates and my mother passed away the previous summer before you moved into my apartment in New York City. And my first interaction with effective altruism was you pointing out that many people who want to do good in the world via philanthropy will act in a way that is often shaped by social status or by the desire to do something that is sentimental, but might not be consequential. And you said there are so many opportunities, Derek, to donate money in a way that can actually save lives. And I was incredibly motivated by this, gave tens of thousands of dollars in my mom's honor to the Against Malaria Foundation to buy malaria nets. I was also very interested in arguments that you made outside of philanthropy. If I recall, for example, we talked about the idea of choosing what to do in a career. You said, you know, if you go to work for a philanthropy as, you know, the some grunt at maybe PETA or some other do good philanthropy, if you don't get that job, someone just like you is very likely to get that job, but if you get a job at Goldman Sachs and you give away 60% of your income, you're probably donating much more of your income than the next guy up or next woman up at Goldman Sachs. And I use that story as a door into this other question I'm interested in, which is how did the ideas of effective altruism escape containment from the world of philanthropy and end up walking into all these other spaces, how we should act as consumers, how we should think about our careers as doing the most good. When did those ornaments start to get hung on the E.A. tree? Yeah. So in terms of the history, in 2009, it was very focused on giving and I set up giving what we can, which was encouraging people to give at least 10% of their income to those most effective charities. But we were talking a lot to students. I was myself a PhD student at the time and there was this question of just, well, should I become a philosophy professor, is that the best thing to do? An undergraduate was wondering about whether to go into doing a PhD himself or to go into investment management with the aim of earning to give and so on. And because there was this demand for just, look, this is a question that many of these students actually need to are grappling with and where they do want to do good with their career. Based on the graduate Ben, Todd and I gave a lecture that turned into a series of lectures that then turned into this organization 80,000 hours that was trying to take this same effectiveness oriented mindset. How can you do the most good with your scarce resources, but applying it to time as well as to money? I wanted to tell us a little bit about what 80,000 hours means and what better organization does. 80,000 hours, the number of hours you'll typically work over the course of a career. And the thought there is twofold. One is that man has a lot of time. So it would be worth spending many hundreds of hours to try and figure out, OK, if you spend a 1% of time, that's 800 hours. That's many weeks of time to figure out how to spend the remaining 99%. But yet, many, many people, at least just kind of sleep walk into some career that other because it's the consulting job that they're expected to go into or what their parents wanted them to do or something like that. But then the second point is that it's also just a finite amount of time. The problems we're talking about in the world, no matter how successful you are, you're not going to be able to solve them as one person. And that means you need to prioritize. You need to think, how can we use that time as effectively as possible to make the biggest dent in one of these really big problems? In about 15, 20 minutes, we're going to get into some critiques of effective altruism as it exists today. But I wonder going back to, say, 2009 to 2013, what the most common critiques of effective altruism were back then? Because I'm sure someone listening who's a bit innocent about the concept of effective altruism is thinking, one should give their money where it is most likely to save lives, one should think about what they're going to do with their career a little bit, considering that they're going to work at 1,000 hours. These are somewhat harmless even anodine, even obvious ideas. Were there identifiable critiques of the philosophy that you were trying to grow in this period? They were very, very many critiques all the way back from the giving 10% to help people in the global poor days. People did call it a cult back then, where the idea of donating 10% just seemed a bit extreme to people, whereas I think that's just now much more widely accepted as just like a bit normal. And I think very respectable thing to do. There were critiques along the lines of being too OK with the current capitalist system. So very common critique was like, basically, if you're not trying to overthrow global capitalism, what are you even doing? Whereas I did and still do just think that even though I'm sure that better economic systems that are possible, that is not the most effective way of spending one's time. And then there was also a critique that also always felt more like a kind of within tent critique of just the question of, what can you measure? Is there a risk that by focusing on what the data tells you that you're going to end up focusing on what's quantifiable rather than most important? That's always been just a very hard question and something we've tried to improve with more data, with more careful arguments and so on, but has always been something to grapple with. I want to keep telling the story of this philosophy's evolution, because one might think of EA1 as being evidence driven philanthropy. And then EA2, which crops up right around the same time, is utilitarianism doing the most good for the most people, but as a philosophy for modern life as a consumer perhaps, if you believe that animals have conscious experience and they suffer, then you should be a vegetarian, as a worker doing the most good with your career. So 80,000 hours and being really purposeful about the way that you are adding value to the world as a worker. Now this brings us to, I think, the next development of EA. We got beer. I don't know if you remember this, maybe in 2015 or 2016. And I was asking you what you were thinking about for the next chapter of effective altruism. And you told me that among other things, you were very interested in this idea that there were a lot of people that would exist in the future. And we needed to think more about their interests. And this previewed what later became known as long-termism, which I have heard you summarize with three sentences. One, future people count. Two, there could be a lot of them. Three, we can make their lives better. Going sentence by sentence, how would you explain what long-termism means? Yeah, so the underlying thought is that, okay, all humans are equal. They'll equal across space. So the interests of someone living in a poor country is that that's just as mad as just as much as the interest of someone living in a rich country. But similarly, the interests of people shouldn't matter across time either. If I can benefit someone in a week or a year or a century or a thousand years, the interest is the same. It matters just the same. The time itself is not really the relevant dimension in terms of how much you should care about that person's happiness for suffering. And then the second point is just, well, once you take that seriously, you should bear in mind and take seriously the fact that man, there's just a lot of future to come. I think quite plausibly, we're very early on in the civilizations development. And so we should look ahead to thousands or millions of years or maybe even longer that human society and civilization could continue into the future. And so then that means, wow, there's just really a lot of stake. A lot of stake if there are things that affect how the long-term future might go. And then the final point, which is most of the book, what we are the future spends its time on, is saying like, yeah, actually, there are things that are happening in our lifetimes. And since I've written this book, I now think within the next few years that we'll have very major impacts, not just on how well this generation goes, but how the whole course of the long-term future goes. And put that all together and you get the view that these things that will impact the long-term future, whether it includes how we handle pandemics, how we handle AI, are among at least the very most important vulnerabilities of our time. - What does it really mean to care about somebody who won't exist for 1,000 or 1 million years? - I feel like I can pretend to care. I mean, in an emotional way, to literally care about that person's happiness. I don't know that I can do it. When I care for my daughter, wife, friend, it's not just that they exist here. It's that I have an enormous amount of certainty about the things that make them a person and the things that they care about. And so I wonder how you deal with this. I know very frequent objection to an aspect of long-termism, which says that we should care equally for people who live a million years from now, as we do about people who live today. Don't we simply have so much certainty about the people who exist today and certainty about our feelings for them and certainty for what they want in the world? That it's completely, it's a different enterprise to care about someone who exists rather than conceive of care for someone who won't exist for a million years. - Yeah, so I think that's a great question because it kind of illustrates a way in which long-termism gets misunderstood, I think. So I think if what I was trying to do was think, okay, what are going to be the problems that will really face people in the year 4,000 AD or something and how can I solve those problems? That would be kind of foolish, I think, because we're not going to know they're going to be quite radically different than the problems of today, for example. However, what it does mean is that we should draw attention to risks that we face now, that short-term risks, but that have long-term consequences, where that can involve a worst-case pandemic that leads to the collapse of civilization or the extinction of the human race, or an AI enabled to talentarian state that can persist indefinitely, or to the loss of control to AI systems themselves that then mean that humans are just no longer behind the stealing wheel of society at all. These are really quite broad thoughts about how future society might go, and we don't really need to know very much about what are the problems that might face someone in the year 3,000 AD to know that it's a good thing if humanity is still around and has not gone extinct or civilization has collapsed, or to know, it's a good thing if we don't live in this totalitarian state, or it's a good thing if misaligned AI's have not taken over and now have humans as the underclass. And so it's, yeah, the kind of decisions are at that level of generality, and you might still disagree, might still think that some of those things are good things, and then we can get into an argument about that, but I think that's at least a lot broader and kind of safer position to hold than some very particular claims about how things will go in the future. - I wanna enumerate some of what I understand to be the guiding principles of EA to on-ramp us to a question that I'm really, really curious about your answer to. So number one, obviously people in rich countries the US can save or or improve the lives of people in poor countries with modest donations to the right charities. One, two, with 80,000 hours in similar organizations, our working careers are both finite and significant, and we should think very deeply about how to use those 80,000 hours to do the most good. Three, humanity can exist for a very long time unless we go extinct or civilization collapses, and so we should try to not have civilization collapse or go extinct and care very much about things that present existential risks to civilization. You've done work on this, Toby Ord has a book, "The Presipist," which is all about existential risk, whether it's AI or comments smashing into Yucatan Peninsula's, and then you mentioned AI, which I really want to say for the end. I am interested in effective altruism as a collection of moral principles, but I am also interested in it as a way of seeing the world, as a way of paying attention to the world. I think moral philosophies, if they're successful at any level, they almost give their adherence like a set of goggles. Stoicism asks people when they're in a difficult situation, hey, immediately, separate what's under your control versus what's out of your control, and you hear athletes all the time, quote back to half time and end of game questions, they'll just say, "I gotta only focus about what's on in my control." So Stoicism has succeeded in giving millions of people a really clear pair of goggles. Aristotle, I think, in Virtuethics trains people to think about virtues as, you know, you're a good person cultivating through repeated action and behavior, the good, that which is courageous. What are EA's goggles? Like once someone internalizes effective altruism, what do they start seeing everywhere that other people do not? - Yeah, I love the flaming, because I think that is like for effective altruism. It's a mindset, and it's a question. Much more so than it's like a set of answers, or even a set of principles. So you can be into effective altruism and very skeptical of long-termism, very skeptical of working on catastrophic risks and so on. And so what is distinctive about effective altruism? One is that, you know, the very basic idea of when you're thinking about, okay, I'm too good, is this the most good that I could be doing? Or is there something even better, even more impactful? A second thing in the mindset is just taking scale seriously. So early on, we would talk about how some charities vary, you know, in their effectiveness, not just by 50% or 60%, but they would be vary by orders of magnitude. So one charity would do 100 times as much to help people as others save 100 times as many lives. And that sort of scope sensitivity is something that is actually quite rare, but I think it's extremely important. And then maybe I'll maybe say like another part of the mindset, as well as just thinking a lot about counterfactuals. What would happen if I don't do this? Where for some things, they're really neglected. If you don't work on them, if you don't contribute, then nothing is gonna happen. And very little is gonna happen, whereas in other areas, oh, there's already just a ton of attention and work so that the difference that you make by contributing, might still be meaningful, but maybe it's a bit more marginal. So these are some of the kind of underlying questions that have this that I think help to define the EA goggles, as you call them. - One of the fact about tourism's most famous adherence, and certainly it's biggest funder for many years, was Sam Bankman-Freed. And famously, FTX, the crypto exchange that he ran, collapsed, he was convicted of fraud, you knew him. What happened here, from your perspective, what did you get wrong? What did the EA community get wrong about him? And what did you learn from that experience? - Yeah, I mean, the whole that FTX collapsed, the bankruptcy of the fraud was just this huge catastrophe for the further world. It was like a huge harm that was inflicted on literally millions of people who suddenly didn't have access to the investments they've made in that exchange. And the whole affair, speaking frankly, was just a matter of great shame and embarrassment, both for me personally and for the movement as a whole. I did know him, he was, had been involved in effective altruism for many years, and then when he was setting up the company, I was advising on his philanthropy, and he became one of the biggest proponents of effective altruism. And yet, it turned out that things were not at all what they seemed. In fact, I think, or I now know that I'd been like lied to one, many occasions to my face, by the people involved in FTX. So what went wrong? I think a really big thing was just like, effective altruism started off as this just small, very tight-knit group of people who were just, you know, everyone really tossed to each other, kind of like a family or something. And it grew and grew. And I think there was this sense of, well, if you're into effective altruism, oh, you're kind of part of the club, we can just extend trust to you in this kind of open-ended way. And that's certainly how I think I was acting. And I think that's, you know, an ever. Saying you're into effective altruism, or even being into effective altruism, does not tell you much about other aspects of someone's character, let alone kind of how they've owned their companies and so on. And so I think, so effective altruism after the FTX clash, like as a movement, made quite a lot of changes. There was big institutional overholes in, you know, most of the leading kind of organizations. And also I think just moved a bit away from being a kind of community to more like a professional network or something. More like, look, this is something we do, but the fact that you say an effective altruist doesn't mean you like suddenly should be extended kind of a lot of trust. And so I think as a whole, it was just, you know, a hugely painful and important kind of growing up moment where we could move them, yeah, being that like small fledgling group of people with a set of ideas who all felt like they could trust each other. So something that's kind of, you know, needing to be a bit more engaged in the real world take issues like in particular kind of institutional governance just very seriously indeed and not lie so much on kind of trust networks, I think. - Yeah, it's interesting to see you, to hear you talk about something like institutional governance, which is something that almost, by definition, can't scale, right? There's a way in which while I don't think someone has to choose entirely between Aristotelian virtue ethics and effective altruism, the weakness of virtue ethics, which is also a strength, is that it's not interested in scale, right? When you say, I want you to model your sense of courage and honesty and the people that you were surrounded by who you believe to be courageous and honest, that's not a feeling that can scale, that's behaviorism, that's just cultivate this thing in your life. And that's different than try to do the most good for the most people and associate the good with a mechanism that can scale. And so I wonder if the SPF catastrophe made you update about effective altruism at all, whether you feel like it exposed a flaw in the philosophy itself, or whether you still believe in the exact same cause that you did before Sam Bankman freed, but simply have a different attitude toward trust and issues like institutional governance, while still believing all the things you did before the fraud was exposed. - Yeah, I mean, I think I believe the ideas at that core, in just the same way, where this idea of just do more good donate a significant fraction of your income, choose a career on the basis of what will help others. Like, I think none of those ideas should get tainted by the actions of a small group of people who did bad things. In just the same way, the actions of Ted Kaczynski, the unabomber, do not undermine environmentalism. I think the lesson instead is just in like, well, what does that mean for how you live your life and how you like run a company and so on? And it's just effective altruism should never, I think, come at the expense of like, either on an individual level, a kind of common sense guardrails, common sense kind of prohibitions, as to how one lives the kind of like, classic ends don't justify the means, it's like it's their, it's part of common sense wisdom for the reason. Nor kind of on the institutional level, they're kind of classic, just best practice of, if you're a company, have a functioning board, have like, you wanna be in a system where it is, you know, if are a bad actors, it's just not possible for them to actually be doing bad things, it gets caught kind of early on? I think that's the kind of constraints that both individuals and companies should be operating under. But then within those constraints, absolutely, continue to try to work out how can we have the biggest impact and go and find make the world better on that basis? Here's the economist on the views of effective altruism that they consider questionable to be on the pale. Some EAs think that insect life is in aggregate more valuable than human life because there are so many insects. Some say it would be better to bring about the existence of trillions of humans whose lives are barely worth living than to have mere billions with genuinely good lives. Others even see the welfare of artificial intelligence itself of being of vital importance. They would redirect resources toward future humans or non-humans and away from people suffering today. Do you agree that A, there's a strong argument that insect life is more valuable than human life, or B, that it is defensible to redirect resources toward future non-humans away from people suffering today? Yeah, so I have a response to this economist piece, defending the slangeness of effective altruism. So these are all ideas that get discussed among people within the effect of altruism movement. And your response might be like, "What? This is just so weird." These are just like crazy folk. However, I think we now have this long-track record in both history of people looking like model weirdos when they advocated for ideas like women's suffrage or the abolition of slavery or civil rights or environmentalism. But then the includes light by history. That's certainly not to say that all the ideas affect altruists are saying are correct and certainly not at the importance of those ideas. But if you're going to make moral progress, then you have to really think about things and you really have to be open to ideas, even if they seem bizarre or weird at the time. And in the effect of altruist movement, we have seen that ideas that seemed esoteric or bizarre at the time, like worrying about pandemics, which many within effective altruism had been for many, many years before COVID or worrying about AI. Again, many, many years before the CHATGBT moment actually reality can catch us up and those concerns seem more pathetic than diluted at the time. And so I just think it's really important to be debating moral ideas that have good moral arguments behind them in a just like quite open-minded sort of way. And all of these topics are just absolutely bread and butter sort of discussion topics in our academic ethics, seminar or something. And so yeah, I just don't think they're beyond the pale at all. And there's a fact of record of saying like, whoa, that's just that idea is too weird. We shouldn't even talk about it. I think is feasibly badly actually. I agree with the argument that many ideas we consider to be morally obvious today were morally insane hundreds or thousands of years ago. But to repeat at least the first part of the question, do you personally believe that insect life in aggregate is more valuable than human life? Yeah, so I don't, I don't buy that one myself actually. The insect. So some people focus on insect suffering and what they're concerned about is just the sheer quantity. You know, it's just take shrimp farming, for example. It's like hundreds of billions of shrimp every year get put in these like awful conditions. They get suffocate or get frozen to death. In many cases, their eyes get ablated. And they feel pain. So if you give the shrimp anesthesia, they don't behave in the same way. They don't like touch the ablated isotope in the same way. So we have pretty good evidence that they feel pain, not decisive evidence, but some. And then the thought is just wow, take really seriously like the sheer quantity of suffering there. I just still think it doesn't come out as the most effective thing to focus on, even if you're just looking at kind of near term suffering. I think other areas of animal welfare and particular effectively farming of chickens look more compelling. So I think like, yeah, so I think with my like, doing the most good hat on hat, I'm not in the like worrying about the suffering of insects camp. But I do want to defend the people who are like come to a different conclusion than I do. And I definitely don't want to say like, oh, these are, this is like beyond the pales that even be thinking about this, because we would have been saying it's the state beyond the pale to worry about the suffering of pigs or chickens, not, you know, really not very long ago in human history at all. Sure. I have a philosopher friend in DC who has a joke about what he considers the weird evolution of effective altruism. He says, EA used to be about protecting humans from mosquitoes. And now it's about protecting mosquitoes from humans. And while I think that's a clever laugh line, I also think it touches on something that is quite fundamental and potentially problematic for a philosophical movement, which is that on the one hand, you do want to cultivate a degree of openness. I am absolutely on your side and understanding the larger principle that if you automatically shut down any idea that exists outside of the boundary of accepted moral thought, you cannot have moral progress because you've made the wall around current moral thinking the only distance that people allow to travel, understand that entirely. But there is also something to this idea that whether it's a philosophical movement or a company, culture is what you tolerate. Culture is what critics see as tolerated inside of that community. And a philosophical tradition that seems to be basically okay with lots of people arguing that, you know, insect life on aggregate is worth more than human life could potentially say, well, if, if they're thinking that, well, now we can argue against all of these other thoughts, whether it's effective out whether it's an artificial intelligence, pandemic preparedness, philanthropic consequentialism, that has nothing to do with Shimp lives, mosquito lives. But nonetheless, the impression that the field is open to those arguments can sometimes reflect on the way that outsiders see the entire movement. So I guess the question that I'm grasping toward is, while I agree that the weird factor might be necessary in part to create forward moral progress, do you worry that the weird factor ends up being a a tar on the philosophies embraced by the larger public now that EA is absolutely escaped containment and is on the cover of the economist magazine. Yeah, I mean, it's certainly, it's certainly a cost. These are certainly sticks that effective altruists can get hit with. And in particular, because the perceptions, you know, what spreads is the juicy spicy takes and so on, where the article you're referencing is written by like a 22-year-old blogger. He's like just out of undergraduate and got big on the internet because it's this like, you know, spicy provocative take or the idea of there is a project called the Shimp welfare project that's concerned about the suffering of Shimp. But like effective altruist, yeah, charities, yeah, we're looking at like in the low billions of dollars that have moved to global health and development, a vanishingly small fraction of that has been raised by I think by the Shimp welfare project. And so if the critics are harping and like really getting worked up about the discussion of Shimp welfare, well, they're choosing to focus on what is just a very small slice of all of the activity that goes on. And I guess I just think like well, judge the movement by the whole, where that's saving hundreds of thousands of lives via donations to the most effective global health and development charities. And talking about the avant-garde frontier of ethical issues as well. And yes, I think that means that some critics can just seize on whatever is weirdest at the time and then pretend like that represents the whole of the movement. That's a shame, that's a cost. But I'm very, very wary of the idea of kind of self-sensoring on issues that might be of great moral importance for fear of critics calling you out as being too weird. I think that that kind of self-sensorship I think would have aged very poorly, or would have done very poorly in previous eras of human history, and I think we need to hold on to it now, and it means that some critics will attack us and say la vie, I guess. I want to build a bridge here from our discussion of the critiques of effective altruism to artificial intelligence, where EAs have been objectively influential. There's a very particular critique of EA that comes from the tech right, from conservatives in and around Silicon Valley, and if I was going to do my best to summarize the upshot of their critique, I think it's something like effective altruists by dint of their obsession with long tail catastrophe. I have essentially fallen prey to the precautionary principle. They're simply too afraid of bad things happening, period. But civilization has always moved forward, not just because of moral progress, but also because of technological, scientific, and economic progress as well. Civilization needs ambition, it needs competition, it needs something more than just an obsession with welfare. I think there's a Nietzschean vibe you could even say to their approach, that we need to be less afraid of unleashing the animal spares of capitalism to move forward civilization. And we can't listen to these whiners who think that we're always about to end the world. What do you make of the tech right critique of effective altruism? Do you think they see your philosophy clearly and do you think there is any wisdom in their critique? Yeah, so I'll say a few things. One is like a drum I will always beat is that effective altruism is compatible with many things. You could be an accelerationist, want to just speed up technology as much as possible and still be within the effective altruism camp. If you really think that's the best way of doing good, and I know some people who are in that category, however, is in fact through that within the EA movement, there's a lot of concern about catastrophic risk from pandemics. In particular, the risk of a future wave of new pandemics wants the ability to create new viruses becomes more and more widespread. And then secondly, there's a concern about catastrophic risks from AI too. Now, this isn't a matter of just general pessimism about tech. I at least, and in fact, most of the people I know are generally very boosters about tech. I think like living standards for humans are radically better now than they were 100 years ago or 200 years ago, and that's primarily through new technology. In general, I think we don't harness the benefits of new technology enough and throw nuclear power, and I think that it's a catastrophe that got regulated to death essentially. However, that is not all, does not need to always be the case. Take the case of AI. I, most of the people working on AI think that there's a very meaningful chance of utter catastrophe coming from development of AGI and then superintelligence, where very meaningful chance doesn't mean like, oh, it's a one in a million. We should play safe even if the chance is so low. It's like, no, no, no. The average AI researcher thinks the chance is on the order of 10% that you get an outcome as bad as human extinction or comparatively as bad. The people who are concerned include literally the most scientists, most highly scientific scientists ever, such as Jeff Hinton and Yoshua Benjiro, who are independent voices. Jeff Hinton even left Google so that he could be an independent voice. They think the risk of catastrophe is even higher again. Then finally, I think there is just forecasting difference between me and some of the others and some of the people in the tech line, where they're excited by more progress. Maybe they want economic growth rates of 5% or 6% per year and think we can get there. I think that would be amazing, but what I think is going to happen is that at the point of time that you get AI that can substitute for an AI researcher itself, so now known as the cursive self-improvement, it's not just that you get 5% or 6% economic growth per year, instead you get a leap forward from AI that is slightly better than what we have today to suddenly AI that can replace all cognitive work, including all science, and maybe even radically better than that. You get that leap within a couple of years. That's really a bit of progress that I think the world has a bit of tech development that the world has never seen before. My guess is that if the accelerationists thought that that was what was going to happen, they'd think that maybe we were ought to be a little bit cautious about it too. It's a huge irony, I think, that many effective altruists and EA adjacent people have been both banging the drum about AI danger and are also on the front lines of building artificial intelligence. I mean, the open AI board had effective altruists and EA adjacent people like Helen Toner from the start. Dario Amade was an early signer of the giving what we can pledge. His sister, Mary Holden Karnowski, who I believe was one of the co-founders of Give Well, one of the organizations that has done the most to identify the philanthropies that do the most with the money that they're given. There's a ton of EA influenced folks that are building a technology right now that you believe is on the cusp of. You mentioned a few years on the cusp of delivering catastrophic outcomes. Are you frustrated by furious about the frontier labs because there is sometimes when I feel like I personally want to scream with these companies saying on a Monday, we think we're building something that could be really dangerous. Somebody please stop us. And then on a Tuesday, they're like, oh, by the way, we solved 700 math problems. So there's simultaneously saying we're doing something terrible. Please slow us down. Let's slow. Let's pause. Let's pace the frontier. And then on Tuesday, we just made a giant leap forward in scientific and mathematical progress. I mean, there's there is something unbelievably incongruous and potentially insane that's happening here. And I wonder what you make of it. Yeah, I mean, I think it's just very confusing for the general public. And I honestly even sympathize with people thinking, well, surely they can't possibly believe this because they wouldn't be building it. I mean, the analogy is like imagine in the 1950s, you'd realized that fossil fuels produce climate change, via carbon dioxide emissions. And so you thought, okay, well, the thing we've got to do then is build oil companies, build coal companies so that we can steer this in a positive direction and accelerate the clean energy transition. And maybe that argument works, but it's certainly a weird move. Now, the situation is a little different in the case of AI because we have not yet crossed the Rubicon where the AI systems actually pose meaningful existential risks. So, in the early days, OpenAI got started as a non-profit research lab. And forpec got started off as a spin-off that was basically again going to be a research lab. And then it makes sense. You want to understand the risks from AI and you want to be able to make sure we align AI systems. Well, you've got to be working on it in order to be able to do that. And now both of these companies have evolved to become some of the biggest companies in the world. And the big question is, what will they do when they meet that Rubicon where the AI's that they have at the moment are disruptive, they're very powerful new technology, but they are still far away from the technology that would pose a meaningful existential risk in the world. And the big challenge that they will face is what happens when, no, actually now the AI's they're developing are posing like meaningful dangers. And I really hope that they're able to do like already that have been signs. They have talked about pacing and they are, you know, being open to having external auditors come into their companies and so on. They're calling for regulation. These are all very good signs, but as the AI systems get more powerful, the sorts of actions they'll need to take will need to be, yeah, a hundred times bigger than this. And the big worry is that they will be so locked into commercial competitive dynamics that, you know, they won't follow through on the kind of lofty goals they have to try and create and steer the technology in a safe way. Yeah, I mean, I understand what you're going with your ex-an analogy, but I think even that doesn't take into consideration the thing that you just said makes AI most dangerous, which is recursive self-improvement, AI that builds and controls itself and escapes human monitoring and oversight. With this is more like, to me, is like building a thousand hydrogen bombs that you believe might launch themselves because you think the Soviets are also building a thousand hydrogen bombs that you believe might launch themselves, even though it doesn't matter who builds the hydrogen bomb that launches itself, it's still really bad for the world and you have hydrogen bombs launching themselves. I mean, it's, it's, to me, it is significantly more catastrophic and reckless than starting Exxon because you think that people need like cheap energy to light their homes. Like people do want cheap energy to light their homes and, you know, I think climate change is really bad, but, you know, oil certainly isn't improving the biosphere choking capabilities of itself and then running rogue away from the Exxon chemical engineers. So I think what's happening here, both has some historical precedents and precisely because our RSI assumes a kind of runaway capability does not have any kind of useful historical precedents. I do wonder, you talk a little bit about, you know, oversight and I know that there are people who are like-minded, who work at meter and redwood research and other companies that study how AI behaves, just give me your number one suggestion, what is the number one thing that you think folks in Silicon Valley who care about both technological progress and human safety should do with AI right now? Yeah, my number one favorite policy that the companies could implement is about what fraction of their computing power they devote to research and development and future training and what fraction they develop to, they devote to infants that is serving customers or to work on safety and alignment where at the moment most of their computing power goes towards experiment and work to make the next even better model. And that's part of why we have such rapid progress and I'm happy for AI progress to keep going. In fact, I think if it kept going at the current rate or maybe a little bit slower is the kind of ideal pace, we could live through this golden age of human AI collaboration that would drive forward like productive science, make people more productive in their jobs, could like give us that five or six percent economic growth per year that we might really like. That is totally feasible and we could make it happen just in part at least via these decisions about how computing power is allocated because if only a small fraction is going towards this R&D process, then the explosive growth that I would otherwise expect, that explosive leap where you go from kind of below humans are suddenly far greater than human capability, the sort of thing that could just automate all cognitive work, that might occur over the course of a decade rather than over the course of one year. That's a lot safer from my perspective and it would mean that it's still, I know I said a decade, it still sounds like pretty fast paced, but relative to a year I think it's an awful lot better and it would mean that people could actually like leap the fruits of the AI technology we already have kind of much, much earlier and it means that everything could get developed more safely and securely, securely and so that's the kind of single policy, I think it's also very easy to understand that it could be implemented quite soon. You mentioned that effective altruism is not worried about slightly crazy ideas, so let's end with a slightly crazy idea. One of the things I think we're going to hear more about in the next few years as these models get more powerful is the question of AI consciousness as insane as those syllables might sound to many people listening or watching. And the logic as I gather is something like if AI is akin to a person in a computer, if we can find some way of demonstrating that it experiences suffering, that it is essentially a silicon shrimp, then open AI andthropic are logically something like slave empires. They are institutions that bring into being entities that suffer and can only do that which is demanded of them that I think describes what we if we thought of a person would call a slave. Do you believe that open AI andthropic are slave empires? So I think the situation we should have with respect to AI's and whether the consciousness is one of great uncertainty in a way that should really freak us out. So you look at the leading theories of consciousness from philosophers and cognitive sciences and so on. And AI systems, at least some of the AI systems that we can create, they tick many of the boxes, they are using similar amounts of computational power as medium sized animals. The idea that consciousness is only a human or moral status is only a human thing is just actually not very plausible. You look at that data in Star Trek or Spock and people very naturally think yeah no, okay, their interests matter. And so I think this is something like among the many of reasons. But data is a character. Does anyone actually think that like the interest of a character in Star Trek matter? Like if data is killed in an episode of Star Trek, this is not a human rights catastrophe. This is just a plot development. I wasn't meaning the character data. I was meaning just if you were in the Star Trek universe, the Star Trek were the all right, but why not why not data was the question and answer here. Why not go to animals? Why not go to the dogs? Is there a question of dogs or conscious? That's oh yeah, I mean like I say, I'm yeah, I obviously think that dogs are conscious and you know, it's unclear like where do you draw the line between dogs and nematodes or bacteria or something. But at least, you know, someone could say well, it's all biological and that's the difference. And historically that has been a very unpopular view in the philosophy of consciousness. And much more popular is the idea that even though we have no idea really what kind of consciousness, it's something to do with computational processes. It's something about like the function of how of what proteins do rather than the fact that they're implemented on neurons rather than silicone. And yeah, I basically, I think that this is an issue that society as a whole should be taking extremely seriously. And the thing that's particularly tough about the issue is that we are not going to get a good answer, a satisfying answer either way as to whether the AI is our conscious or not. Even once they're like humanoid robots walking around with long term memory that people have relationships with, we're still not going to know. And so the issue society has to deal with is what should you do given that we don't know? And I think it's, this is a case where it actually is, but you're fair to be kind of cautious. You said two things that are interesting. You said we should take it seriously. The question of the AI's can never be conscious. And you also said we'll never know. What does it actually mean to take seriously a philosophical question that we'll never know the answer to? Well, I think I'm kind of optimistic. We'll know at some point, but maybe, maybe it's decades or maybe it's hundreds of years into the future. But to take it seriously is to just be not to not dismiss it. So we'll be in a position where it's like, yeah, there's some chance these AI systems are conscious. I already think there's some chance these AI systems are conscious. And so taking the issue seriously is to say, okay, what ethically ought we to do when we're in this position of uncertainty? Where one view you might have is like, well, if you're unsure, then just don't worry about it. Just assume they're not. This is how the medical establishment used to act with respect to newborn babies. In fact, until, remarkably recently, only a few decades ago, newborn babies and infants would get surgery without anesthesia because it was the widespread understanding was that they don't feel pain. I think that's a kind of crazy thing to become with the widespread understanding, but it's also agreed that it's like, no, that's false actually. And so, yeah, I think the ethically responsible thing to do is to be firstly debating as a society, if we are creating a new, like a wholly new sort of being, how should that being fit in to our current society? Should it be that they get treated like other software and get owned by the companies that create them? Is that correct? Or should they have economic rates? Should they have welfare rates? That is a debate society should be having, and it is not currently having it. And then, I think there should also be an onus and just then understand. Yeah, if the AI's do have interests like what So they care about what would it mean from their perspective to treat them well and without kind of anthropomorphizing them or making assumptions that would be similar in any way to how humans are treated? Well, that's a question. A kind of summing up question. I go back to 13 years ago when we first had those late-night scotches and talked about effective altruism in its nascent early period. And that was a philosophy that was about using money to help people and using time to help people. And when I think about the evolution of the philosophy over the last 13 years, I think about the fact that animal welfare has clearly been pulled into the orbit of effective altruism, shrimp welfare, even claims of insect supremacy. We're talking about AGI now and AI moral rights. How do you feel about the argument or I should say, the criticism that one way to describe the trajectory of effective altruism has been the long de-centering of the human being? That as the philosophy has found more and more ways to extend itself over the course of all things to care about, it has found more and more things to care about that aren't people. Is that a fair description of effective altruism? And if it is, is it one that you welcome, maybe I could hear an argument that says the correct frontier of philosophical entrepreneurship should be moving away from a human centric theory of moral good toward thinking about not just the people that exist, but the people that might exist, not just humans, but also insects and shrimp and also AI. That's a good thing, one could say. So I guess two part final question, one, do you agree with this idea that a history of EA has been the de-centering of the human and if so is that moral progress? Yeah, I think on the first question, I think it's largely not the case, like effective altruism has grown as a whole. That means the work that gets done to benefit present day humans has grown enormously, you know, a fact of a thousand or something or hundreds compared to the early days, such that, you know, now is kind of tens, hundreds of thousands of people per year who are getting prevented, who are whose lives are being saved via things like anti-malarial bednets and so on. Similarly, the work on global catastrophic risks, I don't think at all you need to care, but you could care about people past the year 2100, not a job, and it would still come out as among the very most cost effective things you could do in my view, because the risks are so incredibly neglected. And so what has happened? Well, there's a lot more discussion of that, like avant-garde stuff, because the effective altruism is kind of more in the public eye than it was, but again, I think that's just thinking that that's kind of what the effective altruists now all focus on is just kind of misrepresenting the movement. However, I do think that's what we should be talking about. I think the idea that like only human beings have any sort of moral status is a view that's just going to look quite bad from the perspective of people looking back at us in the future. In much of the same way as the idea that like, oh, well, it's only people in the UK, only people in the US matter, and the interests of someone in Mali, they're not, you know, they don't count. I mean, I think when you ask people, you say like, well, imagine it's 2200, and future generations are looking back at us and thinking, oh, what are the moral atrocities that we were committing now in the same way that we look at moral atrocities of the past. Many, many people, maybe most people say, oh, well, it's going to be factually farming, isn't it? And it's like, yeah, it's like huge amounts of unnecessary suffering that we inflict on animals for very little benefit to ourselves. And so the thought of just, okay, well, what are the other ways in which we could be engaging in similar moral atrocities really ought to be thinking about that. And just coming in with this kind of blinkered, well, it's not about present day humans, or, you know, not just about present day Americans or bits or something. I think is going to look really quite now reminded from the perspective of moral progress, from the perspective of history. We want my cask on. Thank you very much. Thank you so much. [MUSIC]