Speaker 1what i find really interesting is to look at how people's quotes evolve over time with their incentives so i was looking at all of the all of the things they've said on the record on podcasts in their blog posts to see how it's evolved over time and dario who was the former vp of research open ai and has now moved on to entropic who are taking a slightly different approach to developing ai said back in 2017 while he was still at open ai that this is a quote i think at the extreme end is the nick bostrom style of fear that an agi could destroy humanity i can't see any reason in principle why that couldn't happen my chance that something goes really quite catastrophically wrong on the scale of human civilization might be somewhere between 10 and 25 and also you mentioned ilia who was a co-founder of open ai and then left i guess the first question i'd ask is why did ilia
Speaker 2so he was instrumental in trying to get sam altman fired and he's another one of the people who over time began to feel like he was being manipulated by altman towards contributing something that he didn't believe in and for you know because i interviewed a lot of people ilia in particular had two pillars that he cared about deeply one is making sure we get to so-called agi and the other is making sure that we get to it safely and he felt that altman was actively undermining both things he felt that altman was creating a very chaotic environment within the company where he was pitting teams against each other where he was telling different things to different people have
Speaker 1you ever spoken to
Speaker 2him i have so so i interviewed him in 2019 for a profile that i did of open ai um for mit technology review and back in 2019 he
Speaker 1has a quote where he said i'm not going to the future is going to be good for ais regardless it would be nice if it was also good for humans as well it's not that it's going to actively hate humans or want to harm them but it's just going to be so powerful and i think a good analogy would be the way that humans treat animals it's not that we hate animals i think humans love animals and i have a lot of affection for them but when the time comes to build a highway between two cities we are not asking the animals for permission we just do it because it's important to us i think by default that's the kind of relationship that's going to be built between humans and ai which are truly autonomous and operating on their own behalf and that was in 2019 the year that you interviewed him
Speaker 2one of the things that i i feel like we should take a step back to examine is going back to this idea of what even is artificial intelligence and what do we mean by intelligence and a huge part of the views of the different people on the quotes that you're reading derives from a specific belief that they each have in this question of what is intelligence for ilia he has throughout his research career felt that ultimately our brains are giant statistical models this is not something that you know we actually know but this is his own hypothesis also the hypothesis of his mentor jeffrey hinton who also was on this podcast this is why they have such a strong conviction in the idea of building ai systems that are statistical and that this particular approach is going to lead to intelligent systems as we are intelligent it's a hypothesis that they have it's not one that has been proven by science and some people vehemently disagree with them on this particular thing but if you step into their shoes and take on that hypothesis and assume that it's true that our brains are in fact statistical engines and that these systems that they're building are also statistical engines that they're making bigger and bigger and bigger until they become the size of the human brain that's why they say that making this comparison where the system will become equal to human intelligence and then maybe exceed human intelligence is relevant in their framework and ilia gave a talk at one point at this really prominent ai research conference that happens every year called neural information processing systems it's mouthful but he gave this keynote where he shows this chart of the size of brains and the intelligence of a species and it's roughly linear the bigger the size of the brain the more intelligent the species and so for him he thinks he's building a digital brain because he he thinks brains are just statistical engines so from that logic it's like okay if we then build a bigger statistical engine than the human brain then we're going to be able to build a bigger statistical engine than the human brain then based on this chart it will be more intelligent and then we will be subjected to the same treatment that we've subjected animals but it's really important to understand that these are scientific hypotheses of specific individuals within the ai research community and there's a lot a lot of debate about whether this is in fact the case and some of the biggest critics say it's very reductive to think of our brains as simply just statistical engines
Speaker 1why does it matter to know the mechanism is it not just important to know the outcome which is that it's going to be able to do make a video for me or agents are going to be able to do the work that i do does it does it really really matter for us to know the mechanism behind it yes and no
Speaker 2so it matters because these companies they are driving their future actions based on this hypothesis so they have decided we think that this hypothesis is true like we should just continue building larger and larger statistical models in the pursuit of artificial general intelligence and that's then having global consequences like in order to continue doing that they're hoovering up more and more data they're building more and more data centers they are having uh they're you know exploiting more and more labor in order to continue on this path here's a question what do you think about this hypothesis and what do you think about the the question that i think is important to ask is why are we trying to build ai systems that are duplicative of humans we're kind of having this conversation right now where we've just taken the premise of this industry as a good thing like they said that we should be building agi so we say that we should be building agi but i would like to ask like why are we doing that why is it that we are building a technology that is ultimately designed to replace and automate people away that is not the enterprise of technology like we should be building technology and the purpose of technology throughout history has been to improve human flourishing not to replace people and so this is like a critical part of my critique of these companies and and these scientists that have just adopted this goal and have relentlessly pursued it and have had enormous capital and enormous resources to pursue it is is this the right goal is this the right goal is this the goal what like why are we doing this why can't we just build ai systems that do things like accelerate drug discovery and improve people's health care outcomes which are systems that have nothing to do with the statistical engines that they're trying to build to duplicate the human
Speaker 1brain so why are they doing it i mean you've interviewed all these people i think it's what 300 people in total 80 or 90 of them from open ai the maker of chat gbt why
Speaker 2do you think they're buying imperial agenda and that is why i call these companies empires of ai what do you mean
Speaker 1by an imperial agenda what does that term
Speaker 2mean empire is the only metaphor that i've ever found to fully encapsulate all of the dimensions of what these companies do and the scale that they operate and what motivates them to do what they do and there are many parallels that you see between what i call the empires of ai and the empires of old they lay claim to resources that are not their own in the pursuit of training these models that's the data of individuals the intellectual property of artists writers and creators they're land grabbing in order to build these supercomputer facilities for training the next generation models second they exploit an extraordinary amount of labor they contract hundreds of thousands of workers all around the world including in the u.s to ultimately make these technologies we can talk about that more and they also design their tools to be labor automated so that when the technologies are deployed it also affects labor rights because it erodes away labor rights and this is a political choice that they have third they monopolize knowledge production so they project this idea that they're the only ones that really understand how the technology works and so if the public doesn't like it it's because they don't actually know enough about this technology they do this to the public they do this to policymakers and they've also captured the majority of the scientists that are working on understanding the limitations and capabilities of ai you
Speaker 1think they're gaslighting the public in a
Speaker 2way they are yeah so if most of the climate scientists in the world were bankrolled by fossil fuel companies do you think we would get an accurate picture of the climate crisis no and in the same way they employ and bankroll the ai industry employs and bankrolls most of the ai researchers in the world so they set the agenda on ai research in soft ways simply by funneling money to to their priorities so that only certain types of ai research are produced but they also will censor researchers when they do not like what the researcher has found. And so I talk about the case of Dr. Timnit Gebru in my book, who was the ethical AI team co-lead at Google. When she was literally hired to critique the types of AI systems that Google was building, she then co-wrote a critical research paper that was showing how large language models specifically were leading to certain types of harmful outcomes. And in an attempt to try and stop this research from being published, Google ended up firing Gebru and then fired her other co-lead, Margaret Mitchell. And so they control and quash the research that is inconvenient to the empire's agenda.
Speaker 1Did you have an example where this is happening to journalists as well, that are asking questions of their team members? I think I was watching a video of yours where there was a young man that was saying he had someone show up at his door, knocked on his door and asked for information, emails, text messages. And this person was from one of the big AI companies.
Speaker 2This was opening, I started subpoenaing some of its critics. Yeah. As part of a, what appears to be a campaign of intimidation, but also what appeared to be a campaign of fishing for more information to figure out, to map out the network of critics, further. But this was a man who runs a small watchdog nonprofit. And they had been doing a lot of work during that time to try and ask questions about OpenAI's attempt to convert from a nonprofit to a for-profit. Ultimately, OpenAI was successful in that conversion. But during the period where it was sort of existential for OpenAI to complete this conversion, there were a lot of civil society groups and watchdog groups like Midas who were, you know, they were, you know, were trying to prevent the process from happening in the dead of night. They were trying to get more transparency. They were trying to have more public debate about this because it's unprecedented. And it was then that there was a knock on his door and he was served papers.
Speaker 1What did the papers say?
Speaker 2The papers asked him to reproduce every single piece of communication that he had had that might've involved Musk. So this was like that opening I had that Musk was somehow funding these people to block the conversion. None of them were actually funded by Musk. So in this particular case, their request, he simply was just answered, you know, I don't have any documents because this doesn't exist.
Speaker 1So going back to this point of empires, you were saying that one of the factors of an empire is a land grab. And then the next one was labor exploitation.
Speaker 2Labor exploitation. The third one, controlling knowledge production. And one of the other ones that's really important to understand about the AI empires in particular is empires always have this narrative that they, they say to the public, like we're the good empire and we need to be an empire in the first place because there are also bad empires in the world. And if you allow us to take all the resources and use all of the labor, then we promise we will bring you progress and modernity for everyone. We will bring you to this utopic state akin to an AI heaven. But if the evil empire does it first, we will descend into a hell.
Speaker 1And the evil empire being in this case?
Speaker 2In this case, most often it's China, but actually in the early days, open AI evoked Google as the evil empire. So all of their decisions were about, we need to do it first because otherwise Google, this evil corporation that's driven by profit, us as a benevolent nonprofit, like this is a, this is a critical contest of who wins.
Speaker 1Do you think the people building these AI companies believe that the outcome is going to be all good now? Do you think they think that it's going to be, it's going to serve everyone? It's going to be the age of abundance. Everything's going to go well. What do you think they believe?
Speaker 2So this is so funny is such a core part of the mythology that they create around the industry. It includes the belief that it could go very badly. It goes hand in hand. Like they need that part of the myth in order to then say, and that's why we need to be in control of the technology because that's the only way that it's going to go really, really well. And Altman has said publicly, you know, the worst case lights out for everyone, but best case, we cure cancer, we solve climate change, and there's abundance. And Dario Amadei, same kind of rhetoric. He's like, worst case, catastrophic or existential harm for humanity. Best case, mass human flourishing. So this is like two sides of the same coin. Like they have to use both of these narratives in order to continue justifying an extremely anti-democratic approach to AI development where there should not be broad participation in developing this technology. They must be the ones controlling it at every step of the way.
Speaker 1Sam Altman did a tweet, saying, there are some books coming out about open AI in me. We only participated in two of them. One by Kesh Hagee. Keech Hagee. Keech Hagee focused on me. And one by Ashley Vance on open AI. He went on to say, no book will get everything right, especially when some people are so intent on twisting things. But these two authors are trying to. You quote, retweeted that tweet from Sam Altman. And you said, the unnamed book, Empire, of AI is mine. Do you believe that tweet from Sam Altman was in reference to your book?
Speaker 2A hundred percent. Because there's only three books coming out about him.
Speaker 1And he'd caught wind that your book was coming out and.
Speaker 2He knew my book was coming out because I had contacted open AI from the very beginning of my process and said, I'm working on a book now. Will you participate in it? And actually, initially they said yes, even though. So my history with open AI, I profiled the company for MIT Technology Review. I embedded within. The office for three days in 2019. My profile comes out in 2020. The leadership are very unhappy. And in my book, I actually quote an email that I received that Sam Altman sent to the company about my profile saying, yeah, this is not great. And from then on, the company's stance to me was, we are not going to participate in anything that you do. We are not going to respond to anything, any questions that you receive. And this was, you know, this was things that they explicitly articulated. It wasn't like me inferring. So I had a colleague at MIT Technology Review that also covered AI. And at one point opening, I sent him this press release being like, we would love for you to cover this story. And he was like, I'm really busy. Will you send it to Karen? And they were like, oh no, we have a history. You understand. And so, so for. For three years, they refused to talk to me. But then I ended up at the Wall Street Journal, where if they felt a bit compelled, because it was the journal, to reopen the lines of communication. And so I started having, you know, more dialogue with them. Every time I wrote a piece, I would always send them, here's my request for comment. I would always ask them like, will you sit for interviews? And we did get to a more productive relationship. And then I embarked on the book. So I left the journal. And I was like, I'm going to do this. I'm going to do this. I'm going to write a journal to focus on the book full time. And I told them right away, I'm working on this book. I want to continue this productive conversation where I make sure I reflect OpenAI's perspective in the book. And so they were like, we can arrange interviews for you. You can come back to the office. We'll set up some conversations. And then as we were going back and forth on this, the board fires Sam Altman. And that's when things started going kind of south because the company started becoming very sensitive to scrutiny. And so then they started pushing, kicking the can down the road, down the road, down the road. And I kept saying, hey, when are we rescheduling this? What's going on? And then I get an email saying, we are not going to participate at all. You are not coming to the office. You're not doing interviews. And I had actually already booked my tickets. So I was already going to fly to San Francisco to have the interviews. And so then, I told them, I was like, that's fine. I will still engage in the process where I'll give you extensive requests for comment. I'll ask through my reporting, I'll keep you updated on all the things that I'm finding so that you can choose to still comment. I gave them 40 pages of requests for comment. And I gave them over a month to respond to all of that. So this was when the tweet came out was we were doing all this back and forth trying to, and that's when Altman, tweeted this. And they never responded to a single one of the, one of the 40 pages.
Speaker 1Sam Altman does a lot of interviews. Yeah. You know, he's doing a lot of interviews all the time. He's done every podcast. I've seen him on everything from Tucker Carlson to, I think he's done Theo Von, Joe Rogan, podcasts all over the world. I wonder why he won't do mine. Well, maybe. I don't know why, I don't know. I think I'm fair with everyone. I just ask, I just ask questions I genuinely care about. I don't come in with huge preconceptions. preconceptions or at least meet people for the first time but i've heard through the grapevine um that he doesn't want to do mine i mean going
Speaker 2back to what you were saying earlier that with this the way that opening eye and these companies control research you asked do they also do this with journalists i mean yes the answer is yes and apparently they they also do it with anyone who has you know a broad mass communications platform it's not just about the conversation that you're going to have with them it's about who you also choose to platform and there's this huge problem in technology journalism where companies know that a really big carrot that they can give to technology journalists is access yeah yeah and they will withhold that access at the drop of a hat if they catch wind that you're speaking to someone that they didn't want you to speak to this is
Speaker 1so true and i don't think the average person really truly understands this yeah so this kind of sounds like theory as you say it but i'm not going to name name names here because i don't think it's important but there is a particular person in ai who um whose team have basically dangled the carrot of them coming here for like 18 months and i'm like you don't you don't have to dangle the carrot i'm going to speak to whoever i want to regardless of the carrot or not yeah and when this person comes if they want to come i'll i'll give them a fair shot i'll ask them all genuinely curious questions about what they're doing their incentives i won't gotcha them i don't have a history of ever gotchering anybody even if i just like even if i have a different of opinion i'll ask the question yeah but they dangle carrots and they say well if you know he he's thinking about it let's think about a date and what the what the strategy is and i don't think they they think most people don't understand is if we just dangle it for long enough then they will um perform in the way that we want them to do and they'll be they'll be pleasant about us they won't be critical they give a give a they won't platform our critics our critics and i think a lot of their game is just dangle the carrot forever yes yeah that's like the optimal outcome is like we just dangle it if we just tell them yeah no we're just trying looking at the schedule it just doesn't work i think in the modern world you just have to go there and give your opinion and allow the clash of ideas in the public forum let the viewers decide for themselves yeah what they think yeah um but this
Speaker 2is a yeah this is such a huge part of their machinery is the way that they use these tactics to really massage the public image of these companies and make sure that information that they don't want out and even opinions that they don't want out there go out there and so this is this is you know i feel very lucky now that opening i shut the door early on me at the time i didn't feel lucky i felt like i had screwed myself over i was like should i have been nicer to them in the profile so that i could maintain access which is a horrible question to ask as a journalist right the truth and you're always supposed to report in the interest of the public like that is the point of journalism and in that moment it i was like relatively junior in my career i was like did i misunderstand what journalism about is is about like should i have actually been playing the access game but it was too late i had the door shut to me and so i had to build my career understanding that the door the front door was never going to be open yeah and that actually strengthened my own ability to just tell it like it is like yeah and just report what i see are the facts being presented to me irrespective of whether the company likes it or not and most often the company really does not like it but i can continue to do the work they don't need to open the front door for me i was still able to do more than 300
Speaker 1interviews what you just listened to was a most obvious episode if you want to listen to that full episode i've linked it down below check the description thank you
PodcastTranscript AI