Generative AI is Not Inevitable w/ Emily M. Bender and Alex Hanna

there's this inevitability narrative that we hear from the tech companies all the time and that is a bid to steal our agency and so you know Alex said it can be inspiring to take a principle stand and one part of that is to say no I don't believe you if you say well this is here we have to learn to live with it no right there's nothing inevitable either in the development of something called A.I.R.A.G.I or in the sort of continued presence of this tech in our lives. Hello and welcome to Tech Won't Save Us. I'm your host Paris Marx and this week I have two fantastic guests joining me on the show Alex Hanna is director of research at the Distributed A.I. Institute and Emily M. Bender is a professor in the department of linguistics at the University of Washington. They have a new book out called The AI Con how to fight big tech hype and create the future we want sounds like exactly the type of thing that we'd like to talk about on this show. Alex and Emily have been on the show in the past of course you know Alex for the series that we did called data vampires looking at data centers and things like that and Emily to talk to us about A.I. and her input on that as a computational linguist their new book is a great primer for people who are looking to understand what is actually going on with the A.I. hype that we've been seeing over the past few years all of the problems with it and what the actual goals of these people are and importantly they also look at how we can push back against these things right both as individuals but also in collective ways through government and through other means and I wanted to make that a big part of our discussion but we started off by talking about you know some of the bigger ideas of the book around A.I. hype but also a number of the things that we have been seeing occurring at the moment that I wanted to get their input on as people who have been paying such close attention to everything going on with A.I. not just for the past couple of years but even before the hype around chat GPT and generative A.I. and the types of things that have been motivating the tech industry and investment since 2022. I have to be honest I really don't think that there is much more context that I need to give you for this episode it's just a really great conversation about where we stand right now with A.I. and how we might want to push back against the things that we're seeing in order to limit the harms that we're seeing come from it but also the broader futures that these tech billionaires are trying to bring into being. So with that said if you do enjoy this conversation make sure to leave a five star review on your podcast platform of choice you can share the show on social media or with any friends or colleagues so you think would learn from it and if you do want to support the work that goes into making tech won't save us every single week so we can keep having these critical in-depth conversations on important issues not just affecting tech but affecting all of us you can join supporters like Maddie from Toronto Alex from Edmonton Vladimir from Brazil Brian from Houston Abe from Grand Rapids Violet from Pennsylvania Matt from Maryland and Jan from Prague by going to patreon.com slash tech won't save us where you can become a supporter as well thanks so much and enjoy this week's conversation Emily welcome back to tech won't save us delighted to be here again thank you and Alex is great to have you on the main show not just the you know one of our series as you were last time you were on the extra evidence prayers absolutely you know I really enjoyed the new book the AI con I feel like if you have someone in your life who was like what do I need to know about AI like what is the problem with this this is like the perfect book to be like pick this up read it it has all the things you need to know in there it's really concise but also still has all the detail that you need to actually understand what is going on with this industry and why you should be concerned about this products and and you know what these CEOs and things are are actually doing here so I really applaud you on the book and so my first question was really like how did you decide that you wanted to put this together how did you decide we need to write this book about AI to really start to dispel these myths that that are so persistent oh let's say it actually starts with our podcast so we run mystery AI hype theater 3000 which I'm here's an accidental podcast if you listen to the first couple episodes you'll tell that we did not know we were making a podcast and you know we're doing the work in that podcast of trying to top the bubble of AI hype and it's a lot of fun we still do it it's roughly twice a month and we do different topics each time but it started feeling like we could reach a larger audience if we could construct the argument and put it in one sort of neat package yeah and so we were we kicking the idea of a book around and got approached by an agent and we're thinking well yeah we could really think about a way to tell very coherent narrative and I think one of the things that we appreciate you naming that it's pretty accessible because the idea is let's start kind of from zero or start from a place of I've heard about this thing called AI and really explaining what these technologies are explaining that AI is not one thing I'm in trying to justify everything that I just have sparkles in front of it yeah I was really mad when they started stealing the sparkle emoji and making it seem like it was AI I know it's a fun emoji right it's a great emoji whenever I see something that has and now I have like a you know a moment it's like a jump scare I'm like yeah no but I want it to be good absolutely so the book I feel like one of the big focuses in the book is AI hype right and you know the the effect that AI hype has on our understanding of this how it's able to be sold to the wider public in a way that hides some of the harms that come of this technology so why why focus on that so much and how do you assess the state of AI hype in May of 2025 when we're speaking and when your book is coming out when we finish the manuscript in September of 2024 there was this like urgency it's like we really wish that we could have had the book on the shelves right when we finish writing it I'm sure all authors do like I'm done how come nobody can read it right and I remember saying it will be a better world if this book is not needed in May of 2025 and unfortunately we are not in that better world right the hype continues and why hype I think part of it is that it really draws on our expertise so I'm a linguist Alex the sociologist and we can look at how people talk about things how they sell things how the large language models play on our linguistic competence as people and sort of open up a possibility for hype sociologically what does that do in the world I think it gives us an interesting angle in that we don't see represented so much in there's other fantastic books that people should be reading about you know the AI industry and the harms that are being done but I think hype was an open space we could move into yeah I would say that one of the kind of handles a hype from a sociological perspective is that it gives you a sense to talk about the political economy of the AI industry it gives you a sense of talking about well these are the promises of the industry and here the institutions that are surrounding it as well as the massive amounts of investment going into it right it's kind of fascinating to here in May in 2025 I had just re-skitted or quote-skitted this study that IBM had done of CEOs and there was some really flooring statistics and study the one thing I posted to blue sky on it I mean there's there's many different parts of it which are flooring but the one that I quote-skitted was surveyed CEOs report only 25% of AI initiatives had delivered expected return on investment over the last few years and only 16% have scaled enterprise-wide so and then the really flooring part of that if you go further down is basically I think something like around 68% of those firms still plan to go full steam ahead so we're still within the the era of AI hype and unfortunately or maybe fortunately for us unfortunately for the world the book is still very relevant it's always wild to see those stats right like it's pretty clear it's not paying off but we're still going to keep pursuing these sorts of things I remember there was one last year where they had surveyed a bunch of employees and a bunch of like management or like see sweet people or whatever and the employees overwhelmingly were like this makes me do more work like it's not very efficient it's not working very well and like most of the managers were like it's great that we're implementing these things it's very effective like there's this like total disconnect there right but I want to double down on that second part of my question because I feel like there have been multiple moments even like you're talking about you know end of last summer and whatnot where we we're talking about oh is the the bubble finally going to burst is the hype finally exhausted are we finally waking up to what is going on here and you know it's going to start collapsing and we'll see what kind of comes on the other side of it but it feels like again and again it has managed to continue limping on so I wonder how you assess the the moment that we're in right now in terms of AI the support that exists for it how it is continuing to to maintain that interest in that investment yeah I mean it's it's interesting because I think what it is is it's it kind of signals that this is a bubble not a bubble shaped thing you know and it makes me think about well what's going to happen here and what happens after AI and you know we spend a little bit of time at the end of the book thinking about when the bubble bursts where however burst because it might burst slow it might burst fast it might have these people you know tossing the good money after bad which then becomes bad money and the way in which you know what is left behind that and that's the kind of data center growth and energy and waste that's been generated by that the carbon that's put into the atmosphere as well as the air other air pollution and small particulates there's the jobs that have been completely decimated the fields that have been decimated the thing is like I don't really know I mean it's hard to sit here from our chairs and try to play you know industry analyst but it's like what kind of shape does this look like and I mean I think there's either probably going to be one of two ways maybe one of four ways I'm not sure what the second axis is see that the Uber model which is well we're just going to keep on tossing money at this and at least one day this is going to turn an investment and that's after a lot of capital has been destroyed but there's going to be some kind of sustainable path where it's I don't know you have a few data centers and that's going to help students cheat on homework or whatever or it'll be the big explosive kind of way which maybe less plausible but maybe there's something that really disrupts it and there's a real upper has no close sort of moment but it's hard to speculate at this point and you ask like why is it still lumping along why why was last summer not the end of the bubble and I'm again not an industry analyst but from where I sit it looks like it's a question of just how much money is there already and maybe something of a sunk cost fallacy on the part of the venture capitalists and the promise and it comes back to the hype right the hype says if we can do this thing if we can actually build the thing we say we're going to build it is going to be infinitely valuable and so somebody whose job it is to maximize profit who believes that hype like why would they not invest I guess. No it's it's a really good point and Alex you know you mentioned what happens after the bubble burst I want to come back to that a little bit later while we have a discussion about some of the stuff you're discussing at the end of the book but there are a few other things I want to dig into before that obviously this this boosterism is very much closely associated with the hype that you're talking about here right and I feel like again and again unfortunately we see this boosterism not just coming from people at the top of these companies and venture capital firms who you would expect to be boosters of this technology right they're hoping to make a lot of money from it and to kind of spread it into the world but we also often see that from a lot of media right and and a lot of reporting on this what role does that play in helping to maintain this idea that AI is the next big thing that it is transforming the world in the way that some of these tech folks want us to believe and how does it distort the conversation in the public understanding. I think every time a journalist uses the phrase artificial intelligence uncritically like they say oh there's a new AI powered tool that connects YZ that misinforms the public and I think about for example teachers who are now faced with lots of pressure to bring AI into the classrooms so that their students are trained for the workplace of the future right is if that's what education is about and that's another sad topic but I think that if we had more critical journalism and there is you know wonderful journalism constantly pointing people to your podcast Paris and really appreciate the work at 404 media and people like Karen how who's fantastic book is coming out so there is wonderful critical tech journalism but there's also a lot of credulous access journalism and one of the things that I hope we can do with this book is help people sort of navigate to the critical journalism and know what they're looking for so that they can continue to get good information from the people who are asking the top questions yeah and I mean the the credulity comes a little bit and I'm I'm be interested in in a bit of a history of tech journalism but so much of tech journalism has been all about access or has been all about just oh look at this fun whizz-bang thing and so you have you know so unfortunately you know you have your folks like Kira switcher who was like Elon Musk was great until he lost his mind I'm like yeah he was great until he called me an asshole and now he's not great it stopped returning my calls exactly yeah like that man lost his mind a while ago or you know was died in the wool someone that you shouldn't trust and you've had a number of other folks and we talked I think before we got into call about Kevin Ruse and Casey Newton and Kevin Ruse has been this person that I think has really been brought in to the kind of he's been kind of dumbfounded by all these different kinds of the outputs of chatbots you know and I think there was at one point where you had this conversation with what was the Bing chatbots Sydney and then at some point at it he had said it had tried to get him to leave his wife and I don't think he's recovered from that because I guess everything from that has really been kind of flooring and I mean it's been this kind of thing that it's just granting the tech billionaires so much more grace than you would grant any kind of area of research that you didn't know anything about like how would you explain this I mean there's a lot of hand waving that's going around and you know we saw this interview with the CBS show they did with Demis Hassabis Hassibis and the way I was reading the transcript and they started off with you know you know he grew up and he was a test prodigy at 12 and you know and then they had this interview where he you know said it was going to cure cancer and cure all illness in 10 years and you can't let people just get away with saying this complete batshit like that right yeah it's just like wow oh my god the future is going to be so incredible you don't have to follow through on it but it sounds so great right but no I think that those are all really important points right because the public really relies on on media in many senses obviously we've seen this erosion in trust with some media over the past number of years but to a large degree people do still look to media and journalism to get an understanding of what is going on in the world and when they approach this and get these takes on AI and technology that are overwhelmingly booster-ish that are overwhelmingly repeating the press statements of these tech companies that doesn't actually help the public understand what is happening but just tells them what the companies want them to believe and that's not how you have like an informed discussion as you're trying to promote an encourage us to have in this book right I feel like another piece of this right we look at the journalism piece but another piece is how these discourses play out within the tech industry themselves and in the book you talk about the tumors and the boosters we've talked about the boosters but you know there's these tumors as well who promote this notion that AI is going to be so powerful and is a big threat to us can you talk to us about how these narratives that in some senses might seem to be in conflict with one another actually work together to promote this notion that AI is is huge and powerful and going to change our lives in all these ways so in a nutshell the AI boosters say AI is a thing it is imminent and it is going to be all powerful and solve all of our problems and the AI tumors say AI is a thing it is imminent it's going to be all powerful and it's going to kill us all and saying it that way you can see very quickly that it's actually the same position and it's just what turns you take at the end and the really frustrating thing is that the booster versus tumor discourse sets it up as these are the full range of possibilities you're at one end the other or somewhere in between that is it and coming back to journalism what are the journalism sets it up that way too and what we say I have the same for a long time and say in the book is no that's actually two sides of the same coin it is one small part of the range of possible ways of seeing things and it is not a reality-based part of it yeah and so we talk on the book a lot about these different I mean we don't we don't go into the different kind of flavors of duorism we kind of gesture to the test real bundle the transmunist extra pianists singularness cosmism okay now we're to the real rationalism effective altruism in long terms I mean you've had to meet on the on the program a few times and respect for having to memorize so well yeah it's it's hard it doesn't exactly roll off the tongue would actually spell at the acronym I finally figured out what extra pianism means actually from Adam Beckett's book more everything forever it's extra pee as the opposite of entropy oh no learning things every day here I read Adam's book on my big pile of books soon and so you know we talk about getting into that and you know the different flavors and sort of you know the kind of thing that's interesting is that you have Yudowski as kind of this person who is like kind of this true believer this person that is this I somebody on blue sky called him big you'd and I thought that's kind of great so you have this kind of towering figure the people who are like really getting high on their own supply then you have people who also have the kind of mix incentives you know like Elon Musk who was you know a long-termist says he's a long-termist but also you know is of course is trying to make money and you know has status and power aspirations too so there's kind of a mix you know it's all the same effect kind of at the end whether you're kind of a strong long-termist or a weak long-termist but it does lead to this kind of dualism and I don't know lies the same bottom line that AI is inevitable it's a thing we need to believe in it's a thing that we should put our trust in and it's just a matter of aligning it correctly and I feel like one of the ways that this becomes really concerning which of course you you wrote about in the book is when you see this kind of booster doomer dichotomy or relationship then be the one that defines discussions in policymaking right whether it's in the US government or or other parts of the world where all of a sudden that is the frame through which the conversation is happening and you know like you discuss it it very effectively shapes that conversation in in a favorable way yeah 100% I mean we saw this we started the book with anecdote about the shivers insight for and especially focusing on the ninth one and what he asked you know what the what is the p-dume and what is the p-hope p-dume is the probability of doom and p-hope is the probability of hope so p parenthesis doom and parenthesis i'm sure this is language that chuck shumer is using all the time in his everyday conversations yeah chuck shumer is he's just walking down the halls of the the dorkson office building in dc and asking hey bro what's your p-dume and so you know this this got some reporting to there was like very credulous reporting in like the sf standard or whatever saying the hottest question around silicon valley is like what is your p-dume and oh my god yeah but I mean anyways on the policy element I mean that became a frame through which to to have conversations during that insight form and then the output of all those insight form were that the whole thing focused on innovation or national security and whatnot and had very little bit to say about present day harms yeah it's a real concern right and and when you think about present day harms I feel like this is something that we are increasingly talking about it's something that all of us talk about a lot when it comes to AI but I feel like it's slowly making its way more into the general conversation and I wanted to pick up on a story that was published just recently in New York magazine talking about how generative AI is increasingly kind of moving into education you know being used by students and educators and even administration and I wanted to ask because you've both been involved in in these spaces for a while what you make of how generative AI has moved into education in the past couple of years and the effects that it has had there so I got to speak at the pack 12 deans conference last spring back when there was a pack 12 so this is deans from universities up and down the west coast I think we not just the us we had some folks from Canada and I was brought in to talk about AI and I got to say to these deans look the reason this is your problem now is not that there's been some enormous breakthrough in the technology but rather there's a lot of money and people are trying to push it on you because they want to start recouping some of their investments like that's that's all this is and the second thing I said to them is the only use of something like chat GPT at a university is as a contrast die test to understand where more resources are needed when a student is turned to chat GPT it is either the instructor has not motivated the importance of doing the homework and so the why bother or that student doesn't have enough resources they haven't had enough background to do the assignment they don't have enough time because they're also a working full time or whatever it is but it's basically and if you've got staff turned to chat GPT again there's some lack of resources there and that is sort of the only useful use case from a university's point of view in my mind we talk a lot to K-12 educators who see we had a wonderful episode of our podcast with Adrienne Williams on talking about ed tech and you know she has a long history in the classroom watching how basically the stuff just getting pushed on students and the students of the wealthiest families in the US at least not sure what's going up in in Canada are in schools with no screens in sight right people who really have resources who want the best as we all do for our kids are staring far clear of this and just in general i think that anytime someone is using a chat bot in the context of any educational assignment it's a miss learning opportunity i mean universities have really fond flat in the response of this either they have made deals with the companies or they have i think they mentioned in that out near magazine article like you know they they've sort of said okay maybe not but like you decide but they haven't really provided any strong guidance for for faculty and it's been sort of a missed opportunity and so in the worst cases of it it's it's kind of like well you as a professor should basically just throw up your hands into clear bankruptcy and you just have to grade this as if someone had written it and that is kind of the worst thing to do because now you're putting the onus on the professor to try to think about this and try to think about what it is in the context of their pedagogy there's also been turns to become just intensely carceral and just turn all instructors in the cops and saying that you need to police this really heavily and both of those are terrible strategies and i think that there really needs to be strong leadership from you know universities but in where universities haven't pushed back some you leadership has come from some education unions where they're like okay this is where you can we can support our members and support or release supportive learning environment and so for instance in california here cal state the cal state system had one of the largest contracts of cal state has i think something like half a million students across all their campuses largest system in the us and they ended up with a sixteen million dollar deal that they had with open AI for just a site license for a year not even beyond that so huge way some money cal state like every system in a budget crisis and the california faculty association said okay slow down no we're not actually gonna have this we need to push back because we're not why are we giving all our money why are we reinvesting in kind of pedagogy and thinking about what it would mean to really invest in our instructors and our students i want to add another worst case scenario which we've seen reported a few times is especially in hybrid classrooms like i teach where you've got some students in person in some online we've seen a couple of universities proudly announcing that they have some a i students joining the class yeah Jesus and the reporting on that is usually appalling right just sort of talking about them as students instead of text processing surveillance machines being added to classes other cases where you get faculty sort of proudly training chat butts on their own works and then putting those chat butts up as if they could be teaching assistants for the students yeah lots lots of terrible ideas yeah really i i hadn't even heard of those ones that's yeah that's terrible when i was reading that story and as i've been seeing stories about this come out more and more like one of my concerns is really around critical thinking right and what this actually does to the broader society and and it's also not to say that this is a brand new thing with a i or generative a i but it feels more like the continuation of a process of reducing education reducing critical thought within society more broadly and this feels like it comes after like a long list of developments over many decades that have further kind of degraded that and it just seems like a very kind of scary indication of where things are going yeah for sure i mean as a sociologist my tendency in this book was to historicize and so just taking the added of many social sciences always be historicizing and so the the onslaughts of LLMs in the classroom has really been part of this process of the increasing casualization of the university the increasing digitization of the university the increasing surveillance nature of the university so in the cases where the university you know at one point at least in the US where we have this private public system whereas you know i remember speaking to someone at a student power conference once and they're from Montreal and we just explained the notion of a private university to them and they were like someone could just start a university and it just you know it is a thing that boggles the the Quebec law mine yeah in Canada generally like the notion of private universities are pretty i don't think there are many up here at all like it's mainly public yeah yeah it's it's it's it's completely outside of what is what is considered to be a kind of a legitimate institution and so in this place where you know you had private investment universities where many flagship universities now have 8% or less investment in funds actually coming from the state where most of the funds are coming from donors or tuition or xyz other other place and that's been happening four years for decades and we have i think a stat in the book i don't know if it's in the main text or in the body of the book or in the footnotes where it talks about the decrease in tenure the increase in casual labor instructors and lecturers and adjuncts the rise and debt financing and debt servicing as being a huge part of expenditure i mean this is the next line of the casualization of the university and which universities are always telling their instructors and department heads to do more with less and so it's not really surprising that this is the next point i mean the the last kind of thing we had it was mooks everybody is going to be on corsera or whatever or whatever you know and your aim is going to replace your machine learning 101 class but i mean it's the next line of this now everybody's going to have a personalized tutor and that's how they really shine that turd and you know if we could drop the scarcity and austerity mindset and think in terms of what would it be like if we wanted to sort of richly invest in education there's so much that we could do and i have a favorite passage from the book that i'm not going to read the whole thing of but this started as a as a newsletter post that i authored just in frustration at all of these universities putting out their a i statements about how we're going to be the leader in a i we're going to you know invest in it throughout and it's like what if what if just one university would say instead and this is this was a point of an edit in the book where i had so the introduction is imagine of just one two or four year college put out a statement along the following lines and the original version of that head oh any and then parentheses you know digit number one and i was like this joke isn't gonna fly so that's out of the book but the the imagined university statement ends with we recommit to our mission of training students to be critical thinkers across disciplines who can consider sources of information and locate them within their context who can evaluate toolkits for the tasks they are taking on and decide which tools fit which task and you can see through the glib marketing that power cloaks itself in and i would love to see that or something along those lines where you know we really inhabit education for what it is and that to me sort of speaks of this larger thing that we can do in resisting AI hype which is to really make vivid what it is that we all do in our lives and in our work or some people art is work but not everyone right and when education is sort of held up as oh a chatbot can do it it completely devalues what it is that we do as educators and that same thing happens over and over and over again in all of the fields that are supposedly automatable via AI it's important to recognize or to aspire to want to promote that critical thinking ability in as many students as possible but also to promote that notion that people should be able to assess the technologies in front of them or you know the different tools and systems in front of them to see what is working for them and what is not to be able to properly do that right before we get to those kind of broader ways to address you know the AI hype and the harms that come with it I wanted to ask you both a broader question you know I asked about education and the rollout of generative AI there but what are your kind of biggest concerns that you've seen with the AI rollout over the past couple of years there's so much different things I mean and generally the kind of areas that I'm most worried about are the just the kind of question of how it is being used to this place work or you know in being promised to do a whole bunch of different things in different forms of labor in the book we talk a lot about journalism and science and medicine and I think the last one is probably the one I am the most worried about I think all those social services are kind of a close second medicine worries me the most because I think that's where there's probably the most excitement and the most money medical AI startup is is a phrase that I've seen way too much but I mean that has real media effects right and we've it's sad that at this point we have more than one story where someone was speaking with a chatbot it said one thing or whatever led them to kind of a psychiatric crisis and they they took their own life and that's very upsetting I mean that happened with a character that AI bot happened with this spot that was developed by child research and we mentioned this in the book I think of the outset of chapter four that's a pretty alarming thing and there's more of those forming not fewer right and we're seeing these and not just in mental health but also in skull nursing in different kind of clinical settings it's not a thing that professionals nursing or medical professionals want these are things that people in that national nurses united have done so much political education about they're like we don't want these in our in our offices my sister's a nursing practitioner she's like I don't want this stuff here to be with me in the clinicians office right I feel like the United States is probably going to be like an innovator with scare quotes around innovator on that because of the private health care system right I feel like even before the generative AI moment you already saw amazon and a number of these other tech companies trying to find their kind of niche in health care to tap into these profits and so unfortunately it's no surprise to see them trying to use generative AI for that as well yeah we're just to add to that for a moment I think that you're you're right that you know the disaster that is our health care system makes it right for this but I also think that those with nationalized health care are not necessarily safe because it can look like cost-cutting and you were just telling us before we started recording that you have a new minister for AI in your national government and so like what's going to come and I worry for the UK as well on those lines oh and I would just say I completely agree with you I don't think we're free from this at all yeah unfortunately but so so yeah I mean health care is one really concerning area I have a hard time ranking these things another area that is super concerning because it is I think invisible too much of the public is the use of so-called AI systems in warfare and I forget where I came across this line but but somebody was saying well I mean surely autonomous weapons no one wants war but so long as we have war then wouldn't autonomous weapons be good because they keep people out of harm's way I'm like who is being kept out of harm's way right that was a potential question that someone was oh it's about going podcast we haven't done that one yet I think I think there was something about that in one event reasons kind of manifestos as well that seems very likely yeah I mean also very dehumanizing just the people that the autonomous weapons are talking are not considered a human hmm exactly the people the weapons are pointed at art yeah so you know there's an awful lot of stuff there and there's a really big potential for sort of displacing accountability and so there's some great reporting in 972 magazine talking about how Israel and their war on Gaza created this like target identification system and the people behind it are like this is great because it allows us to generate targets faster so they kill more people and oops if we get the wrong person well the computer told us to kind of a situation and I think a lot of that is you know less immediately visible and often bound up in while it's national security we have to keep we have to do this to stay safe and also band up in the long history of AI where you know the start of the project was very much military funding yeah no it's a very important piece of this and one that I feel like doesn't get near the the attention that it deserves and certainly as you say we've seen that great reporting on how AI is being used to justify increasing death in Gaza and I'm sure the West Bank as well and I have no doubt we're seeing that in other areas of conflict around the world too and we're just going to see that expand as these tools are continuing to be promoted in that way I want to pivot to what you discuss near the end of the book because this is something that I thought was really interesting and it's probably something that we haven't discussed as much on the show and this is how we think about addressing AI hype and these broader harms and you kind of go through multiple layers of potential points to target these particular systems and so I wanted to start by talking about regular people right I think there's often a recognition and I talk about this on the show a lot that to really address these things there need to be broader structural changes right you need policy to target these things it's individual actions aren't always going to be able to take it on but I feel like on the flip side that doesn't mean that there's nothing individuals can do to try to push back against this to try to educate themselves to try to think about what they're doing so I wonder on that front right what do you what do you think that individuals can do when they're thinking about AI hype and and the rollout of these tools one of the strategies is just around asking questions right if someone is encouraging you to use an AI system encourage you to buy an AI system maybe you are actually in a decision making thing or maybe you are doing some community advocacy to want to petty for examples an amazing leader who's managed to push back on various facial recognition technologies and we can all ask questions like this privately and in public so we have some proposed automation or it's been in place and we want to fight back against it the first question is okay what's being automated right what's the input what's the output and can you reasonably connect them to each other right so if the input is audio and the output is text and in the audio there was someone speaking and the text is meant to be a transcription of what they said that is a reasonable task it might not work equally well and there's lots of things to worry about where it goes wrong but that's a reasonable thing in contrast you sometimes see these research projects where people claim that they can tell from a picture of someone's face if they're a criminal or not there there is no sensible connection between input and output there and so that is a place where you can say this does not make sense you can also look at how the systems are being described right if you've got this anthropomorphizing language then that is a really clear sign that they are being sold as doing more they can actually do and that's sort of a place to push back you can ask how was it evaluated do we know how well the system works in some test conditions and then how do those test conditions relate to how it's actually being used in this environment where I'm considering it or it's being used on me or whatever you can ask who's benefiting who's harmed and if something goes wrong what's the recourse that's in place and these are all questions that we can all ask and use them to make informed decisions but also help other people if we are not the decisions makers make informed decisions and so that is sort of the serious work but the main message that I want to get across is you don't have to be the kind of person who can build the system to ask these questions and oftentimes the relevant expertise doesn't sit with the people who build the systems but it's just with the people who are existing in the environment where the automation is happening but you can also have fun right right and so I mean in that case there's the different kinds of things for the individuals to do there's the thing that we like to call ridiculous practice which is a thing that we that came out of the podcast in which if there's something that makes no sense that is absolutely ridiculous on the face of it you should make fun of it and I know you do this quite well online pairs and I'm sure listeners of your show do as well but I mean if it just makes absolutely no sense make fun of it if somebody posts a silly ridiculous cheap looking AI generated photo no troll the hell out of him you know I would even do it for things that ostensibly look good like the studio jibbly jibbly I set it in a way on another podcast and Emily trolled me like it was gleaming on jibbly studio jibbly so your studio jibbly I mean make fun of say hey do you know how much you know energy it took to generate that and you know that the spending people did that you know how much water that probably took you know much I mean it's it's worth really digging in and and also because it is not good quality I mean it's what trusty mcmillan cotton had the near times I don't see how to call him on it but she put it on on blue sky like it's mid it's bad looking you know I mean it's effective I mean there's a cultural weight to making it uncool to making it seem out of fashion there's also strategic refusal I mean just not using it I mean if it's I know there's pressures to use it from all corners well there is student or a worker or whatever but if you make your principal stand I mean it does have an effect and it can inspire and and I think relate to other folks to do it and then I think there's those collective solutions I mean we talk about policy in the book and we're at a you know terrible policy environment in the US right now and you know the states are a little better but you know we have our own battles in California and in elsewhere including this thing that might sneak itself onto I don't know when you're going to publish this podcast but there's a thing in the that's under consideration in the in the US house right now where Republicans want to prohibit any kind of AI regulation for 10 years to be enacted by states and they're trying to sneak that in through the budget reconciliation process which means they don't have to they don't have to get the the necessary votes that would they don't have to invoke closure it can't be it would have to be it only if they have 51 votes and so you know there's those policy solutions but the collective ones I think are very powerful so what are collective ways in which we can push back whether that's through our unions through our communities through our through our schools and our clubs and whatnot what are ways that we can really lean into relationships and really make a stand and say we're not going to rely on these technologies in our cultural production in our operations work and really pushing back can be very very effective there and so I think the same way that people are relying on each other for things like mutual aid and kind of mutual support I mean we can think about ways of doing that as kind of consumers and workers yeah and I feel like given the state of things in the United States right now that is probably a really good path to try to go down especially seeing politically and legislatively what might be occurring the ability to kind of get people together to use unions and other organizing structures to push back against this seems really opportune I just wanted to add that on the sort of individual but also inspiring people around us point that there's this inevitability narrative that we hear from the tech companies all the time and that is a bid to steal our agency and so you know Alex said it can be inspiring to take a principle stand and one part of that is to say no I don't believe you if you say well this is here we have to learn to live with it no right there's nothing inevitable either in the development of something called AI or AI or in the sort of continued presence of this tech in our lives and to sort of claim that space is really important especially as you're both saying in this context where our policy environment is so terrible yeah really important point and I was just going to pick up on a couple of the other things that you were saying like in terms of being able to ask the questions that's where that kind of ability to have that critical thinking comes from right you need to know what to be able to ask and you need to be kind of trained to get an idea of how to interrogate these sorts of things and then of course on the strategic refusal people are sometimes surprised when they hear I've never used a chatbot or at least you know willingly sometimes I've had to engage with these customer service chatbots because that was the only option that I had at least to try to get to someone human right but I read how other people assess them and I don't feel that I need to go and do my own little personal assessment to Uber is another company that I've never used despite how common that is in society so people can do these things and I don't think I'm changing the world by doing that but I just feel that that is a decision I have personally made. I'm totally with you I'm not using chatbots I do my very best not to consume synthetic media I sometimes get trapped like people will send me something and I don't realize quickly enough and like I've heard some of that I don't want to listen or when notebook LM came out from Google there was a couple of journalists who wanted me to comment on one each of those fake podcasts that's why I had to listen to them but for the most part I will not willingly consume synthetic media and people are surprised and they like to say well how can you possibly judge I'm like what I can tell you a lot about how these things work and about how we work with them I don't have to subject myself to it to make those arguments. Yeah and I mean I wrote a newsletter post that was that was something like I hate I hate having to look in trying to discern whether something is AI slopper or not like it is not a useful skill so far is it's useful on so far as like this is something that I cannot trust but I'm so upset that there's so much of this here on the web now I have to use cognitive energy to discern whether this is an AI generating image and I have to then change my opinion of the brand because they don't respect me enough to actually use or put some put some effort into this or to hire somebody to do this you know the same thing with text I mean you can discern it kind of reading long enough but it's and I think this happens a lot when we we get lots of podcast pitches as I'm sure you do too and I'm like this this reminds me of synthetic text this seems this seems like somebody did not do their work here yeah I'm sure so many of those pitches are are generated and just sent out on mass I have no doubt about it we need to boycott Coca-Cola now right for using generative AI in its ads and enforcing that on so many of us I wanted to move to talk about legislation as well right because you you talk in the book about how there are existing laws on the books that we can use but there's also new legislation that can really help to challenge AI in important ways so I was hoping you could talk a bit about the legislative front and and how you think about that in this moment given what is happening in the United States just to back up a little bit there's a few tropes that come from the tech lobbyists to the general public and to policymakers that I think it's really important to push back on the first one is tech is moving so fast policymakers can't possibly keep up and related to that this is something that only the people who build the tech understand that's absolutely false right the point of policy is to protect rights and that's not moving so fast I mean in some cases we would like it to move a little bit faster but the question is not you know how do we regulate this technology but how do we protect rights given changes in the environment that are due to the technology and that starts with how do we use the policy that we already have and under Lena Khan the USFTC was doing some fantastic work sort of predicated on the idea that their role is to regulate business activities and if those businesses are using automation to do those activities that doesn't change anything so there's there's a lot that can be done with existing regulation another one of the tropes is that this is a race usually within the US it's you know framed as a race between the US and China you know very xenophobic framing and if we you know hamper ourselves with regulation then we're not going to win the race so you know that is one to push back on and then finally there's this trope that regulation stifles innovation which is connected to the the second point there and that's absolutely false right regulation harnesses shapes guides innovation towards the goals of the regulation and the only thing that it stifles is the ability to just like amass all the money which kind of is the point right and you can't use innovation to refer to that and be honest at the same time so I think there is scope for protecting rights more thoroughly I think this status of privacy law in the US is is really not where it needs to be and you know there's principles of like data minimization that I would love to see more strong and then my sort of like pipe dream regulation would be to set it up so that anybody who produces a chatbot is actually accountable for the output of that chatbot because a lot of our issues with this come from the fact that nobody has accountability for what's coming out of the chatbot in most cases there was the air candidate situation where I actually got held accountable for their chatbot but for the most part that's not well established I don't know if you saw but there was a recent story I think in the financial times that insurers are now offering a new type of insurance to companies when their chatbots make errors or mistakes or say things that end up costing them a lot of money right yeah which it's interesting I guess I'd like that it suggests that the company should be responsible for it but I think there's also interesting questions about like how do we allocate accountability so if you think about the instances of the fake mushroom hunting books that get posted on Amazon and people are actually getting injured because they've followed that it's like okay so how do you allocate the accountability among the chatbot producer so we know whichever system was used to produce the thing in the first place to the person who used that to produce the book and put it on Amazon to Amazon like you know sort of where's the accountability across those those three sites I think that's an interesting place to be working in yeah the state of policy in the US is pretty it's pretty rough I mean there's on the federal level it's pretty much stopped right on the state level there are a few proposals so I mean there have been some things in the last California legislative session there were some things that were targeting things like non-consensual deepfakes there was something that was that SAG offer was promoting in terms of preventing like this is for being used that didn't prevent them outright but then put some guardrails around that right now I simply remember Bauer Kahan has a bill called AB412 which would at least allow copyright holders to make a request of AI model developers to get notice of if any of their copyrighted works were in the training data and they actually have to respond within seven days if they don't respond then they can file a civil suit so there's something I mean it's it is something on the copyright front which is again a pretty weak mechanism but it's kind of the only one in which artists have been able to file any kind of grievance with companies and we're probably quite aware of the suits that many people have taken including New York Times and the large lawsuit by Sarah Silverman and in Jodie Pico and Game of Thrones guy whose name he's forgetting right now George R.R. Martin sorry but they're at the same time you have this attack on the legislative front that I mentioned earlier that the Republicans are trying to force through the budget bill there's also attacks on the California Privacy Commission right now which they've got rid of the person who is the executive director and if someone who's much more minimal road much more industry friendly and so you've got these things happening and so anything that's targeting AI particularly has been sort of defamed but as Emily said there are existing laws on the books and if it's going to be the kind of matter of if there are people in regulatory agencies who are willing to enforce them yeah you know obviously I agreed with all the proposals and arguments that you made around legislation the one that really stood out to me was thinking about not just data rights but minimization of data collection right because we know that these systems rely on all of this data that has been created and that they have kind of scraped off of the internet to create these models and I think that's a real point to think about pushing back on is just how much data these companies can collect can create can store on us and trying to push back against the power that comes with that is one thing that I think would be a great thing to look more at into the future but we're running closely in of our time so I just wanted to kind of end off by asking you a broader point you've put the work into this book you've been hosting your podcast and being really vocal critics on AI for the past few years as we have entered this new hype cycle around generative AI and so I was wondering you know what you most want people to take away from reading this book you know what once they pick it up that's a great question and I want to have sort of a pity answer to it but but I think that I think that I want people to take away that we all have agency here and that it doesn't matter if you are not a tech developer it doesn't matter if you feel like you're bad at math and computers your expertise is still relevant you still have something to say and you still have you know ground to stand on for your own refusal and your own decisions. I mean my people you think is that AI is not inevitable it's such a tagline that we also have it dear and I think it's the one that this you know this book re-inscribes and and provides a little bit more meets on how to push back and so you know I want folks to go away and know that amongst the horrors of the horrors of the world AI surely is not inevitable and part of one's own kind of political action you can incorporate this push back as well. It's been great getting to know both of you the past few years I'm really excited for people to pick up the book to read it to engage with it I know it's going to be helpful for so many people in getting a good perspective of the critical view on AI thank you both so much for taking the time to come on the show I really appreciate it. Thank you Paris. Thanks Paris it's so great to talk with you again. Alex Hanna and Emily and Bender are the authors of the AI Conn. Tech won't save us his maiden partnership with the Nation magazine and is hosted by me Paris Marx production is by Kaila Houston Tech won't save us relies on the supportive listeners like you to keep providing critical perspectives on the tech industry you can join hundreds of other supporters by going to patreon.com/tech won't save us and making a pledge of your own thanks for listening and make sure to come back next week.