Technology and Race with Ruha Benjamin

This is a Headgun Podcast. and I'm gone over. And for decades, we've heard about how technology will change the world, right? In the 90s, it truly seems that the internet and revolutionary, even utopian potential. We were all told and we believed that would break down the barriers that separate people and information. We were told tech would fix all of our problems, it would solve educational access with free, massively online classes. It would eliminate discrimination because no one could tell who was who online. So we would all compete on the merits. We even thought that tech would be strong enough to topple dictatorships. Remember the Arab Spring? The media literally told us that Twitter was going to be stronger than decades of authoritarianism from Tehran to Cairo. And this was back when Twitter only had 120 characters. Each of those characters was doing a lot of heavy lifting. Well, suffice it to say, if you're living in the same present that I am, no of these dreams has come true. A decade later, the democratic gains of the Arab Spring have largely rolled back and authoritarian governments have proved to be even more adept at harnessing the communicative power of the internet than democratically minded citizens are. And in America, well, let's just say our utopia of technologically secured equality has not arrived yet. Let's start with a claim that technology would be a boon for education. That sure would be useful right now, right? After a time when students and teachers are unable to gather together in the same place, why surely after decades of talk and iteration on the idea of online learning, we should be ready for this moment, right? Well, unfortunately, we are not because the digital revolution has not come equally to America. According to the Pew Research Center, about 15% of all households with school-aged children lack of high-speed internet connection. It's pretty hard to log into the Zoom with your teacher when you're stuck guessing the neighbor's Wi-Fi password. And the reason for this gap, this digital divide, is that rather than our government exerting its power to make sure the internet in this country was built fairly, we've allowed corporations to do it for us with very little oversight. And the result is that wealthier areas and populations have been prioritized throughout the history of the internet while lower income neighborhoods have had to wait or have never received a connection to broadband internet at all. Advocates call this digital redlining because just like residential redlining, it's had a disparate racial impact. Black Americans are less likely than white Americans to have a broadband connection at home. A future of internet equality, this is not. Now, look, if you listen to the show, you know that being skeptical of the promises the techno-utopians is not new to me, it's a recurring theme on this show. But I don't want you to think that I'm turning into a technophobe either. I'm not here to tell you that technology is actually bad because the truth is the source of the problem is deeper. The truth is that technology is not produced in a vacuum by abstract thinking techno innovators with their minds up in the cloud. No, technology is produced by a society. And when a society is based in injustice and inequity, our tech ends up reflecting and reproducing that injustice and inequity. So you know, if you start with a racist police system, we shouldn't be surprised when the algorithms of predictive policing produce racist results. Why wouldn't they? The technology arises from the society. So it follows that if we want to fix our society, we're going to need more than just technological fixes. Giving children in a broken school system new laptops is not going to be as effective as reforming their school system so that students don't receive a poor education just because of what zip code they live in or how about addressing the poverty that those students return home to every afternoon. A free iPad doesn't fill up the empty belly that makes it hard for students to focus in class. None of this is to say that technology is useless, but on its own, it'll always be insufficient because social problems need social solutions. Well, to talk about how technology, when improperly used and deployed, ends up reproducing rather than solving social problems, our guest today is Ruha Benjamin. She's a professor at Princeton and the author of Race After Technology, Abolitionist Tools for the New Jim Code. This interview was so much fun. I found her incredibly fascinating and lively to talk to. You're going to love it. Please welcome Ruha Benjamin. Ruha, thank you so much for being here. My pleasure. Thank you so much for inviting me. So look, I grew up with in the middle of the tech revolution, right? And with all these ideas of techno-utopianism that the internet was going to solve every social problem. On the internet, no one knows you're a dog, the famous New Yorker cartoon, right? Algorithms will bring in a world of equity and fairness across the land. To the last decade, there's been a lot of criticism at that point of view. I think you're part of that criticism. Could you talk about that and could you talk about this phrase that you've coined the New Jim Code? I'm really curious about this. Absolutely. So there are two dominant stories that we often tell about technology. One sort of goes back to that phrase techno-utopian. And that's the cool aid that Silicon Valley is trying to sell us. And it's about, you know, the robots are going to save us. They're going to, you know, they're going to make things more efficient. They're going to be more equitable, all the good things. But there's another story that we're also accustomed to hearing, the techno-utopian story that Hollywood sells us, which is the robots and the technology is going to slay us, right? So it's going to destroy humanity. Take away agency. Take all the jobs. And so although on the surface, these seem like opposing stories, like the surface is pretty different. When you dig down, they share an underlying logic that technology is in the driver's seat. We're either going to be helped or harmed by technology. But the human agency and agents behind the screen get lost from the narrative. And so partly one of the things we have to begin to do is to tell different stories about technology that recoup the elements of power and agency that are already there. And part of the issue that I take is that right now, a small sliver of humanity is doing that imaginative work and doing that design work and that programming work. And so their worldviews, their imagination, is being embedded into our physical and digital infrastructures. And so part of what we have to do is explode that phenomenon and really make it much more participatory, democratic. And to do that, we have to think about the existing power relations, which bring us to the new Jim code, which will sound familiar for those who say, red, Michelle Alexander's critique of mass incarceration, which she terms the new Jim Crow, which itself is a riff off of the way that we've talked about white supremacy and racial segregation in this nation when we term the era of Jim Crow of explicit legalized segregation. And so in the same way that Michelle Alexander is trying to get us to see how mass incarceration continues to perpetuate social and racial control, my concept of the new Jim code is trying to get us to think about how technology continues to do that work. That it hides so many forms of inequity under the guise of progress, under the shiny surfaces of AI systems, automated decision systems, machine learning, et cetera. And so that code, part of it is key, that it's coded, it's harder to discern, but hopefully now with the growing language, as you mentioned in the last few years, of people and organizations and movements shining a light on this phenomenon, we can start talking about it and pushing back against it. Oh my gosh, there's so much in there that I want to dive into. I mean, you had this incredible idea at the beginning about how we talk about technology as neutral and as something that's just coming like a force of nature that we almost have no control over. It's literally the robots, it's literally algorithms, and you're right that decenters, the people who are making it, which is really interesting. Yeah, can you talk more about that? Absolutely. And so I think part of the thing that realizes that so much oppression happens out of solutions to things. And so mass incarceration itself was a reform from the penal, sort of like just killing people, now we're going to hold you in cages. So now the issue is that how do reforms, how do things that seem to present fixes for things actually reproduce certain dynamics, insertive social hierarchies or forms of oppression. And so in that way, innovation goes hand in hand with inequity and oppression. So often we associate like technological innovation with social progress. That's part of the cool aid is to get us to believe that those two things are the same. When in fact innovation has long been gone a hand in hand with all manner of oppression. And so I often think of about like the first person who put up a white's only sign in their store, that was an innovation, right? - Yeah, and then later they put up a neon sign and they innovated. - Exactly, exactly. Or like the first person who said, you know what, you know what's a good idea? Let me create a colored water fountain. Like that was a bright idea. And like whoever did that, like, you know, it caught on. And so the things in hindsight that we think of is so like backwards and regressive at the time, they were innovative, which should get us thinking about what now are we so enamored with? You know, the we think, oh, this is the next bright thing. And so that's why often when I'm talking with folks, I refer to that better off-ted episode. It's title racial sensitivity. - I love that show. This is a great pull. That show was canceled before its time. That was a big fan. - It was too subversive. It was too subversive, Adam. So there's this three minute genius clip in which the company decides to install sensors all over the building, but the sensors don't detect a darker skin. And so none of the black employees can open the doors. They can't use the elevator. They can't use the water fountain. But that's what happens is they, you know, when they bring it to the CEO, the response is, oh, oh, don't worry, we'll fix this. We'll put a manual water fountain for the darker skin employees next to the one with sensors, which to me is just a really great illustration of how, you know, you create all of these kind of like splashy things to make life easier. And then if it doesn't work for some people, we retreat back to this really iconic, you know, example of the manual water fountain. And all over the water fountain, it says, this is for black employees because they can't use anything else. And so again, getting us to think about our assumptions about what innovation is and what it does. Well, what this connects to for me is, I've done work in the past. For instance, we did an episode of my TV show called Amruin's The Internet. And we open that with a critique of technopesimists, right? People who say, oh, no, this new technology is destroying our minds, right? Oh, our cell phones, we're all addicted to our cell phones is destroying human society. And we made the argument, hey, same as it ever was, right? This is what people have said about every form of technology. People said about books, oh, that they're a destructive new technology that would destroy the younger generations. Literally they did. People said that about paperback books and train travel and telegraphs, et cetera, right? And we were saying, no, you know, technological innovation has always been with us. And there's always been people saying the sky is falling. Yeah. But this is the argument that you're making is the same argument, just the obverse, that it's that, hey, technology also doesn't fundamentally change our society the way we think it does. If we've got a built-in power structure where some people hold power and some people don't, then likely the technologies that that society creates are gonna just keep reifying, reinforcing that power structure. Is that, does that track for you? That definitely does. And to the first part of your comment, I was thinking about that meme that shows, like maybe it's like the 1950s, like a train scene, like a New York City train scene and all the guys have their holding up their newspapers and not talking to each other. And it's like, see, this is like people never talk to each other. It's not the phones that did it. But certainly this idea that it's not the technology that's inaugurating these forms of sort of anti-social, you know, anti-social behavior or inequity. It's the larger ecosystem. It's the social inputs that actually continue one era after the other to continue to produce the same predictable outcomes. And so rather than just say, you know, get rid of all the technology, let's look at ourselves. Like really, let's use this black mirror as it were, you know, to shine a light on what we take for granted about our social order and think about addressing the root issues rather than just trying to throw the technology out. Yeah, when we see those stories about, like you said, from that better off 10 episode, which was, that was like 2009 or something. That was very ahead of its time. It was, it was like five years later, we started having actual stories like that come out all the time. Absolutely, press it, press it. Yeah, about like this or that facial recognition technology, not recognizing people of certain races, etc. We see those stories and we say, wow, bad technology. But that's not really a story about the technology. That's a story about our society, and which built the technology. Absolutely. There's a wonderful line there where the one in the boss that said, this is not racism. It's not racism because we're not targeting black people. We're just ignoring them. And for, and, and for me, that's just a perfect, like, just expression of so much kind of corporate diversity culture and, and all of the things that has less to do with those sensors and the technology. But really how people, how so much indifference continues to perpetuate inequality. It's not the big, bad boogie band behind the stream, like, let me get these people. It's like, I don't give it down. I don't really care about them that, like, allows things that the wheels to keep turning. Well, and so let's talk about that because that is literally the way you put that. And that's a great line is that is, I don't see color, like made manifest, right? It's like, I literally don't see it. My tech, not the algorithm I designed cannot see it. But the thing about I don't see color, these sort of neutral ways of doing it is that it seems a lot less pernicious than the racism or the, the racist structures that we were taught about. But I was taught it on sixth grade, right? Exactly. So, yeah, we have the image of like, you know, the police officers sticking the dogs on the black youth and the water fountains. Like, really our imagination of what counts as racism is like, it requires the white hoods. It requires a snarly, like, white men who are out to get you when the vast majority of racist systems just rely on us clocking in and out, like just doing our job, like put your head down and just follow the rules. And so, there was a really great study of last year this time that came out that was looking at healthcare algorithms. And this widely used healthcare algorithm that basically was like a digital triaging system, like you come in as a patient and it'll tell the healthcare provider whether you're high risk or low risk. And so, if you're high risk, they use this particular algorithm to get you more resources to prevent whatever, like bad things are predicted to happen to you based on sort of past data of people like you. And so, what the researchers who studied this algorithm found is that it was flagging white patients at a much higher rate to receive these coveted resources, like more time with your doctor, more services outside of the hospital, it was basically designed to help people stay out of the hospital. And that black patients who were sicker were not getting flagged for these services. And so, this particular algorithm in many ways was carrying out the work of a white's only sign at a hospital, like you can't help, you're not doing it, but it was through this neutral system, right? And so, when the researchers opened it up to figure out like what's happening here, why is this, is it like out to get these black patients, this system was in fact race neutral. There was no, it wasn't keeping track of race at all. Instead, it was using a proxy variable, it was using healthcare costs, like how much we have spent on particular patients, that was used as a proxy to say, if we've spent more on you, that means that you're higher risk, like you're more likely to get sick, but we have a system in which people who need care can't get to the hospital or don't have insurance. So, lots of sick people, they're not getting anything spent on them, which makes the system think, "Oh, you know what, you're fine, you're low risk." And so, mostly by using cost as a proxy for the healthcare need, the system was perpetuating this past inequality and projecting it into the future under the guys of like a neutral system. And so, millions of black patients for years were not getting these coveted services because the healthcare professionals were relying on this system called Optum. And so, this is just one of many examples where really we see that race neutrality, or as you say colorblindness can be deadly, because we're ignoring the past, right? The data that's being used to train a system, it has all of these patterns of both institutional and interpersonal discrimination, 'cause both are at work in our healthcare system, like individuals and other ways in the policy and the insurance is structured, but it was using that as if it was just a straightforward reflection of reality and teaching this algorithm, make more decisions like this. And so, this is what we get when we ignore history, or we ignore social inequality in the building of technical systems. And the people building that algorithm probably had no idea what they were doing. They were like, "Hey, we're just making a little cost algorithm here, probably if you told them they'd be like, no, no, is that the racist algorithm?" That's almost a direct quote from after this study came out, that's like a direct quote from a million articles where they were like, "We didn't try to do this, this is not what we meant to do, but again, we go back to better off Ted, in difference is really one of the main drivers of racism." And so hiding behind the kind of, "I'm not racist, but whatever comes next is probably going to be racist." Yeah. Are there other examples of this that stand out to you? in almost every, every significant area of our lives where important decisions are being made. Those who the human beings, you know, the gatekeepers are outsourcing those decisions and consulting technical systems as if they're neutral. In the car in our penal system, it's everywhere. From every single stage, from who gets police in the first place, who's paroled, who's, you know, every single stage, we have a risk assessment tools that are being employed that are deeply racially biased and the studies are starting to pile up that have been auditing these. In fact, now during the pandemic, there was a system called Pattern that was being used to decide who would be released because, you know, our prisons are overcrowded and so COVID is running around. So they're like, okay, we've got to figure out who to let out so that we can sort of, you know, deal with the overcrowding. And so some places use this pattern assessment again to decide risk, like who's low risk or high risk. And then the people who were deemed low risk were the vast majority were white people who were in white men who were in inmates, people who were homeless were high risk, black men were high risk, people with mental illness were so all of the most marginal and this already marginal and oppressed population were deemed high risk and so we're kept in, you know, caged where others were released. And the difference was something like 30 plus percent of white men were deemed eligible for early release versus like six or seven percent of black men. And so this is, and so pattern, this was during the pandemic, but those kinds of risk assessment tools are being used up and down the penal system and our education system there are examples when it comes to getting like a home loan or other kinds of like loans. So any or area of our lives that you can think of where people are making decisions about a lot of people at once, this kind of new gym code discrimination is happening. Yeah, just mentioning housing like it, there seems to be a connection back here like to redlining, for example, which I've talked about, you know, extensively on our show and on this podcast before, but because that was like for me, really, really learning about the history of it and the massive effect it had on American society, that's to me the most vivid example of how, you know, these how the structure of our society can be set up in a racist way that you can end up perpetuating without even knowing it today, like those the restrictive covenants that they literally had in the suburbs in like levitown, you know, only Caucasians apply, you know, will be given these mortgages. Of course, that was overt racism, but then hey, 40 years later, well now if you're just continuing to operate on, hey, what is the home values of the neighborhood? What is the neighborhood? Quote high crime or not, you know, et cetera. Then you end up perpetuating that original sin, right? Exactly. And all those people, they just have to go to, but you know, go do their little office job at the bank, like fill out the little films. I love to show people like the actual like bureaucratic forms that go into that to sort of demystify it because it's not like a big bad banker standing in the front and we're like, no, get out of here. We don't want you. It's like a like a little bureaucrat sitting there and filling out, okay, this many, this many Italians live in the neighborhood. This many knee grows, this many people who are getting welfare, calculate, calculate, sorry, you can't get a home loan to go in here. So it's like for me, the forms of harm that we associate with kind of like these scary boogie men of the past were often carried out by people again, just putting their head down and doing their job. In my name, my grandma's neighborhood in Los Angeles, Lamert Park, I dug up this flyer from like the 1940s when that neighborhood was getting developed and the housing developers were trying to entice white families to move there. And they put up these flyers that basically said, come, come live here. You're, we have beneficial restrictions. Your investment will be secured. And of course, beneficial restrictions was a reference to racially restrictive covenant. So they were basically telling them, you buy your house here, you have these covenants that ensure that it will stay white. The neighborhood will stay white and your investment will be secured. Now I was following that kind of rabbit hole and I learned about a black family that was trying to move into the neighborhood, the Wilson's. And when they did, the Homeowners Association rallied around and there was a white family that wanted to sell their house to the Wilson's. And the Homeowners Association sued that white family and like, no, you're going to mess up the neighborhood. And it's so interesting when you read like the interview of the main plaintiff from the Homeowners Association, he says, I'm not motivated by any racial animus. This is strictly an economic, you know, issue. And so even in the 1940s, he's like, yeah, I'm not racist, but this black family is not moving into this neighborhood for ex-wires. And so like that rhetoric that we are familiar with now, it's been with us for a while. People don't want to own like what the ill feeling, but they, the economic motivation, the idea that racism is productive. It doesn't just harm people. Yeah. It actually garners wealth and status and all the good things of life to those who are perpetuating it. And so even this guy back in the 40s was like, you know what, I don't care, I don't give a damn about the Wilson's, but you're not going to mess up my, you know, my property investment. Last thing I'll say about this story of the Lambert Park, is that? Please. There was a Reverend and a rabbi who went door to door knocking on people's, you know, doors and talking to all the neighbors. And was like, this is terrible. We shouldn't be suing this family. We should let the Wilson's move in. And so they went and kind of did this like labor of trying to like impress upon the folks in their neighborhood that this adherence to white supremacy, however sort of hidden behind the the language, the legal ease of this lawsuit was not like the values that we should be upholding. And ultimately through their efforts and others, the homeowners association dropped the suit. The Wilson's moved in. There was like a party for them. And so this is just one example, like we don't always have to wait for the laws to change, to start to like force changes in our relations and in our own backyards. And to me, the example of this rabbi and Reverend who were like not having it, were like we're not waiting for the federal government, but our neighborhood is not going to do this. And so that's really I think a call to action for all of us. I do, it does raise a question for me though that we were talking earlier in this conversation about. And by the way, that's an incredible story. Thank you for sharing it. I'm glad it had a someone happy ending. But we were drawing this contrast between the quote neutral algorithms today and the sicking the dogs on the on the folks on black folks, you know, in the old days. But what you're describing is a is a story that doesn't sound too dissimilar right now. I mean, we still have homeowners associations. Yes. And there are homeowners associations that still use cloaked legalism and economic arguments to keep out black people. And guess what? We still have cops physically attacking and brutalizing black people. So I mean, I love that. I think it's really so different. Or is it same as it ever was? Yeah. To me, I think that that that insight is like the key because it's not there hasn't been this transition from an old timey racism to this new fanguels coded racism. So one of the things that I'm really trying to trace in race after technology is the continuity. The fact that now computer codes are doing this work of coding. But legal codes have long been doing this. There's all kinds of ways in which this coding of racism, this embedding it into our systems. Other tools have been used before we've had fancy algorithms. And so that's again the kind of point we started with is it's not simply the technology that's inaugurating new forms of racism. It's providing a new kind of like twist on something that's been with us. And so one of the things I do is really show exactly how legal codes have done this. But even if we go to something a little less tangible, like I think about the way that we culturally code our names. I start the book talking about people's first names and how we often use that as a proxy for other qualities about people. And it's often used as a pretext to open doors and shut doors for people. And so there's a great audit study from about 2002 or three in which two economists from the University of Chicago sent out thousands of resumes to employers in Boston and Chicago. And what they did was they just changed the first names. They changed like some of the resumes had names like Emily and Greg, some like LaKisha and Jamal. And all the qualifications were the same, the number of years of education, all the things. And they waited to see who the employers would call back. And of course we wouldn't be surprised that those white sounding applicants, the names with the white sounding applicants received many more callbacks and calls of interest. And the economists calculated this to mean to be equal to the assumption that those white applicants had eight additional years of experience, work experience that they didn't actually have. And so they received 50% more callbacks. And so this is a way in which our names code certain assumptions about us for good or bad. And people use them all the time to ask actually provide opportunities for people. Now, someone hearing that, the results of that study might think, well, man, humans are crap. We have this implicit bias, we're discriminating. So shouldn't we let computers make the decisions about employment? That's the shift. It's like, okay, acknowledging our bias and then saying, okay, let's let this AI powered system in which I sit here in front of this screen and it kind of tallies all of these data points. And then what these firms that are selling this do is they say, now we'll compare job seekers scores to those of existing top performing employees in our company in order to decide who to flag as a desirable hire or who to reject. And so again, the assumption is that this system, which is presumably created by human beings and had to be taught how to judge applicants is somehow gonna be more neutral. And in fact, that doesn't turn out to be the case. Like if you in your own company have been hiring mostly men or mostly white guys for the last 50 years and that's your base, that's your standard for who a good employee is. And now you're judging everyone else according to that. However, you code that in terms of body language and posture and accent and all the things that the AI systems keeping track of, you're likely gonna get more of the same. And the danger is that people actually think that that system is more neutral than say a person looking through resumes and deciding, I don't want Jamal working here. You know. - Yeah. Well, a simple way to put it is like, you know, AIs, I'm a AI expert but I've played around with some AIs. I've talked to some AI experts on the show. You train them on data, right? Like a very common form of AI now is a neural network. It's sort of this general learning machine. You give it a whole bunch of data. It sees patterns in the data without you even knowing how it's quite doing it, right? And then it's able to tell that it's being A and B. Well, if you're training the system on a racist society, right, you're saying, let's train it on every employee in America and you know, how much money they make and how many skills they have, et cetera. Well, of course it's gonna, you're gonna end up with an AI that says, oh yeah, white people are more qualified than black people because in the system I currently have, those are all the lawyers and, you know, paralegals and accountants, right? So the smarter it gets, the more racist and sexist it becomes. Like if we're judging intelligence by how closely it mirrors human decision making. So like this intelligence, and quote, is actually like the most racist and sexist version of human, you know, thinking. And so in fact, a couple years ago, Amazon's own hiring algorithm was weeding out women precisely because that their workforce is predominantly male. So it was like seeing these resumes with like, you know, Laura or Tanisha and I was like, oh, this company doesn't want this, like, they've chucked it out. But then once they got rid of like gendered names, it got smarter and it started looking, okay, this applicant was on the Women's Chess Club, throw that out, this applicant went to Brynmar, throw that out. And so, and then it started looking at how applicants talk about their work, like what kind of adjectives people use and we know through other sort of social, psychological research that, you know, that has a gender dimension to it, the kind of language we use to describe our work. So it got even more intelligent. It was like, okay, like throw those people out. And so eventually, Amazon had to recall this whole thing. So if Amazon can't get it right, then we often be pretty wary about like outsourcing these really important decisions to systems that we assume are neutral. Well, I want to ask you about what we could be doing otherwise, but we've got to take a really quick break. We'll be right back with more Ruha Benjamin. (upbeat music) Folks, we're a couple months into 2026 and I don't mean to brag, but I already nailed my primary resolution to get comfy. There are a lot of ways I get this done, practicing a little more mindfulness, treating myself to a little nap when I need one, but the core of my comfort venture is replacing my basics with everyday go-to's from Bombas. The all new Bombas sports socks are engineered with sport-specific comfort, no matter how you stay active. I love to hike and these socks are perfect for it. They're cushioned where I need it. Most sweat-wicking and loaded with other tech features to keep me comfy and locked in for every day around the house comfort. Bombas also, as you covered, with the comfyest footwear imaginable. Every weekend, I practically live in my luxurious Sherpa Sunday slippers, they're like walking on clouds. And for every item you purchase, an essential clothing item is donated to someone facing housing insecurity. One purchased, one donated, with over 150 million donations and counting. So maybe it's time you start treating yourself nicer, make a commitment to your own comfort and head over to bombas.com/factually and use code factually for 20% off your first purchase that's B-O-M-B-A-S.com/factually, promo code factually at checkout. When your company is growing fast, order fulfillment can make or break your success. Ship stations platform brings order management, rate shopping, inventory and returns, warehouse systems and comprehensive analytics all in one place, saving customers 15 hours per week on fulfillment. Ship station compares rates across all major global carriers, USPS, UPS and FedEx, including your own negotiated discount rates to find you the best shipping option on every order with discounts up to 90% off. With ship station, everything you need to manage getting orders to customers is in one place. Connect to over 200 sales channels, instead of five to seven disconnected tools, you've got one. You know, my business is growing and if I ever get to the point where I'm running my own e-commerce operation, this is gonna save me an incredible amount of time. So take some of the tedious tasks of running your own business and put them on autopilot with ship station. Try ship station free for 60 days with full access to all features, no credit card needed. Go to shipstation.com and use code factually for 60 days for free. 60 days gives you plenty of time to see exactly how much time and money you are saving on every shipment that shipstation.com code factually, shipstation.com promo code factually. (upbeat music) Okay, we're back with Ruhah Benjamin. I wanna talk about what we could do differently. You're a science fiction fan, aren't you? - I am. - I know that you use science fiction in your work. - Yeah, I'm teaching a class this fall called Black Mirror Race Technology and Justice. (laughs) - Oh, that sounds great. I wish I could take that class. - Yeah, it's gonna be fun. It's gonna be fun. - Well, what is, why the focus on science fiction and what do you, what does it change? - Yeah. - I'm sorry, let me take that packet. Let me edit that question. Well, why the focus on science fiction and why is it important the question of who gets to imagine the future that we're gonna have? That question of imagination I'm curious about as we think our way towards the solution. - Yeah, I should say first, my earlier work before I got to all this stuff around AI and algorithms was in the life sciences. And so my first book was about stem cell research and regenerative medicine. So I was hanging out with all of these really fantastic scientists that were doing cutting edge work, growing things like heart cells and a petri dish in a lab so that say if your relative needed a heart transplant rather than having a donor, the idea, the hope is that one day we can reverse engineer their own cells and grow them a heart from their own cells so that their body doesn't reject it. Right, cool. It's like out of this world. And so hanging around people that are doing this just like as their day job and for anyone else seems like this is like this is science fiction. But I realize in talking to them how many people were initially like their interest in science and technology was sparked by seeing like a Star Trek episode or some wild like reading some interesting thing in a book. And so like from a young age, like those seeds are planted and then they eventually follow their pursuits and then some a small slice of those people actually get the opportunity, the scholarships, the education, the mentors, the institutional affiliations to be able to take that early, those ideas that sort of sparked when they were young and actually get to materialize it in an actual lab where now they're growing actual heart cells or developing a scanner like they had on Star Trek to say, okay, let me figure out what's wrong with you. And so for me, it started with realizing how important imagination is to the things that end up becoming science and technology for the individuals and similarly for me. But also like there was a real lopsided investment in imagination like I would be one of the few social scientists in these spaces and I would be like, okay, that's great. That's nice, we'll be able to grow people's organs. Now what about the fact that so many millions of people can't even get like the basic healthcare? Like now we're in the middle of a pandemic. Like can't even get a test for this deadly virus. Like how do we match up this great imaginative and actual economic investment in these cutting edge things with a lack of investment in some basic like social provisions and social safety nets and healthcare. And oftentimes my questions around that those would be cats. as like farfetched. Like, oh, we can't ever ensure everyone. Oh, there will always be people who will have to die for the common good. Like, you know, like my basic questions about like public health or the common good, like that was seen as the thing that was out of this world. Like, we'll never be able to do. So for me, it's like, how can we have so much optimism around sort of biological regeneration or AI and so little imagination and so little investment in our social ills and our social well-being, right? And so it's that's what animates me is like, okay, right now our collective imagination is being monopolized by the people who are able to do all this fancy shit. Whereas people who just want like to be able to take their kid to the ER when they crack their chin open, play riding their bike, they're the ones who have to sit there and like, hope that thing closes up because they can't afford to get there. Well, that's what oh my gosh, you're clarifying so much about the techno utopian worldview to me because what it is is these are folks who say that our social problems are inherently unaddressable, right? They say, ah, racism inequality, like all these things, you can't fix those are too thorny like all like all who's going to what we need to do are technological fixes that are going to skip right over them, right? That'll that'll that's the easy thing to do. That's the thing I can address. It's Elon Musk sitting in traffic. We beat up on him all the time on the show. But I really do think this is not being gratuitous. I really think this is a great example. He's sitting in traffic going, I hate being in traffic. I know I'll build a tunnel so that rich people like me can skip the traffic and go right underneath right now. First of all, I don't know why the mother fucker doesn't just buy a helicopter. Like he's rich enough to get a helicopter. Why is he in traffic to begin with? This problem has already been solved for rich people, dude. Jeff Bezos is just in a helicopter. He's not trying to build tunnels all over the place. But why can't Elon Musk who is like this, you know, he's made his reputation as being this visionary imaginative guy, this guy who's like, you know, thinking about the future of humanity. Let's go to Mars. Let's, you know, have AI. Why can't he imagine a world with no traffic? He sees traffic as being an unsolvable social. How could you get rid of it? I don't know. There's too many people around whatever. Why can't he imagine? Why can't he ask the question, well, what if we, how could we get to a world with no traffic, which would be a world with I think and hope public transportation, more walkable cities and like, you know, less people who have to drive an hour to get to work because they had to only afford to live out in the suburbs, so more affordable cities, all these sorts of questions. But that whole sphere of questions, which also involve questions of race and inequality are put in the bucket of unsolvable by our technological because they don't have it. They can't imagine solutions. No, and they're not, they're not, they don't have any reason to. And so I love that you use that example because the epigraph for my first book, People Science was from someone I interviewed and she said, before we figure out a way to get to the moon, can we just make sure everyone on my block can get to work? That was her position. So it's so perfect. But again, you know, it's really like this lopsided investment and part of what we have to do is really push back on the inevitability of technology solving all of our problems. I was recently talking to a group of students, they had organized this conference on innovation over here at Princeton. And one of the other speakers was the guy behind the Singularity University. And so he was like beaming in through zoom and he gave a talk and he was ending his talk and he was right before me. And he said, he's basically telling this auditorium full of students like painting a picture very much like a Elon Musk type picture of the future. This is what's happening. You either get on board or you get out of the way you're irrelevant. You either are signing on to this and figure out a way how to navigate. And so it was like the the inevitability of the future that he is invested in. And so I had the first thing that I said when I got up there was like, he's wrong. He's wrong. No, you do have a choice. That is not inevitable because that's part of the anti-democratic underpinnings of that whole thing is that they really don't want to hear from anyone else. This is the vision of what the collective good is. And anyone else who raises questions about a critique is painted as anti-technology or anti-science. And so the last thing I'll say about that is that even the phrase that we use and we call people "ludites" it misrecognizes who the luddites actually were. They were not against technology. They were against the social costs of technology. They were against the fact that the inclusion of this technology into industry was going to push down the wages. Was going to have all of these social costs. So they weren't against the machine itself. They were against the way that the machine was actually going to reproduce power and inequality. And so even that kind of insult that we cast about, oh, you're just a luddite, it actually, you might actually be more inclined to say, yeah, maybe I am because it's not being against something. It's saying we need to talk about the social costs of these technologies and do better. Are there technologies though? We've been, we've been shitting on technology this whole conversation. Are there technologies that fundamentally do shift the balance a little bit? I think about how our new communications technology have allowed people who were, we took down all the media gatekeepers. And now people who were formally marginalized are able to be loud, spread their message connect. Now also, I think one of the side effects is a lot of people who were kept out of media who we don't like, like, the fucking, like, overt, like white nationalist, right? Didn't use to have a platform and now they do as well, right? So there's a given a take, but, but is there, are there any technological advancements in your view that did make a positive change in this in this sphere? There's quite a few. And the way that I would characterize them is it's not even simply that the technology is the thing that's so radical or subversive or liberatory, but it's like before you even get to designing a particular technology, you have to identify who is this for? What is it for? How is it actually going to intervene in business as usual? And the subversiveness or the power of a given technology to do that starts well before you start coding well before you start designing a programming. And so it has to do with the question that we pose that technology is supposed to answer, because that's majority of, especially when we're talking about these risk assessments and automated decision systems, they cast their view on the most vulnerable populations and they try to predict, for example, the riskiness of the youth to, you know, get in trouble at school or someone to be on parole or someone to not follow their meds or something. So it's looking at individuals who are already vulnerable. The technologies that I find to be so important actually flip the script and point the direction of the technology or the data collection and prediction to those who wield power, to those who are monopolizing and producing risk for the vast majority. So for example, there's a great project called the anti-aviction mapping project. So going back, I think you've had Matt Desmond on show. And so rather than try to you have some kind of, you know, risk assessment that tells a landlord, okay, is this renter going to default on their loan? It puts the tool in the hands of renters and people who are experiencing housing insecurity to actually look and to judge real estate owners and landlords to tell them how these people are treating their tenants and then to be able to mobilize and rally people together in terms of housing justice movement. So again, the technology doesn't save people in that case. It gives them the tools and the data to be able to look at where how the trouble is being produced and then to move in that direction. A second example along those lines, that's more of a parody example, but I like it because it really shows the absurdity of so many of our tools that are used by policing and our carceral system is called the white collar early warning system. So it's like this system where it flags all of the places and cities where white collar crimes are happening. And it has an app that shows you the image of a perspective criminal and the when they design the algorithm that produces that facial recognition system, they use the profile photos of 7,000 corporate executives on LinkedIn. And so naturally the pace of a criminal is white in male. And so here you have this data mapping, you have this facial recognition system, and it's throwing in our face the fact that this exact set of crimes and populations always go under the radar. They always go through the tunnel to go back to your. And they are the subject of this kind of surveillance. Yeah, I want that version of the citizen app. You know, rather than the citizen app that says, Oh, there was an altercation with a knife at, you know, a couple blocks away. And you're like, ah, well, I'm not there. So like, oh, but I'm scared now for no reason. I want the citizen app that's like, yeah, someone's embezzling downtown. Yeah, beware of the banker. There's a landlord illegally converting an apartment into an Airbnb. Exactly. Like, let's create the techno dystopia for those in power. like let's think about technology if we're gonna use it in that way. And in both of those cases and many others, it really starts well before you start designing to think about how we, what we think of has the problem that technology is supposed to solve. And too often the problem is the racialized community or the same old kind of problem spaces. And so technology needs to subvert that. And we have some examples of that coming down the pipe. Well, what this makes me think and going back to your point about the litides is that, you know, your argument isn't anti-technology. It's anti-techno-utopianism. It's anti-these sort of views that some have about technology. That technology is neutral and it's gonna solve our problems. But I think you'd argue technology is a tool. What we need to solve are our social issues. - Exactly. - And we can use technology as a tool to do that. If we're mindful of it. - Absolutely. As long as we keep technology in its place, as long as we don't think that technology is gonna save us as like one half of that narrative. And so really like putting technology in its place, not as the kind of magical fix, but as a tool, but also recognizing that it doesn't mean that any given tool is neutral. Because if the point about tool is to calculate the risk of someone who's been locked up in an unjust system, then it doesn't matter who's holding it. If it was designed to calculate the risk of those individuals, it's oppressive. And so it has to also do with rethinking that design process so we can produce tools that can be used in ways that empower communities rather than oppress them. - What is the, I'd be Wells just data lab. What do you tell me about that? - Sure. It's here at Princeton in the African American Studies Department. And it's an umbrella initiative that connects students, researchers, artists and community activists in order to design just tools. And so over the course of the summer, for example, I had 10 teams of students working from everything. There was a housing team and a work team and a policing team. And each of the 10 teams collaborated with a community organization to build some kind of data justice tool that could be used in the context of advocating for some anti-racist initiative in the context of COVID. And so it's a space to create those connections where academics aren't seen as having all the answers. Like we need to also humble ourselves and learn from people who are working in communities about what's actually needed. I think this goes for technologists too. I think too often the cool aid of Silicon Valley is assuming that they can come up with the interventions without talking to the people who's all of these things are supposed to help. And so part of it is really creating an environment that that can happen. And so for those who are interested, we've posted all of the tools from the summer at thejuststatalab.com/tools. And you can take a look at what's been developed over the last few months. It's-- oh, sorry. I need to edit that moment out. I apologize. No. All right, I'll take a sip of water. I'll wait for you to finish. Then I'll get back into that. That sounds so cool. I want to ask, what do you advise for folks listening, right? When they're-- engaging with technology, right? What questions can they be asking about it to help them improve their relationship with it and sort of see these systems a little better? Yeah, I think what I've found in the last few years is many more people who aren't necessarily working in the tech sector have become rightly skeptical about the promises that were so commonly marketed to us for the last 20 years or so. And so I find that the average person I talk to is already thinking critically and engaging these things and basically not taking things at face value. Like always look like, OK, what's really happening? And so I think when it comes to the data that's collected behind the screens, behind the scenes, in terms of all of the things that we use for free, as the saying goes, if it's free, then you are the product. Your data is the product. And then so I think-- I remember a few weeks ago, Zoom made this announcement where they said that people who used their service for free, Zoom had the prerogative to sell all of our communication and data to law enforcement. But people who paid, their data would be protected. And so for a week there, my lab decided we're not using Zoom. We're going to find another platform. But the outweigh, the public outrage was so loud and vigorous that within a week, Zoom reversed course and said, OK, OK, OK, we won't do it, so. And so that's an example of us collectively voicing what we want out of these things, and not assuming that we just have to submit, submit, submit when we press those forms. If we think that something is not right when it comes to what my colleague Shishana Zuboff calls surveillance capitalism, for example, we need to speak up. This is true, say, for parents right now. The more that classes have gone online, remote learning, learning management platforms, like find out what the school that your kid goes to, what their policy is, their data policy is, about all of that time that your kids are on, whatever the learning management platform is. And I will say that young people in particular are becoming more savvy around this. I know about a year or two ago, there was a student in Brooklyn staged a walk out out of their school, not around the data issue. But because they were spending all their time on this learning management platform and saw their teachers like 20 minutes a week. And so they were like, this is not education. And so we're not doing this. We're voicoting. And so that's another example. But in both of those examples, you see it's working together. It's like it's not simply thinking about ourselves as individual consumers, like, OK, I'm not going to buy this product. I'm going to go to here. That's fine. People should do that. But the more powerful change happens when we team up, when we organize, like those students in Brooklyn or like the public outrage around Zoom. And so I think more and more we really need to stop thinking of ourselves as users. Because as I say in race after technology, users get used. And so we really think about our relationship to technology as, really as stewards, as citizens, thinking about holding accountable. What values do we want to be embedded in these structures? Because if we say and do nothing, it's really going to be the same old kind of corporate surveillance values that we see as the kind of dominant ethos of surveillance value. And so we have to voice our outrage when it's warranted. And we need to be able to articulate proactively like what kind of ecosystem do we want technology to be designed in? What do we want the social inputs to be? And I think we do that best in collective. So finding like your local just data organization and teaming up with them too. I really like that because we're so used to seeing tech companies as being outside of society. And for the first 20 years, hey, they were all insurgents and these weird small companies that were making really groundbreaking technology. And a lot of them had, there don't be evil type slogans or they seem chill and they seemed like they're the breath of fresh air coming through. And now these are the most massive companies in the country that have the most entrenched advantage. And we need to start looking at them. I think you're right. Not as users who are just like clicking a button. But no, we're members of the society. And those companies are also part of our society. And what sort of relationship do we want to have with them? And how much power and what kinds of power are we all right with them having? Exactly. And it feels like that conversation starting to happen. I mean, just seeing the antitrust hearings that were happening on Capitol Hill a couple weeks ago was, I mean, that would have been unimaginable five years ago. And it was still not quite enough, but maybe we're starting-- Yeah, we're getting there. Are you starting to see progress? Yeah, absolutely. I mean, for me, it's been a dramatic shift just in the last few years in terms of us just kind of being like, oh, yay, I phone 70 fill of millions out. Let's stand in line overnight and like two being like people being much more like, you know, savvy and skeptical about all of the shiny things. And so I think that-- I love the way that you described it in terms of recognizing that that little image of kind of like the little outliers innovators in their garage. Like now, the Silicon Six, these big companies, not only are they like the biggest entities kind of monopolizing power and resources in this country, but many of them have net worth that are larger than many countries in like in the world, right? And so like in terms of the power that they wield in monopolize, we really have to culturally like, put them in a bowl. different sort of category of actor and understand what an influence that they're having on public life, but behind private doors, like their decisions have such a huge impact. We got to completely shift up the regulatory infrastructure, the accountability, and maybe even ask, like, do we want them to be that big? Do we want them to continue monopolizing even as they fail to pay billions of dollars in taxes every year? So they say they're doing all this stuff in the name of the greater good, but they don't actually put their money where their slogans are in terms of paying back into the public good. And so that's like a basic 101 thing we need to be demanding in terms of their commitment. And one of the things that really struck me from Tim Wuse book, The Curse of Bigness, about, you know, monopoly in the history of antitrust is that like the original idea, when we talk about monopoly and those issues, we talk about them and turns a money a lot. Oh, they make prices higher and they have too much money and, you know, inequality and things like that. But the real, the original argument against them was about power. Was that a single company having so much power when you talk about the standard oil or whatever, right? That's so much power. They have more power than the government and then the than the democracy, which means that it's inherently anti-democratic. And we would say in America, we don't want a single person who's the CEO of this company to have that much power. And that is happening again with these, you know, with look at Jeff Bezos, right? And how much power he wields over so many different sectors. And that's the question we need to ask is not, it's not just economic. It's also power. Exactly. Exactly. And the last thing I would just add to that is that, you know, these companies and these individuals, they recognize that the tide is turning. They recognize the shift in public discourse. I mean, even if we just go to the Cambridge Analytica scandal and, you know, the 2016 Brexit and US election. So part of their reaction, and this is something we have to be very wary about and and and keep a vigilance around. Now they're trying to what I think of as domesticate the critique, they're trying to create in-house these methods of accountability and ethics and all of that hiring people like in my field, you know, Facebook created this board to oversee what it does. And some of my colleagues rightly called it like Facebook's Supreme Court because they're trying to create like in-house what really needs to be independent and third party. And so they know that we we won't accept the status quo anymore, but we have to be careful about what their solutions are to it, which will just be them kind of creating their own mechanisms of, you know, at least giving sort of, you know, a face of ethics. So are we are trying to be publicly accountable? And we need to say, enough with all of that, those slogans and all of that, we need something independent that's in the power of people to be able to govern not in-house in terms of these companies attempts to do that. Yeah, we need a voice too. Like this is this is a society and a democracy and so democratically we should all have a voice in how our data is used and who's wielding power and these issues. I think that's absolutely right. Well, I can't thank you enough for coming on the show. There's been such awesome fun conversation. My pleasure is great to talk to you Adam. I hope once I get another book done, I hope you invite me back. Oh my gosh, I love you. I learned I've learned so much from talking to you and I'm sure another hour I would learn just as much. Thank you so much, Ruhah. My pleasure. Take care. Well, thank you again to Ruhah Benjamin for coming on the show. I really hope you enjoyed this conversation as much as I did. If you did, please leave us a rating or a review wherever you subscribe. I know. I know every podcast host says that. It really does help us out. Open up Apple podcasts, open up Spotify, open up that Google podcast, give us a five star and a review if you like the show. If you want to send me a comment about what you might like to see on the show in the future, why shouldn't email the factually at AdamConover.net and I will be happy to take a look. That is it for us this weekend. Factually, I want to thank our producers Dana Wickens and Sam Roudman, our engineers Ryan Conner and Brett Morris, Andrew WK for our theme song. You can find me at AdamConover.net or at AdamConover, wherever you get your social media. Thank you so much for listening. We'll see you next week on Factually. Hey everyone. This is Natalie and this is Charlie. We're from the podcast exploration live. It's really funny. It's really good. It's really, really very good. Now we have a YouTube channel to go with it. That's exactly right, Natalie. You can watch full video episodes of our podcast exploration live at youtube.com/explorationlivepodcast. That means that in addition to the audio component, we're also getting a video component. Exactly where you're seeing our reactions, what kind of clothes we're wearing. There's a whole suite of dynamics and physical expressions that you can really only get from a full video. Body language experts to the front. Exactly. Come check out "Expression Live" either audio or video.