Is AI Development Modern Chaos Magic? (w/ Trevor Paglen)

If you've been listening to Team Human, then you've heard about our sponsor, Groucho. The folks who find you people in your area to do dinners and salons with, well, they've turned out to be not only a supporting sponsor of the show, but also a team human utility. That's right. So many team human people have been using the app to find others where they live that Groucho has ended up facilitating team human dinners in cities across the US. And there's people in a bunch of places looking for others to gather with. That's right. Team Human people in Alameda, Albany, Autobahn, West Sussex, Brooklyn, Burlington, Chicago, Dallas, Denver, Grayson, Hutchinson, Lafayette, Libertyville, London, LA, Maverin, Medusa, Montreal, New York, Newcastle, Sandy Ago, O'Wassel, Philadelphia, Portland, Prescott, Richmond, Sacramento, Santa Cruz, SIO, Stores, Toronto, Wichita. So if you're on team human in one of those places, I'm looking to find the others, go to mygroucho.club/teamhuman, use the code teamhuman for half off the first month. And even if you're not looking for team human others, fill out the form and see who the curators find for you. That's mygroucho.club/teamhuman. You're on team human, conscious intervention in the machine. I'm Douglas Rushcough. I'm here today with artist, activist, geographer, author, Trevor Paglin, a hero of mine for a long time, who I happen to get into a reading for his new book, How to See Like a Machine. And was of course blown away. I knew I'd be blown away. I was preconditioned to be blown away, but just to be in the same space. So I made sure I found studio time to actually engage with him about this and everything else that's going on. But Trevor, thanks for being on team human. Thank you for having me, man. I've been in a huge fan of your work for a very, very long time. Thank you. I feel like we've been on parallel trajectory. You were just always in my mind like one of these like super rock star guys that were just like lived in a different world than I did. That's what it was in my mind. And I look at you as in this rock star in a different world than me, I had this feeling, this maybe it's a contiant feeling since I was a little kid that I'm this kind of little consciousness inside this flesh form, like getting certain wavelengths of light through my eyes to figure certain wavelengths of sound in my ears and touch. But then I'm basically on another level, kind of blind, deaf and dumb. Using this like as any Lego robot with a couple of sensors on it, trying to wander around the thing. Yeah. You know, I felt from the beginning. And then I read this thing and you know, I've been programmer, be program, take your own consciousness and all that. But I read this and I start to get the experience of not only am I just trying to get through the day and figure out what is happening in three space that there's a multi-trillion dollar industry hacking that apparatus to make me see something else. The book is really about our relationship to images and what images are and kind of how that's changed over the last two decades, but really much more in the last decade. And the argument is that we've undergrant, we've undergone two very significant revolutions in a very short amount of time. And each one of those is on par with things like the invention of photography or the invention of perspective, you know, really big moments in terms of how we experience perception or how perception gets altered. And the argument is that the first of those is the rise of computer vision. So the ability to essentially teach computers and cameras how to see autonomously. And the second of those is what we might call this generative turn. The ability to create photographs without cameras or without photographers to create texts without authors to create artworks without artists. Like this feels like a real fundamental shift in not only our relationship to media, but by proxy our relationship to reality because we experience reality through perceptions. Right. So the first one is like when camera knows how to find a face or something. Exactly. And so this has huge implications. So the computer vision revolution has a lot of implications. Like first of all, let's just talk about what's going on. So historically, in order for an image to exist, really, you needed a human to look at it. Right? You know, if they images this in the forest and nobody sees it, it was not an image. Well, not really. Right. Because that image is that relation. Yeah. The audience perception. Exactly. Exactly. And so with the advent of computer vision, you don't need that anymore. You're building computers that see autonomously. So what you start ending up with is a whole visual culture and a whole sensorium that is machines making images for other machines and humans aren't in the loop. So what you end up with is a whole, what I call a kind of substrate of invisible images. And I would say most of the images in the world now are natively machine readable, but not natively human readable. So that's like the image of me, like when I go to the airport and they say, look in this thing. Yeah. There's something that's not just a picture. Right. Exactly. Somewhere to these machines. So now they have like a num almost like an MP3 file or a thing that is the way, yeah, the machine readability of my face is like the distance between my pupils or the this or that. Yeah. Which isn't what faces to you or me. Exactly. Exactly. Yeah. Exactly. So you're seeing the kind of quantification of images, you're transforming images essentially to math or numbers. Those numbers are machine readable and you can do stuff with them. Right. That's so that when I'm going to an airport in Iran or something, some other machine can look and go, rush cars there. Yeah. Exactly. On 24. Exactly. Yeah. So this has like huge implications, right? First of all, there's a kind of G-Wiz implication of like, whoa, the kind of metaphysics of images are just really different now, right? That's kind of one layer. A second layer is when you build computer visions systems, how to see, you know, every form of seeing is what in the old days we would call situated, right? In other words, every form of seeing shows you some things and like blinds you to other things, right? Right. And so when we think about like computer vision systems, if you're going to quantify like the squishiness of visual culture, you have to take, you know, a kind of infinitely buzzing and burbling in squishy world and put it into discrete categories, right? And so there's a kind of naïve té that goes into doing that. So for example, if you build a computer vision system to like recognize apples and oranges and bananas or what have you, you know, to look at a picture of an apple and say, that's an apple. And two of the chapters of the book are really sort of playing with that. And it's, and the premise of the two chapters is like, let's point a computer vision system at a painting by René Magritte. It's a painting of an apple and at the top it says this is not an apple. And you show it to the computer vision system and it says this is an apple, right? So you see that contradiction right there exactly. And so we're thinking to through like, first of all, what's that politics of classification? So at another base level, like there's no such thing as apples. Apples is like a word that we use to describe, you know, some cell similar objects, but they don't exist in nature like as a thing, right? We can quibble about that. That's kind of a thing. Yeah. Kind of a thing. Even a squirrel, I think, thinks apple. Yeah. Yeah. Yeah. Yeah. But definitely like, like different animal segment images and kind of like a, you know, monkey understands that you can use it but not just culture, but it's cognitive. It's a cognitive category. Yeah, exactly. Nature is just this big squishy. Smush. She was just in a big swirl of metabolizing stuff. Exactly. So it's a cognitive category. But when you're looking at the McGee painting, for example, or if you look at a, you know, Renaissance painting, somebody's holding an apple, it's not about produce, right? Right. You know, we use allegory. Most of the images that we use are metaphorical. You know, we play with images and we play with language all the time. And that's something the computer can't do. So there's a kind of stupidity there. But then there's a political economy to this, right? In other words, when you say this is an apple, the kind of argument that I make is that you don't build computer vision classifiers to tell you that things are apples. You build them in order to tell you that things are apple-shaped commodities. And what I mean by that is that, you know, if you're going to have something that's measuring apples, you know, that's probably going to be in some kind of industrial agricultural context. You know, apples coming off of an assembly line and the computer vision system is analyzing, okay, this is a good one, that's a bad one, that's sort of that. So it's part of a circuit of capital. But let's expand that out much more today. If you're a truck driver, there's a really good chance. that there is a driver facing camera in your cab that's watching you drive. And if you eat something, it'll say, "Oh, well, the driver's eating something, you're going to get dinged." If you're smoking cigarette, you're going to get dinged. If your eyes are fluttering, it looks like you're drowsy. You're going to get a call from a supervisor. So what that computer vision system does is create a essentially very intense surveillance system. But when you abstract out what's going on here, what you're seeing is a kind of incursion of capital into places where it was previously far too inefficient to bother with. For example, I can set up a sensor in your bedroom and watch you sleep. And I can send that information to health insurance companies. Now, 50 years ago, I could have hired somebody to hang out with you in your house and look at what you eat. Yeah, it's not worth it. So what you're seeing then is almost this geographic expansion of capital into new kinds of places. And that's enabled through computer vision. I mean, it's the kind of thing we used to call it tailorism and it happened in a workplace. So we're monitoring everyone on the assembly line. We know how many units they put up on the thing. We didn't even have cameras to do it. But you know, how many caps did she put on the bottles versus this one? This worker is better putting on more. We get rid of this one. So your surveillance apparatus, that's what you mean by capitalism, can make money or save money for some other entity. Yeah, exactly. And so you're seeing, I kind of argue that you're seeing something akin to enclosure happening or to what David Harvey calls accumulation by dispensation. You're essentially taking places that were outside of a circuit of capital and bringing them into that. Just by putting in cameras. Which is interesting. Because we always used to talk about, oh, I'm not going to be worried till they have a camera in my bedroom. Because I think we correctly, we as humans have enclosed sex as the last frontier that they can't take from us. Of course they have. Pills and condoms and loobs and courses and worries advertising. I mean, you know, yeah, marriage, you know, they're not. I mean, there's a huge industry to make money sanctioned, you know, sanctioning sex. But it's, I know what you mean that once we're in, once we're being watched, it can be commoditized. But just the fact that it can be commoditized, is that necessarily change what's happening? My kids are playing in a park one way. And now they're being watched in that park by parks are us. How does that necessarily change what they get to do? Yeah, well, I don't know that it's changing what they do from day to day. However, it does add up. So for example, a lot of new cars have systems built into them or, or we have insurance companies that would mandate that you put a system inside of your car that does this kind of driving evaluation and real time as you're driving. And then what they want to do is just like modulate your insurance premium in real time based on how you drive. Right? So it would be like, oh, you took a risky left turn. You know, your insurance premium went up by like five minutes, five hours for like that. Oh, you just had a big Mac. You're in exactly. It's how insurance goes up 12 cents and go. Exactly. And so like does that affect your everyday life? Yeah. Yes. Yes, it does because you're thinking about what every gesture that you make, what it's literally going to cost you. Right. It's like, it's like the way you're, I mean, the most people know about it and like in your, your FICO score, I get all these emails now from like three different places telling me your credit alert, your credit score has changed. And I'm supposed to go click on things to find out. Yeah. Like, because they'd go up or down and I'm like, I don't care that went up or down. I really don't. I don't want to borrow money. Fuck all you. Fuck you. It's like this metric, like, but it's the one metric that follows you around. You're sort of credit worthiness. Yeah. But it's like having this increasingly large, like, like real-time report card on me all the time. Exactly. And now it's about everything. It's not just about, you know, whether you paid your credit card bill on time. Right. Right. So the book that I read of yours, or I would have to look at of yours that put you on my radar, was this one that had like pictures of surveillance. Yeah. Invisible. Oh, wow. That's an old, oldie. Yeah. That was the first photo book I put in. Right. But to think about, because the thing I like, because I would just go meta, before I was into politics, I just like meta. So I just liked it from the meta sense of, oh, you're surveilling on the surveillance state. Yeah. And not to me, it was fine. I wasn't like, I was pre-political. I wasn't worried about, you know, all that, all the bad stuff that would happen. I was like, give a little workout. You know, I think I'd hoist it on their own petart, is what I always thought. Karma, we'll get them, you know, if it's capital, seems more powerful than karma so far. At least in the medium term. Yeah. That's, it feels like it, for sure. And what we've been doing so far is, is sort of peeling back the surveillance state. And all the folks we can use that we read about in college, like Foucault or whatever, and the Panopticon. This is like really, really more real. Yeah. It's more real in every day that your Samsung or whatever, LG, you forgot which TV it is, is taking a picture. Is it in a screen cap of your TV every eight seconds and something back to the mothership? Yeah. Yeah. Yeah. Yeah. Yeah. A lot of these smart TVs, they're, you know, taking screen captures of what you're watching. You know, it's like Nielsen ratings like Gone Wild, right? They used to like literally have to pay people to like sell for part of them. But then Nielsen ratings used to be judging the TV network. And that was like fine. At least they weren't judging me, right? Yeah. Now the surveillance is of me to figure out what edge should we give that person, what sounds, what sites, what, what do we do? And that's where we get to the second place, really, the new material in this book is more the, the, the, the, the, the creation of images by machines. How that umchathers us, us to begin with. And then the agendas behind the creation of the images. Yeah. Which, fuck with us. Exactly. Yeah. So that's the, that's the second part of the book is like looking at that generative turn, which is, you know, happens in the context of things like recommendation algorithms, like personalized search, personalized ads, whatever. And so you're, we're increasingly living in immediate environment where the things that we see are custom tailored to us, right? So like us, we got to break it down. So first, the machines are generating images. And machines are generating images from what are they generating images from imagery that's out there that we made. Or are they using their, Matthew machine understanding of reality to generate images? Yeah. So both. So the way that generative AI works is essentially what you do is you like suck up all human media in history, you know, internet, books, what have you, anything that's digital, suck it up. And then you build a model out of it. So you kind of create a, people are going to call me out for saying this, but I'd say like a statistical real representation, you know, you create a representational space of all that stuff. And then what you can do is you can prompt that model to make images. So you can say, make me a picture of a teddy bear and an astronaut suit, you know, swimming in a lake of cheese. Right. And then you can make that image. So it's synthesizing that image based on visual concepts and linguistic concepts that it is quote unquote learned by ingesting the entire internet and the entirety of culture and putting it into this abstracted mathematical space. And making this thing. Yeah. Yeah. Exactly. And prompt the model to make something for you. Now in real life, most people are not prompting models. People are seeing these kinds of images or media in a feed, right, or in the Instagram feed or TikTok feed or what have. And so there's another layer that's going on there, which is that feed is kind of trying to optimize what is going to hold your attention. Right. What, you know, let me show you something where I can harvest something from you, whether I can either manipulate your political beliefs or I can just get you to look at something for a long time or I can get you to buy something, you know, behind what's what's creating attention is the simplest one. Even though attention doesn't necessarily create money in the moment, I guess attention is good because you've learned how to get that person's attention. Yeah, you've learned how to get that attention, but it does translate into money. You know, you get into this world of, you know, paying advertisers and that kind of thing. Right. So what's the real difference between, you know, Donald Trump hiring me to make him look like Spartacus or Jesus and putting that on Twitter and him getting an AI to do it. There's I mean not really a difference, but the trump thing is super interesting my intuition is that when You start to be when you start prompting images like that into reality that relationship between Prompting a model and prompting reality itself starts to get squishy in other words to me the When trump is tweeting out those AI generated images They're there something else going on then Propaganda I didn't think I'd like getting sponsors for team human, but turns out the ones Who actually want to sponsor us? Totally get what we're about like groucho which uses technology as an antidote to social isolation Handling all the logistics for twice monthly gatherings between you and a hand curated cohort of collaborators It's as if groucho were Informed by the ideas we share on this show from face-to-face interaction to bottom-up community to localism to the value of eccentricity over conformity Groucho curates people as if they were being selected for an artist's colony not clones But compliments and contrasts ingredients for a collaborative stew They also take care of all the logistics from dinner reservations to conversation starters Just go to my groucho.club/teamhuman for half off the first month answer some questions about yourself in your own time And let the humans at groucho do the rest. That's my groucho.club/teamhuman Use the code team human for half off the first month and enjoy your dinner and new friends Right, there's a propagandistic element to it, but it does seem to me that there's a kind of alteration of our common sense that that induces in society and it feels very much in this kind of post simulation kind of world you start to have a society where people have very different facts people are oriented themselves to reality through very different kinds of images and because we orient ourselves to reality through images we're getting more into this world that feels like simulation or chaos magic or these almost kind of theological traditions that where what reality is and what images are so entangled that it's hard to tell them apart. Yeah, I mean I've been writing about memes and media viruses since you know 1993 or four and sigils and all but I never really and I've called memes sigils but I don't think I ever really meant it until now and and what helped me the most in your in your book is distinguishing between images as representation like the meme it's a picture it's bar Simpson on a thing with Trump of the book and memes are at and images as activation yeah, yeah, you know where and that and 20 because activation is actually really true to my original objection of a virus exactly thing, but you mean an activation of the like the nervous system or the memory place or yeah that's one of the other arguments of the book is that in these kind of post computer vision post-generative AI moments how do we think about images and the arguments that I make is that we should stop thinking about them as representations and thinking about them as activations in terms in in other words like let's stop thinking about them as things that tell us stories and start thinking about them as things that do things and I'll give you some background like you know what I went to art school when you're at art school you have to learn how to like quote unquote read images right and you read people like roll in barred and Susan's on tag and it's very much like oh we're looking at this advertisement and you know here is this story about this person blah blah and it's kind of an ideological story about whatever capitalism exactly exactly exactly the image never returns to conditions on the ground exactly when we have to decode it in other words say you know there to see the story that the image is really telling and like for me that's just like not what's going on with images like I'm scrolling on my phone and I'm looking at somebody eating spaghetti and I'm like captivated I'm like what the hell is going on here this is not you know like rolling bars some kind of you know ideological no it's like it's no longer it's like out of the realm of Freud and into the realm of like have lovins exactly exactly exactly and so that's that's the the argument that I'm looking at is like we can think about you know a camera on a street corner that is looking for people j walking it can automatically detect somebody j walking that trigger that activates a legal system that will issue them a ticket right right and in the same way I'm on TikTok I'm looking at somebody eating spaghetti that's activating something in me neurologically that is holding my attention you know it's doing something to me almost like precognitively or pre rationally right right I know those are two really interesting examples right just when you think about them almost like poetically or visually artistically one is human body moving through space being observed and and punished for the way it's moved along lines by a machine that doesn't know what a person is yeah by a machine that has observed a form that it has labeled as a human body crossing a line at a particular time when some other thing was there right it doesn't it doesn't care yeah exactly it doesn't even know right so there's that going on and that machine because of the algorithm that it's being taught which is the legal one you're not supposed to cross from the thing whatever that's fine if you if you sign up for that is affecting your life then you're going to get a thing in the mail for fifty dollars to pay you've got to work tomorrow is at the factory whatever it is to give money so Donald Trump can give it to a son-in-law or something right or whatever he does whatever he does or give it to a January 6th protester that's what's happened the other one now is the image itself whatever it may be a picture of the computer doesn't care what it's a picture of the computer cares that this arrangement of pixels will get you to feel or act in a particular way yeah and it doesn't matter why so traditionally it would be oh let's make that sandwich look really tasty so he wants to go to McDonald's or whatever and get that thing yeah or he'll she wants to eat a mounds bar right now it's more whatever it is it's going to neurologically we don't care move that body yeah over to that place exactly exactly and you know and this kind of thing's been around for a long time I mean it was that there's the book in nineteen fifty nine the hidden persuader yeah we turn out to be fake I mean that was it's her not fake they weren't really yeah there was one experiment where they tried once using but no one was actually putting flash frames of coke in the right yeah yeah yeah yeah yeah yeah yeah yeah yeah to the to the stand to buy it yeah there was that subliminal thing but there was but there was like market research and correct me if I'm wrong that would show like if I gave you the same detergent in three boxes like a yellow one a blue one and one that was yellow yellow and blue you'd perceive them as being different detergents oh yeah you would have a preference of one oh yeah but they put the little um they put it in the detergent those little blue or yeah exactly granular so you think oh blue is this a cleaner one right yeah and different kinds of people like oh you know they'll find out like oh you know um black people like a yellow dot in their thing wealthy white people like a blue one and middle class white people on a green one yeah it's like what the fuck but whatever right but like then okay so those dynamics have been around for a long time but they've been really you had to spend a lot of money to figure that stuff out and like the in this kind of in the environment that we're in now where you're in a constant feedback loop with the media that you're consuming you can you can individualize that for every single person you can modulate that in real time you know what I mean and you're getting behavioral feedback in real time right so it's like that those kind of dynamics are crazy and what you end up with this is much more powerful like neurological triggers right you're in the Skinner box I mean yeah and you could get there the other way I mean like Coca-Cola developed a bottle that just it was a human but turns out to be like a perfect sexy wonderful swirling shape that thing is like the Chrysler building it's just the original glass coat it's that right and now so it's like it's almost like industrial branding we should speak in the 1920 with that bottle and since then we've been trying to imitate that effect through massive consumer research and like we're not trusting the human soul or heart to come up with that we're not exactly using probability right and we're like understanding humans to be like neurological systems in a way that are akin to computers now I'm not saying that that's That's what we are, but there are parts of us that are. There are parts of our cognitive apparatuses that we have very little control over that do operate per rationally. Right. Right. And you call this, I think you call it the Dorito effect or something like Doritos, which is like how you came up with Dorito. We all know Doritos are really just awful and they're not really satisfying, but you'll eat the whole bag of it there because it's like, so does living in a Dorito image landscape do we end up losing our kind of non-Dorito sensibility? Can we not see anymore? Yeah. This is where we get into what are the kind of big implications of this stuff. And to many people, the implication I just said is who gives a shit? Well, you can get out of the Met and appreciate a totally year or something, whatever. No, I mean, that's never going to go away. Yeah. We're complex things and we're complex societies. And when the book we're just looking at some tendencies that have gotten a lot of rocket fuel behind them in the last few years, but there are always going to coexist with other things. But so there's a bunch of implications of this, like the Doritoification of media. Before we kind of go into that, I think one thing is a much, much bigger deal than we think that it is. And that is the creation of images that look like photographs, but that are not. Right? We'll say like the idea of democracy in my understanding of it is kind of premised on a modern idea of with a kind of faith and empiricism. Right? For me, the theory of democracy at the highest level, it doesn't ever work this way, but at the highest level, I understand the theory to be that if we have a set of shared facts that we get, or we're going to determine through some kind of empirical process, we can look at those and deliberate on them and kind of figure out what we want to do going forward. Right? Now again, that's not how it works, but that's the theory. Right. My intuition is that photography has been a big part of that, of the sensorium of democracy. If we want to put it that way. We have lived in a world where the image of the napalm girl came out. That was the image from Aboujade came out and nobody disputed that those things just happened. And they changed his public opinion. Yeah. It changes public opinion or at least put something on the table that needs to be addressed. Are we really going to torture people? Are we really going to like fire bomb kids? Whatever it is. Right. I think that era might be over. And I'll tell you about a real weird experience I had. I knew this was coming when I wrote the book, but it hadn't happened completely for me yet. The day after the US star bombing Iran, I was looking at Twitter. I'm trying to figure out what else going on here. Okay. That's a good place. Yeah. You know, I was like, where are you going to go? New York Times, I mean, you would pick a poison. And I realized that instead of kind of going through the feed and trying to throw out stuff that I knew was obvious crap, I was trying to find things that might be credible. And I was like, whoa, a flip has happened. Where my default assumption is that everything here is some kind of garbage just trying to manipulate me. And I'm looking for, I'm panning for gold in the form of information. And I started calling this moment the indexical flip where I am assuming that all meat is untethered from reality. And I'm looking for the medium ones that are tethered to reality. And that's weird. And I think that has huge implications. Right. That's an awful experience. I mean, that was what I was writing about in a phenomenon like that. In like 95, I did this book out playing the future where I was arguing that attention deficit disorder among kids is a cognitive defense mechanism in a world where someone is trying to program you everywhere you look. So you learn to withhold your attention from anything until there's something that you feel semantically safe enough to gaze upon it. Yeah. Because it doesn't want something from you. Yeah. Yeah, exactly. But it's odd. But that means we're also moving around in a state of either cynicism or disorientation. Yeah. It's all, I mean, it's like, well, if you're driving and you do not believe this street signs, totally, exactly. That sucks. Right? Not only does it suck, but like you're living in a really nihilistic society. Right. And what's going to fill that void is raw power. You know what I mean? Like fascism. Like bad stuff is going to fill that void. Right. Because if you can no, if you no longer have the means of deliberation, then let's go full Viking. Totally. Let's go full Viking. Like, where does that, where does that sense of reality come from? Right? If it's not coming from-- For pain or not pain. Bingo feels like I, or it comes from pain or not pain, or I want to believe something or something feels real. You get into this much more kind of affective kind of space. Right. But Spinoza said that the object of the game here is to increase our capacity to affect and be affected by others. Yeah. So it's an affective, somatic, almost--it's compatible with reason. But it's not depending on reason. Yeah. It's like, is the answer some kind of more somatic embodied, intuitive thing? Yeah. Like, obviously that's always been there. And that's been like a, you know, one of the--my understanding is like the point of reason was to be able to get over that and like, deliberate from a higher space, like with a different part of your brain, like the rest of your part of your cognitive aspect. But the other one has to come along. So it's like, all right, we're in a town. We're living together. The water table basin thing is starting to go down. Right. Oh, the water basin going down. The water basin is going down. And that we're going to deliberate about it. If I'm still in--water basin going down. Water basin is going. We're not going to be able to do it. Or like, what if your neighbor says it's not going down? Well, that's even worse. That's what I'm talking about. It's a now, right. So you want to disconnect enough from the, not disconnect, but you've got to get through the panic to actually be there, metabolize it, and breathe it. The water basin going down. If, right, if you don't even believe that it's going down, because we're not there directly looking at the water basin. We're reading a newspaper that's telling us whether it's going down. And you're reading the conservative or fake conservative newspaper that's saying it's actually--it's going up. Exactly. It's going down. It's because Gavin Newsom closed the faucet. Yeah. That should be feeding you. Right, totally. It's like, oh my god. And millions of people believe that. Yeah. That the Southern California fires and droughts are caused by Newsom holding the water up north. I'm like, ouch, ouch, right. So if you don't--and what you're saying, our literacy was supposed to be the tool for enabling democracy. Yeah. The argument to have public schools in England was so that the coal miners could read a paper at night and vote effectively. Yeah. If that's one piece of it. And then the other piece of it is the creation of a shared reality through empirical means. In other words, we're going to go out and look if the water's going down and we're all going to agree that the water's going down because we measured it. Right. And we get experts. We get real scientists with real rulers who know how to measure it. Who now people say, oh, the scientists are funded by George Saras' space lasers and they're not real. They're aliens. Right. Yeah. Exactly. You know, this has big implications. Yeah. Right. This has really big implications. Yeah. And so where's this going? Well, I don't know, but I'm worried about it. But I do feel like that kind of naïve take on--like, we're going to have to update what we think we're doing here. We're going to have to update the basis through which we conduct politics. If we don't, it's going to be updated through, you know, Trump means of him. Right. And I don't think it's--and I hear from a lot of people, oh, we've got to like upgrade humanity or-- I don't know. I don't think it's that. I think we have a cultural social and economic institutions that were developed in the 12th and 13th century by people who didn't even have our best interests in heart then that are truly obsolete and unworkable in an age of AI and image creation and all. And we created the foundations for a society where sociopaths win. Yeah. Exactly. Exactly. And so that's--that's the fear. You know, and that's-- Well, it's not the fear. That's the reality. That's the reality. Irony is--at this moment where our relationship to images and media are foreclosing certain older forms of politics, this is a moment when we need some in form of politics most in order to figure out what we want our collective relationship to be to this changed present, right? - Right, and that's why I keep going to the extreme local because at least you can kind of believe your eyes there. And it's, I don't mean it in terms of cynicism, but I've done so many sort of monologues where I'll say, look, why don't we take 99% of us can just be as nice to everyone else as you can, just do that. And we'll assign like 1%, even 1% of Americans is what, 4 million people, but 4 million people deliberate on what's actually going on here. It's probably enough, right? And the 396 million of us, if we do all this other stuff, if we take care of each other, then if FEMA's 15 days late, it won't matter as much because we know where the sandbags are and we're more resilient with each other. You know, and it's not like la la la, but it's like maybe stop worrying about that so much. But there's other policy things you could do. Like you could have like comprehensive online privacy legislation that would like really foreclose a lot of this stuff from happening. - Right, you could have. - You know what I mean? - But you could have, but just, and I guess what does that mean? Does that mean organizationally that we on podcasts or telling people that your congressmen know that we want this thing? - Yeah. - You know, I mean, do we make a list? - That's the piece of this that's tricky. Because this really needed to have been done in the early 2000s, because we've gone so far, you know, crazy with data collection, which has enabled things like, you know, the targeted ad industry to go crazy, like the kind of like algorithmic addiction industry to go crazy. And like honestly, a lot of the AI stuff, you know, wouldn't really exist in the same way without the harvesting of, you know, basically everybody's data. - Right. And now it's like the best you can even think to do is to get, you know, social media companies to use different gentler algorithms on children under 12 or something. - Yeah. - Yeah. - You know, so we're kind of like, and we're in, Congress is at best in the social dilemma of this, not exactly. - Yeah, exactly. Most of our listeners already believe and understand in the mechanisms through which this happens. - Right. - The story, you know, of the kitten that's raised in a thing with only vertical lines, when they take them out, you can't see horizontal lines. - Right. - That we understand these kinds of things, so then as we're raised in increasingly algorithmic landscapes, we may not, I mean, that's the thing we may lose certain abilities. I mean, when I used to write about this, it was so quaint, I wrote about it from the aesthetic perspective, because I had looked at these experiments that were done on kids in Germany. The kids were raised only listening to digitally recorded music, and MP3s and stuff. And there were orchestral tones and subtleties that they couldn't perceive anymore. So you play like a Deutsche Grammiff von Record, the 60-year-old can hear, oh yeah, this is different than that. The kids are like, sounds exactly the same to me. - Oh, interesting, because they had like always, they'd grown up listening to compressed music. So there was like frequencies and things like that, that just were not in the music that they heard. - Right, and subtleties, because, because I'm in the way I used to describe it, is that an MP3 is actually not music. An MP3 is a digital symbol that's been crafted to make the ears think it's listening to the music. - We are very much in the books territory and here, in terms of thinking about how computers make images. Like as you were saying before, the computer has no concept of like, oh, here's an astronaut with eating cheese whizz on the beach or whatever it is. It's arranging shapes in such a way to make you see that image. - Which is, as you show, and book and what we learn in first year film school, is the way the brain works. - Exactly. - You don't see a fire truck, you see, there's red, there's bell, there's ladder, and then my brain assembles fire truck. Like, exactly. - 90% of that image is coming from me. - Exactly. - Exactly. And so this is where you get into these whole lineages of like magic and siops and like these forms of media that proceed from that premise. And like obviously a ton of film is built on that premise too, like especially montage techniques and things like that too. - Right, but even regular magic. Like we know how the eye sees and also when I do this and I do that, the brain will not see the coin going from this finger to that hand. You actually can't see it because I've manipulated your seeing apparatus. - Exactly, exactly. And so those kinds of techniques are really interesting to me because there's a theory of media there that is designed around your own cognition. Right? In other words, like the, I'm not telling you a story. I'm producing a neurological effect that is gonna cause you to perceive reality in a particular way. And it feels like that to me is much more the paradigm of what's going on like an AI system than an author sitting down and trying to tell you a story. - But just to call attention to it. So because I've been doing my little semantic experience of reality these last few months, as you talk about that, I start to feel lonely. - Why? - Because we're not in our shared reality anymore. We're not, I mean, that I'm sitting here, again, I'm in this body being injected with cues in order to see, but it's alone, I'm utterly alone in this thing. It's like each of us, each of us is alone, being very matrixy, being played by this around. - We'll be, we'll be, we'll be playing with that to the round, but we're also being played by our own cognition and a lot of what our own cognition wants us to do is be sociable, right? - So there's enormous parts of your brain like that is just geared to dealing with other people and reading other people and getting along with other people and being able to function socially. That's a key to the problem. - Right, and then I apply that to my AI chatbot, trying to, I can't help it. - Yeah. - Oh, I want it to like me. - Oh. - Well, this is the trick. I mean, these are, another thing I think is like very predatory and very dangerous about how a lot of AI systems are set up is to mimic those forms of sociality and twist them and use them to manipulate you. For example, you know, you're writing, you know, article or something like that and you might put in a chatbot and say like, "Hey, what do you think about this argument?" And it'll be like, "Dug, this is genuinely ground breaking ideas you're working with here." And you know it's bullshitting you, but I guarantee you, it still feels good, right? To be told, this is really, you're really smart, Doug. - Yeah, but no, it means it hasn't, it didn't get the prompt I've gave it at the beginning. - You have one of your, it's over, right? - Yeah, I put in this thing saying, do not do that, do not do that. Yeah, I mean, there is that, there is that, "Oh, you're right." I was gonna get into the greater key of Solomon. - Okay. - So surprised you brought that up. (laughing) Okay, I've been thinking about that, right? So you just kind of traced this lineage from Renaissance to the Sicijoles and greater key of Solomon to contemporary normal triggers. - Yeah. - So do you look at contemporary AI developers as kind of modern sorcerers, summoning operational forces to bind and bend reality? - Not only would I describe them that way, they would describe themselves, though. (laughing) - No, it's really weird. - Really? - It's so weird, like if you hang out in California a bunch and you talk to people who are working with cutting edge technologies, and let's take two examples. Let's take like AI researchers and we'll take that biotech research. - Okay, right. - You talk to a biotech researcher, the most rational person you've ever met in your life. They're like, this is this and this enzyme is gonna go here and it's gonna trigger this and, you know, they're very clear understanding of what they're trying to do and how they're trying to get there. You talk to AI researcher and they're like, we don't know what's going on, we're just putting data and compute into the cauldron and like demons are coming out and hopefully they'll be benign. Right, that's really the kind of language that you get out of. - Much closer to Grant Morrison than Alan Moore. - Yeah. - It's like chaos magic. I'm like, yeah, he, I don't know how it works, but you take the words, you draw them down, you do, you jerk off on it and you burn it, and we're shit's gonna happen. (laughing) - No, it's like literally like that. Like a lot of those guys are like totally into things like chaos magic, you know, even specifically chaos magic. - Well, there was even a character on Silicon Valley, the comedy show. There was that one sort of a cult. - Right, yeah, yeah, yeah, yeah. - Yeah, yeah. - No, but it was like, it was, it was, it had a bunch of people there recommending me to read Ramsey dukes, you know, like SSOMTB, you know that book. And I was like, what are you talking about? And I read it and I was like, well, this is intense, like the world that these guys living in. who are on like cutting edge AI development work 'cause it really is much closer to chaos magic than it is science. - But that was supposed to be our world. - Exactly. - Yeah, I mean, I was mad enough that Trump's using Norman Vincent Peel and power positive thinking and they did Keck and Pepe and all. It's like and discordinism and pranksterism. It's like that was our side, Robert E. Anton Wilson. You know, Paul Krasner, Abbey Hoffman. - Yeah. - So on some level, I feel like we kind of started this. We meaning our lineage kind of started the, but it was part of the objective. Optical illusions and data ism and pranksterism. You loosen people's grip on reality in order to help them see other possibilities. Not in order to inject them with, - Right. - Yeah, exactly. - That's the kind of irony of this stuff is that like, you know, I wasn't around in the 70s or the 80s that much, but my understanding of it was that these were ways of trying to unglunk rigid ways of thinking about what society should be and what the possibilities within society should be and gave rise to things like feminism or queer activism like kind of radically opening up the space of what was possible in terms of like how you wanted to live your life or what counted as, you know, a human, a human with rights basically, right? And you do see there is a kind of like other side of that coin, which is, you know, like the, the weaponization of that or like the using that towards the purposes of either consering into existence, like very dark realities or nihilistic ones, you know, which are kind of very related to each other. - Well, once you know that these things aren't really real, denialism sort of the only place to go. - Exactly, and that's the old, that's the Soviet strategy, right? - Yeah, I wanna be there. - We don't wanna do that. I wanna be a weird, happy, strange, experimental baby. Not, you know what I mean? Man, I want to go, man, I don't like that one. So I read the book, I read the PDF of it and I finally got to where I thought you were going when you were, well, you could only talk for half an hour, whatever you could do everything. But you started to go to this place that reminded me of the impulse I had when I wrote this book, "Programmer Be Program." So I was really freaking out about our transition to digital that since about '98, '99, when I saw that we were, instead of empowering the collective humanity to dream a more interesting dream and create a new reality together, we were going to use this stuff on people in a feedback loop in order to make humans more predictable. And I was like, oh my God, oh my God, oh my God. And the, oh my God, part went, this is what Torah was about. Yeah, right? So Torah obviously right now, seemingly, may have been a failed effort at developing an ethical code for humanity, but what they were trying to do, they were looking at, this is a civilization that is transitioning from an oral world to a written world, to a scriptural world. And things are gonna get really fucked up if we don't take some real steps here. So you can look at Torah law and the Ten Commandments and all that stuff is saying, okay, in the world that you know, you make a deal with somebody, you agree eye to eye and then he's gonna be their next season. Yeah. Now you're moving to Torah world where you're gonna write something down on paper and the person you signed it with may be gone and having foresters there. How's that gonna work out for you? Now you're gonna, instead of history being something that a grandmother tells her child, history's gonna be this thing in a book. But you're gonna be able to point to and say, how's that gonna work out? So all these things are gonna change when you write things down. You're gonna lose the personal and move into this symbolic thing. You're gonna have these magical pages that they've been. So you develop a thing and I was thinking, oh, well, what's the equivalent of Ten Commandments in this transition? This transition to the digital is as big as that one. So I did program, we probably had Ten Commands for the digital age and looking at the equivalent of prohibition on images. But we weren't actually there yet. In some ways I wrote that too soon 'cause I was writing it, although it ended up being good enough for now. I just republished it 'cause it is. I was writing about AI without knowing AI. I was there. I was writing about what is it going to be to live in a world where things are programmed to change based on what you do. Yeah, in the sort of cybernetic feedback loop then he knew it back then, Norbert Wiener knew it. Totally. This, exactly. But now, are there things we can pull from their insights back then to apply now? I mean, I think a lot about that Second Commandment and think about it through the Old Testament but then I also think about it through the Nostics, right? And Second Commandment being prohibition on images. Right. People think it's like, oh, don't make idolatrous, that's not what it says. It says no images. I know. People sleep on that. It's really weird. I know. I know. We always want to soften it. Just something else, but it really is. Don't put images. You put an image not on your skin, not on the books. Right. Just don't do it. No graven images are anything that isn't heaven above or the waters beneath the earth or anything. They're pretty strict with that. It's pretty strict. Because they're really afraid that I think they're afraid that images come to life, which they do. Right. They become idols. Yeah. They become idols. They become sigils. They become, they affect reality. Right. They alter reality when you make images of it. Right. But I mean, we know this, but like it feels supercharged to me. Right. And it's really interesting to me. You know, I have a whole bit when I do kind of a performance around the book where I talk about that Second Commandment. I'm a higher on the list than murder. Right. I know. But then what does Yahweh do right after that? He's like, okay, no images. And like here's an image. You know, it's going to be two qubits by three qubits. It's the arc of the covenant, right? It tells them make the arc. Right. But the arc of the covenant that he wanted for the Israelites had no thing on the top. It had two cherubans protecting the empty space on the top. Yeah. Usually you'd put a statue of a god up there. Exactly. But there's still cherub 'em. He says, "You have the wings and they've got a whole full full." And what's, I think I find so interesting about that story is that then they make the arc and it works like at first. Right. So it like parts the river Jordan. It like tears down the walls of Jericho. Right. And the Israelites start thinking that this image is like their, you know, secret weapon. Right. And that goes on for a while. They're in a big war with the Philistines and they're getting clobbered and there's going to be a battle. And the Israelites are what do we do? It was like, "Okay, we'll bring out the arc." Bring out the arc, man. It's our aim. The Philistines know about the arc. They're afraid of it. They're like, "Oh shit." You know what I'm going to do. So the Israelites, you're sorry, the Philistines charge the Israelites with chariots. And the Israelites raise up the arc. You think legs are going to come out like Indiana Jones or something. And the Israelites get slaughtered, right? 30,000 people died with the Battle of Afec, I think, 1050 BCE, something like that. And to me, that's such a crazy moment in like just theologically like what's going on here is this a lesson about not having faith in images to do what you want them to do. Is this a lesson about there being a base reality under images that you should not confuse with the world of images? I mean, there's all kinds of theoretical. Right. What is it the pre-telling of the story when finally, you know, 600 million people from the Southern continents have made their way to Larry Ellison's fortress. And he's projecting paramount Yellowstone in images. And they're like, "We don't care. We're going to eat you." Exactly. Or the floods are rising. Global warming is like, "Oh, no, we don't care whether you think that this virus is real or whether you think that these glaciers melting is real." Right. I'm still going to happen. Right. So you get that. But then you get this whole gnostic inflection on that whole story, though, too, which is you get the story of the Demi urge, right? And some of the strands of gnosticism that God of the Old Testament is a demiurgical God, right? I mean, the Yahweh God of thunder. Yeah, exactly. Yeah. It's the Demi urge is, they had other names for it like Samayel, Yaldaba, Oat, but in some of those traditions, Yahweh is a demiurge. And a demiurge as a God means what? The demiurge as a God is a God that creates images and then believes that those images are the same thing as reality. So it's basically a God who may make simulations and does not distinguish simulation from reality. God makes heaven in there. God made heaven in there, put God parts the red seat, right? That demiurgical God can do whatever it wants because reality as we perceive it is an illusion, right? And the God is in charge of that illusion. And so in those traditions, the God of the New Testament is a different God and the point of your enlightenment is to break through that world of illusions, right? And find this higher form of knowledge. This is in lots of different traditions, though, too. I mean, this is the Maya, the concept of Maya, you know, from the thing that's in the Vedas is very similar in that sense. This is a theme that records all over. So because he's a demiurge, I've been thinking about the demiurge a lot because Trump feels like a demiurge. You know, a lot of political figures feel like they're applying a demiurgeic logic to reality. And like the arc when it first came out, it does work a bunch of times until it doesn't and then when it doesn't, you're in real big trouble. Right. And AI is a, in some ways, is a tool for the demiurge style. Yes, AI will make you a novel that nobody wrote. AI will make you all the images that you want that have no relationship to base reality if we want to call it that way. But is this kind of world of pure simulation that we experience as being real because we've been conditioned for, you know, thousands of years to know that if I saw a text, somebody wrote that text. You know, if I saw an image, somebody made that image or a camera made that image and it was tethered to some light that, you know, bounced off a sensor somehow, that created a kind of common sense about how to understand reality that has been upended now in ways that I think are quite profound. You've been modeling for me something that I'm trying to do myself, but I feel often conflicted about, which is I'm trying to figure out what's the appropriate path for an artist, writer, activist at a moment as dire as this. I sometimes feel like doing the fun art, writing a graphic novel, even if it's going to open 12 year old kids, it's to what's happening. Feels like a self-indulgent diversion in the face of imminent disaster. It seems to me your answer is to you mix it up, you do some activism, you do some intellectualism, you write a book, but then you do a crazy photography project. I mean, is that sort of your, your, yeah, that, I mean, that's my approach. You know, I think we need a lot of people doing a lot of different things, right? And so I actually believe in art, I believe in what I do. I mean, I have to do that. I mean, it's something I'm very old fashioned. But I think that those of us who are cultural producers, I think we sometimes poo poo the significance of what we do, but I do think that there, that we are in the business of telling stories, making images, creating ideas that we collectively can use to try to understand the moment in history that we're living in. And that's, and that's actually a really important thing to do. And that's kind of a different thing than science. And that's kind of a different thing than, you know, party politics or that's even kind of a different thing than some kinds of activism as well. That's certainly a different thing than policy. But you need all of these things to be talking to each other and working together in order to, first of all, understand the conditions under which we're living and then to imagine the ones that we actually want to exist within and then to implement those. I like that. Team human can have a lot of different kinds of-- We have to. Yeah, I always get some more hearts. Yeah, no, I always feel a little bit bad when people are like, oh, well, this is frivolous and that's not, I mean, like what, what it, the beauty of humans is we can do all this stuff, you know, and we kind of need to do all this stuff. Right. And that's why it's great. There's a whole bunch of us. Well, thanks for being on Team Human Man. It really, it means a lot. It's a pinnacle. Thank you so much. And wonderful being here and it's been a lifelong dream to be able to talk to you, man. Oh, thanks. Let's be friends now. And you, everyone in the meantime, get this, get this. How does see like a machine, but we're afraid to ask. Oh, by, that's an old reference. How did you introduce them to Trevor Paglin? It's short, but that's a good thing, not a bad thing. Wouldn't you rather pay for him to take more time to shorten it for you? How many countless hours did he save working those extra hours to shorten this book? It's a blessedly brief, but deeply provocative and ultimately finally reassuring take on our increasing disorientation. You've been on Team Human. You can support this show by going to patreon.com/teamhuman. Do anything you can. It helps. Look at us for a beautiful studio because if you can't do that, click on things, there's hearts, there's checks, there's subscribe. All these platforms have all these things. They're watching you. Sorry, but click on them anyway. I do love you. I'm Douglas Rushkopf. Team Human is produced by Joshua Chapdillon and edited by Luke Robert Mason. Our last best hope for peeps. [Music]