Petter Törnberg on the Dynamics of (Mis)Information

Hello everyone and welcome to the Mindscape Podcast. I'm your host Sean Carroll. There's an idea in social science circles called Physics Envy. Economics especially is susceptible to this idea. It's not supposed to be a good thing. You're not actually supposed to be Envy-as-of physics, but social science is hard. People are messy. There's a lot of variables going on. Physics is able to make enormous progress by simplifying things. A great deal. In part, that's because the fundamental ingredients that we study, even though it's like quantum mechanics and cosmology and relatively other things that sound way out, but there aren't a lot of moving parts. The basic things that we're looking at are sufficiently simple. You can describe them using relatively few variables and you can isolate all the interesting things that are going on in these systems with small numbers of variables. As a result of this, you can make tremendous progress. You can prove theorems. You can do experiments. The test, your theories, to many, many decimal places. It's a lot of fun. Of course, people would be envious of this, but it's a disease or at least something to be avoided to therefore try to make your social scientific research too much like physics. When you do social science, you should admit that there are complications there that cannot be abstracted away in the same way that we abstract away air resistance or friction when we're doing physics. Nevertheless, I'm sure that everyone who listens to mindscape on a regular basis knows I do think that there are contexts in which physics like reasoning can be helpful or even interesting in the social science. There can be contexts in which physics type of reasoning and concept borrow from physics can be really useful, very interesting in the social scientific contexts. Ideas like equilibrium, ideas of emergence in general, ideas of what is collect behavior like when it arises from the sort of mindless, non-directed interaction of many small things. These are things that physicists think about all the time and are very, very relevant to the social sciences. So today's guest is Petr Turnberg, who is a professor of computational social science. I promise I didn't know this, but he admits on the podcast that he actually has a physics background, so this makes some sense. But he uses models, agent-based models that we've talked about recently with Don Farmer and others, as ways to study the behavior of social systems. Can you make a little model where the individual pieces are either simple agents that always act in some way, or maybe there's a little bit of stochasticity in there, or maybe they're even very complicated. We'll talk about an example where Petr used LLM's large language models to model human interactions in social media landscapes. And then you can ask what is the robust behavior? Do you get things that we observe in the real world? Do you get polarization? Do you get sort of an accumulation of influence in certain people rather than having it be completely uniform? Is this good or bad? If you intervene in certain ways in the social medium, can you make things better? Is it all the algorithms fault? Or is it just the preferences of the individual actors? All of these kinds of questions can be addressed with this sort of slightly-physiccy attitude, but put into a social science context. I don't want to give away too much about Petr's results, but they're not great. They're not very encouraging for those of us who want social media to work. There are certain natural dynamics that seem to come up that drive things in a bad direction, that drive polarization and echo chambers and things like that. It doesn't matter whether we like it or not, if that's what happens, knowing how it happens and that it happens under certain circumstances hopefully will be helpful in making social media or media or information ecosystems in general more functioning for the purposes of democracy, but also for the purposes of just learning fun new things and having a good time and being good human beings and connecting with our peeps in various different ways and building connections that otherwise wouldn't have been possible. So we want to keep the good aspects of these wonderful technologies without being subject to the bad aspects and I think this kind of study helps us figure out how to do this. So let's go! Petr Torenberg, welcome to the Mindscape podcast. Thank you for having me on. It's a real treat as a a long time listener and fan of the podcast. It's really great. Okay, very good. Well, then you know how it's going to go. Let's just start setting some stage a little bit. In one of your books you have this provocative line, "Power has an epistemology." Do you remember that one? That's from seeing like a platform. So what does that mean to those people in the audience who might not use words like epistemology every day? Yeah, so it's a good starting point, I think. So a lot of my research is kind of informed by this understanding of society being a complex system. So I kind of, I actually have kind of physics background myself, but like long time ago, so don't don't quiz me on that. But and come very much from the kind of complex systems perspective. And in my PhD, I was kind of focusing on taking that perspective to try to see how can understand society using the methods of complexity science and using computational methods. And so this this book is basically centered around this idea. Okay, we're seeing this notions of complexity becoming used throughout the social sciences, but also in kind of urban planning, you know, and also in kind of when we talk about digital platforms and how they're shaping society. And so we try to kind of understand what does that actually mean. And so I mean, in the social sciences, the way the complexity tends to be understood is, you know, like this bottom-up system. So often you have like, you know, there's complicated systems like a car or a spaceship that you can kind of take apart. You can decompose them. You have like an engine. And it's quite easy to figure out how they fit together. You can kind of understand the system by taking it apart. But then on the other hand, on the other side of the kind of this line, you have complex systems. So like anth colonies or flux of birds or whatever. And these systems, you if you take them apart, you know, like you take out an individual land from an anth colony, you can observe its behavior as much as you want. But it won't tell you very much about how an anth colony functions because the kind of intelligence of the anth colony emerges through the interaction of a lot of ants, right? And so the book kind of stems from the observation that we've shifted to more and more talking about society through this complex lens. And that it's kind of intertwined with this notion of new forms of democracy because obviously a lot of those ideas come from kind of from, you know, from your field, from physics and from computer science. But as they're entering, they're changing quite a lot as they're entering into the social world. And they also begin to have kind of political implications. And so those are the kind of implications that we follow. And ultimately, in the social sciences, this becomes a question of kind of epistemologies. So the question of like, kind of how do we envision what society is? And we went from, you know, in the 60s and in what we use social scientists, we referred to as fortism or industrial modernity, where we tended to see societies as a machine as a, you know, as a complicated system. So basically that way of seeing and way of understanding society stemmed, we argue, from a society built on large industry, large mass production. And they led to kind of a mass society. And it led to the kind of ambition of like, we can design, we can plan society, we can build it as if we were building a machine. So this is fortism as in Henry Ford and his assembly lines. Exactly. So that's that image of the factory that Ford produced because Ford, he didn't only produce a kind of way of producing, but also a way of consuming. And social scientists have looked at how that kind of idea of the industry, how it kind of leaked into society, leaked into society and shaped how schools functions, how companies function, you know, like that it became a kind of way of organizing society, broadly speaking. And so what we are kind of observing in this book is the fact that we've moved from that kind of machine epistemology into an era that's defined by kind of complex systems, where we're talking about society as kind of swarms or as self-organizing. We're using these different metaphors that stem from they're much more organic, you know, they're not the kind of machinery, you know, and we've also kind of in some ways abandoned the ambition of having the state to design society to produce certain outcomes. So this kind of utopia, this hope of improving the world, has we've kind of given up on that. And instead, and there is this idea that, you know, now that it's implying that the outcomes of systems, like the bottom out outcomes of systems, somehow would be inherently better. That's a kind of underlying assumption of this, what you might call ideology. And so that's kind of what we're interrogating and questioning, because we would argue that this isn't just something that's natural. There are still forms of power that are shaping society. they're just much less visible and they're operating through by shaping kind of how we interact by things like algorithms. And so they often become kind of difficult to see, but they can still have very large structural outcomes on society. So for instance, it's by kind of fiddling with the rules of interaction, which is basically what platforms are doing, right? And that can have really important large scale outcomes and there is still power there. It's just power that has new epistemology. So that's very, very helpful. Thank you. I mean, I read the beginning of your book, but I can get through the whole thing. So this is why we have the podcast so I can just ask you questions. So in other words, let me try to rephrase it and see if I'm understanding. You know, we might have had this dream of planning an organization where it was top down and either Henry Ford, who I guess was very capitalist at heart, but still, you know, he was trying to organize his factories or, you know, central planning from a government. And there are various arguments you can make that simply letting things come to an equilibrium is more efficient, whether it's a sort of thermodynamic equilibrium or an economic free market equilibrium or whatever. And you're making the point that maybe so, maybe that's sort of a better way of calculating some optimum, but it's still carries with it its structures of domination and power. For sure. I mean, there is this, you know, even emergent outcomes, even when we arrive at an equilibrium, it's still like those outcomes are still defined by the conditions that we came in with. Right. And so, I mean, it's like these arguments that you sometimes hear from certain certain parts of academia where it's they build some model of the economy and they find that certain people become very poor or like, in large majority become very poor and certain people become very rich. And they say, well, it's an inevitable outcome of this system. We shouldn't try to change it. But that's just not, it's just not a good argument. Yeah, it doesn't follow because you could have produced other rules that would produce other outcomes. And if your outcomes that are produced by your system are problematic and are harming a lot of people, then maybe you should reconsider the rules that you're operating under. And also with the notion that like, that this idea that the market somehow be natural and not, you know, like that it would not be shaped by state power has been, you know, a question for about a part of 100 years that actually the market is very much a construction of state power. I remember the epiphany I had when I was thinking about different probability distributions. This is purely a math statement, but still it's, you know, important like we have uniform probability distributions, power laws, bell curves, whatever. And you might ask, well, which one maximizes the entropy? And the answer is they all do subject to different constraints. Like it's all about the constraints you put on the system, microscopically. Nothing is there really inevitably. Yeah, no, and I think this, I mean, it's really, this is very much at the core of what I am interested in to certain degree. And I think it's also very much at the core of many of my papers and much of my research on this, which is the fact that as we've moved from a kind of a spatial society, you know, we moved from a society that we could, you know, if we were a if you know, as a physicist, one might think of it as a kind of lattice, you know, a nearest figure. Yeah, exactly. And then we moved to a digital society, which is characterized by network structures. And those produce, you know, like they are associated with different type of distributions, as social scientists, we, you know, everything is a bell curve, right? Yeah. But, you know, as the computational social scientist, everything is power laws. Everything is power laws, right? And that's not just like a question of how we study these systems and what assumptions we need to make. It's also, you know, that certain people, when we have digital network structures, become very powerful. And most people are, you know, powerless and they don't get attention. They don't get resources that are important. And, you know, that's, that is profoundly problematic for society. And they are attributes of these networks or the structures. I mean, maybe a good inroad here. And I think you just gave us a good segue to it. But there's this classic work on self-organization in society by Thomas Shelling and his segregation model. So you have an updated version of that. But why don't you tell us what Shelling's version was? Sure. I think it was published in 1969 already. And it's still kind of pointed to, I mean, I use it all the time when I'm teaching. I think it's just still somehow the best model. It's not just the first, also, we peaked, you know, as computational social scientists. And so basically Shelling was kind of, he was, I think he was the back stories that he was like walking through his cafeteria and the university and was looking around and he saw like, it's weird that, you know, all the, you know, geographers sitting in one table and sociologists are in another table and like, then he was like, yeah, and the same thing with the city. And he was trying to figure out like, why is segregation such a common outcome across systems? And so what he did, he actually took a checker's board. So he wasn't actually, you know, he wasn't even a computer simulation back then. And then he had coins on the checker boards and he was, okay, so let's imagine that the system is a lattice and agents are randomly distributed on this checker's board. And each agent, they follow a simple rule. So if more than like a, you know, a very high number of their neighbors, of their kind of more neighbors are of a different type than themselves, because they're two types of coins, then they move to a random available space, space on the, on the lattice. And so, and you can have the rule something like that they're completely satisfied if, you know, 70% are of a different type than themselves. But if it's like 90%, they're like, yeah, okay, like, you know, I'm tolerant. Come on. So they want some neighbors to be, they want some neighbors to be like themselves, but they don't insist that most of them be like themselves necessarily. Exactly. Exactly. And then so they move to a random space. And so what happens? You would kind of expect that if they're happy with, you know, like 80% being a different type, you would expect the system to settle on maybe something like 80%, you know, like, like, pretty much 50/50. You wouldn't expect very high level segregation to merge. But what actually happens is that you get almost complete segregation even with very kind of high thresholds. And why is that? Well, basically, you get this kind of cascade effect. So one person is leaving the neighborhood and neighborhood is left more segregated. His like neighbors also move and you get this kind of cascade. And so basically, as a, what the model is telling us is that the integrated state is just very unstable. And so the system tends to tip over like any neighborhood tends to tip over to one to one color or the other. And so to me, I mean, it's always been my favorite model. And to me, that seemed to tell us something also about the digital world. So I was interested in like, can we generalize this to other type of interaction structures that are non-spatial? And so I look at kind of forums and different platforms. Because basically, when looking at platforms, when looking at social media, there's been like for the last 20 years of debate on, because we often see echo chambers, right? We often see spaces being very homogenous. And then there's been a longstanding debate about whether that is driven by the algorithms, like the filtering algorithms. So there's this notion per resource notion of from his 2000th book, The Filter Bubble. So this idea that the algorithms create this cocoon, they only show us content that we already agree to. And then especially in recent years, there's been more and more arguments that that's not true. In fact, we want to be segregated. We don't want to, you know, we don't want to be exposed to other ideas. We don't want to encounter someone that disagree with us. And so that's been the kind of the debate between those two positions. And to me, it seemed like that might, you know, maybe it's neither. So basically what I do in this paper, it's a very simple and short paper. I implement the shaling model, but I move it online to certain degrees. So I look instead of having like a lattice, I look at, you know, we have different groups, so that can represent kind of subreddits, it can represent websites. And the agents are randomly located. So it's again, very similar to the shaling models. It's as simple as possible. So the agents are randomly located to these groups. And then in each round, they interact with some random people in their group. And then it's the same rule as in the shaling model. So they're happy with, as long as at least some of their interactors are of their same type. And if there's no one of their same type, or you know, maybe a very few percentage, they move to a random other group. And I mean, with this background, it can kind of guess what the conclusion is, but what I find is that actually the shaling segregation effect is even stronger in this kind of communities that they're even more prone to segregate. And of course, that has kind of interesting implications for this debate, because it's not necessarily either that this is driven by filter bubbles to this driven by algorithms, nor is it in the interest of anyone, it's just something that follows from having social interaction and structure in these ways. And there's also some kind of counterintuitive results from this. So for instance, actually having filtering algorithms, having a filter bubble, it actually reduces segregation. Because if you have a filtering algorithm that always shows you someone who agrees with you, like, or whatever you're shown, some messages, you always get someone who agrees with you put into those messages, you will become less prone to moving. >> To moving, okay. >> And so that will, for the system level, it will reduce the amount of segregation. And so the system will be much less likely or to be stable under those conditions. >> So this is, yeah. >> In Shelling, it's literally a checkerboard. Even in my book, The Big Picture, I talked about the Shelling model a little bit. And so it's literally your nearest neighbors, which makes sense, if you're talking about racial segregation in a city or something like that. And so you're saying you're putting it on a network, basically, right? Where there's different nodes that you can hear, or is it more dynamical than that? Is it just like a different spatial structure or something that it changes with time? >> I focus on groups in this case. So more subreddits. So it's more like a join a community and then you're exposed to random people within that community. You can also run it on networks. But in this sense, the network structure are less prone to this emerging as kind of shell-ending dynamic because you need the kind of transitivity. You need something like if you're leaving the community, the community becomes more segregated, and that increases the chance of someone else moving. So you get this kind of threshold effect where one person triggers another person. And then network, you have to make really strong assumptions for that to be the case. >> Okay. If you get annoyed with someone, you just unfollow them, but that doesn't change your friends networks. And so they're not going to be more like that. >> So there's not the positive feedback you get in what you did. >> At least not this kind of shell-ing feedback. >> Good. Okay. And by the way, my impression is that shell-ing was offering an explanation for urban segregation that did not require like, you know, racism handed down from on high via redlining or whatever. It was all just individual preferences. But in fact, when the social scientists have gone to look at it, the reason why real cities are segregated is in fact because of racism from on high forcing it to happen. >> Yeah. No, I think this is a really important point. And it's quite funny in some ways. I've been in geography for not anymore, but I was in as a postdoc in the geography department for about four years. And quickly realized that the only social scientists that do not really know or engage with the shell-ing model of segregation are the chargers because it just seems fundamentally incompatible with that way of thinking. And I'm very much, you know, in agreement. And it's quite interesting to certain degree. And I think it connects to the question of epistemology because it's kind of the Thomas-Shallings segregation model. It gives a very deep insight into the kind of dynamics of segregation. But it's also really hard to kind of bring that insight into dialogue with the existing literature on segregation in cities that, as you say, very much point to kind of structural racism, redlining. But to certain degree, I mean, both are true, right? >> Yeah, exactly. >> It's just difficult to make these theories kind of speak to each other. >> Well, my line has always been that the shell-ing model is really good at explaining exactly what you started with, which is where people sit in the cafeteria, right? Like, there's not rules, like, you know, the jocks and the nerds have to sit on different sides, but they always do because of exactly these preferences. >> Yeah, no, I think that's a good point, and it's probably also less of a provocative example than using it to think about our segregation. And so this idea that, you know, we do change our social network or social media usage to be just a little bit more within a set of people that we want to hear. Like, where does this apply in the real world? Is this, are we thinking of Twitter or YouTube or TikTok or Facebook or what? >> Yeah, so this has basically been a long debate in the social sciences. This kind of question of how pervasive echo chambers are or not, and it's still a very, very heated kind of debate. But what I would say is basically that there are suggestions that there's quite a lot of communities that are relatively segregated. And so looking at, for instance, most subreddits are, if they are political, they tend to be towards one side or the other. But I do think it's an interesting, I mean, Twitter has historically been a good example of the opposite. It was for a long time, quite inclusive in the sense of having both political sides. And it's an interesting because it's Twitter is kind of function as the kind of model of organism for social science research, for looking at platforms because it's been one of the few platforms where we can actually get a lot of data or good. And so a lot of computational social science research has looked at Twitter and used it as a kind of way of speaking about social media. I would say that Twitter is like, it was a very different platform from everything else because it actually had all political sides. And it was characterized much more by a kind of conflictual debate. But if you look at kind of smaller communities, they do tend to be much more segregated in terms of opinion. And that can be a problem in the sense that if political theorists, when they talk about what conditions need to be fulfilled for us to have a kind of functioning political discourse functioning deliberation, one of those conditions are that we need to have kind of the diversity of opinions. We can't just have like political side, which is, I mean, pretty obvious, I guess. I guess I was going to ask a question about that. How bad is it if people on social media interact with people who are like them? Like I can imagine maybe a utopian political structure wouldn't be like that. But you know, most people on social media are not there to be utopian political actors. They're there to talk to their friends and be reinforced. Is that so terrible? So I mean, it's a, I would say that I think in a lot of cases, it can be even very beneficial. I mean, in some way, one of the ways that social media transformed society, one of the key ways was this possibility that we couldn't connect with anyone from all over the world. And so for a lot of communities, especially in minorities, if you are LGBT and you grow up in a small village somewhere and you don't have anyone connect to, it's been shown that it's very beneficial for your mental health and for your experience, for your, for your living experience. At the same time, the way that it affects politics is not always as beneficial. Because obviously, if the minority that you belong to happens to be kind of, you know, some extremist form of neo-nazism, it seems to have similar kind of consequences for those communities because it allows them to come together and form a kind of shared sense of community. And it transforms them from being, you know, someone isolated to a kind of confident political community. And that can be quite dangerous in terms of radicalization. And is this, I mean, impression is that it is something that comes from newfangled technology, social media, things like that. The ability of these smaller groups to come together, like some of them are just going to be people who like to crochet and others going to be neo-nazis, right? But is there data that backs that up in the sense that have we seen more viability of these small groups than we did in the 1960s or whatever? It's, I mean, it's very difficult to kind of look at those kind of changes, right? Because and we unfortunately, we only have one society. It's hard to compare how society would look different without social media, without digital media. But what we can say is that we've seen a kind of increase in political violence. We've seen a kind of democratic backsliding in a lot of countries. And we've seen the kind of political extremist movements entering to the political mainstream. And whether or not that is costly linked to social media is very difficult to say, but it is very clear that it is in our current society very much entangled with social and digital media. And I've looked so in my previous book, Intimate Communities of Hate, we look at one of these online communities. And basically we try to answer this question by going in depth and looking at the Stormfront community, which is a very old kind of Nazi community in the US. They pre-date social media, right? Yeah. So basically it goes back to like 95. Okay. And the nice thing about it is that you do like all of the data, all of the conversations over this, you know, long period of time is all available online. If you're able to scrape it and backpaste, they're very secure. It's from preventing you from scraping it. So we have all of the data and all of the conversations over this. you know, 20 plus year period. And so that allows us to kind of look at how the users are changed by interacting with this community. So we can kind of use natural language processing and various forms of text analysis and kind of see of how individuals when they interact in this community, how does it change their language and different markers of like how they perceive themselves and so on. And we've the kind of image that we come out with as much more, you know, it's a very much a kind of a question of a community formation of changing identity and so on. And so just an example, we can kind of see how when they first come in, they use I and my and speak of themselves. But then over time they start saying we or as, you know, for a stormfront because they start, you know, it's kind of a marker of them starting to think of themselves as part of a collective and as part of something larger. - So that's very interesting because it's not just about people and it's not that we were just talking about, you're talking about associating with people who are like-minded and here you're talking about the feedback acting on yourself and the individual people are sort of changing their identities and response to those interactions. - For sure. I mean, I think it's very clearly, you know, kind of a feedback process, right? Where we have a kind of segregation mixed with a kind of changing of identities. And so we can kind of see, you know, in the stormfront case, especially after 2008, Obama election is the few days after the election, there was just a huge search of users coming in. So new people joining and looking at what they're saying, we can see this kind of emotional kind of confusion and anxiety and they're trying to, you know, they somehow feel confused about this world that they're living in where, you know, a black person, there's like, can be compressed and what does that mean for their self identity and how can they make sense of this. And then the community function is emotional talk therapy that allows them to find new narratives and resolve these emotional anxiety and turn it into from something passive like anxiety to something active like anger or outrage. And they come out with these narratives about, you know, there's absurd narratives about the Jews and like the white star of the superior race, it was just whatever that happened. And so it's very much a kind of process that is on the level of identity and emotion and kind of self narratives. It's very interesting because it all cycles back with a very first podcast episode of Minescape. I interviewed Carol Tavris, who's a social psychologist. And she has this idea of the pyramid of choice when if you imagine two people who are basically 50/50 as to how they could make some certain choice, right? You know, what was sneakers to wear or whatever. But once they make the choice, if they make it in different directions, they start justifying that choice to themselves. And they end up very far apart even though they were essentially indistinguishable before they, you know, collapse their wave function on that particular option. Yeah, no, that makes sense. And so I mean, ultimately it's kind of the question of the kind of structural context in which people are interacting that can produce this kind of outcomes. And so, yeah, can I ask about either your model or the shelling model. There seems to be like as the physicist in me thinks of the icing model, which is similar, but not exactly the same where you have spins that are interacting on a lattice. And the thing we do there is we introduce probabilities by having a temperature, right? There's some chance that the spin is going to flip and whatever. And in the shelling model, there's a probability because when the person decides to move, where they move to is random. But the choice whether to move or not is not random, right? That's just determined by how many neighbors they have of each kind. So have people done that? Have people introduced a probability of moving rather than a certainty and seen if that changes anything? Good question. I honestly don't know. I don't know. I don't know the interesting. Someone should do that. Someone listening out there. Maybe something for us to do. Yeah, absolutely. And then, okay, so you then did a different study, which came out also very recently using large language models. And here, well, I'll let you tell the story. But the idea is rather than just having these mindless dots on a grid or whatever that are interacting with each other, you literally had little agents talking to each other and making choices in a social media context. So how did that go? Yeah. So basically, the aim of this is to address, in part, this like kind of longstanding criticism from social scientists or at least from a lot of social scientists when it comes to agent based modeling, which is that, you know, this rule based agents are, you're not very good representations of the full spectrum of human behavior, which is very enough. And I think, I mean, in a lot of cases, like the Shellings, the creation model is simplicity can be, it is very useful and it allows us to, you know, throw light on some emergent phenomenon that is ultimately structural. But in other contexts, it's also limiting, right? So looking at social media, like politics on social media, for instance, it's an example where these richer behaviors, they also can really matter, right? We cannot really separate the cultural from the structural. We need to look at them as kind of intertwined and as interacting. And that's kind of the background because what I'm interested in here, what we're interested in is this question because we basically, you know, we spent 20 years or something criticizing social media pointing to the problems and a link to various problematic outcomes. And now there's more and more kind of interest than like, can we be a little bit more constructive, can we actually like do something about this? Because ultimately, if social media can shape a politics that is like outraged driven and radicalizing, it should also be able to shape a form of politics that is, you know, pro social that has like healthier political and social outcomes. And so that's the kind of idea, but how do we study that? Well, using observational data doesn't really work. So that's kind of modeling approach can be really beneficial. And so we're using agent based models, but having instead of these rule followers, we're using large language model. And the work is kind of stand in for humans. Maybe if I get in a rep, just quickly, I mean, maybe give a little bit of background on to the concept of an agent based model, like as opposed to what? What is what kind of models are not agent based and what are agent based and what is that used for? Sure. I mean, so in the social sciences, the way that we have traditionally approached the social world is in the link back to where we started very much as a kind of complicated system. So we tend to think of society as kind of variables and interacting, which in a lot of cases can work really well. But if you're thinking of this kind of complex aspects of the social world, where you have interaction between agents and then leading to unexpected outcomes, those traditional kind of variable based approaches just don't work at all. Like how would you like using used variables? How would you like study the admiration, you know, like of all birds? It wouldn't be possible. And so agent based models is kind of using this kind of bottom up modeling approach where we. And so one example would just be the shelling model, right? Like so you have agents that are individuals, they follow simple rules and then you look at the outcomes. And that allows you to think together the kind of micro behavior of individuals with system level outcomes that can often be kind of unexpected from the rules that you're coming in with. So the individual agents need not be very complex themselves. Traditionally, they haven't been so they traditionally have been kind of simple rule followers. You have like, you know, like the shelling threshold rule or you have maybe an optimization rule. But basically building agents that would kind of mimic human behavior in terms of reasoning or language production, it just becomes impossible traditionally, right? Like it would just be extremely complicated. Yeah, you would have to build a kind of a reasoning agent, which we just haven't had up until a few years ago. But so basically when Shachypt came out and with the kind of rise of large language models that just became a huge amount of interest in like whether we can use these models to as part of agent based models to kind of simulate social behavior. And so that's the kind of what we're doing, but we're trying to actually use it to contribute some kind of social scientific theory and contribute to our understanding of social media and its dynamics. And so what exactly was the experiment you did? I think of it as roughly speaking, letting lose a bunch of LLM on a fake social network. That's pretty much it. But basically our idea was to try to create a social media platform and make it produce the negative outcomes that have been observed on real social media. And then try out a bunch of suggestions from the literature on how we can address those problems. And so our expectation coming in was kind of that we would have to fiddle a lot with the system and try to make it produce the problematic outcomes. And then so we could then see how stable those outcomes are and how easy, what's kind of solutions are best for addressing the problems. And so the problems that we focus on, it's kind of conditions of social media that make public deliberation or public conversation difficult. It makes it difficult to have a kind of a functioning politics playing out on these platforms, drawing on kind of political theory. And so there are three different things that we've already touched on a little bit. But so one of them are echo chambers that you do need to have if you're going to have kind of constructive conversation across the political divide. You need to have both sides of the political divide present. Otherwise, it's going to be really hard. So that's one condition you need. And then the second is this kind of question of attention and equality that we also touched on. If you want to have functioning political discourse, you need to have relative equality among individuals. So you can't just have like two or three individuals dominating the entire conversation because that's not a public discourse that's just broadcasting. And then finally, what's been referred to as the kind of social media prism. So this is the idea that you need to have a kind of constructive debate where people are actually trying to come to a solution. And so that speaks to this question of that social media has kind of tended to benefit loud, polarizing, conflictual voices. And there's very much kind of undermining functioning conversations. And so those are the three outcomes that we were trying to kind of see if we could produce. And we were expecting that to be quite hard to be honest. That's so sweet that you thought that would be hard. Well, I mean, the literature has kind of pointed to or argue that a lot of these are, especially the kind of the social media prisms or this kind of the polarizing tendency, that those will be expression of engagement algorithms. And so that they would be like the expression of social media identifying the most outrageous things that are being said and then shove it in your face to kind of make you upset and increase the probability that you will comment or engage with your post. So sorry. So the sort of two alternatives that we're trying to test here are one is that when you get these echo chambers and polarization things like that, it's the algorithms fault or the platforms fault versus this is just human nature. Well, so I'm not sure if I would put that as the context really because it's also like in this study, we don't, I was honestly just kind of assuming that it was from the algorithms at least that it's not just something that would emerge from human behavior. But these are kind of structural outcomes from the interaction between people and the rules of the platform. But to me, at least this kind of social media prism, it's such a rather specific thing. It's kind of odd outcome that the most extreme voices get more attention. And so I was expecting that to be, and I've written about this before arguing that it's kind of what I've called it, the trigger bubble. So it's not the filter bubble, but the trigger bubble. It's the social media algorithm trying to trigger you, make you upset in order to make you engage, because that's how the platforms ultimately make money. They make you, as well as all her duties. I was something they draw information, they figure out who you are and they sell ads. And so that was kind of my expectation. But and basically what we started was just building the most bare bones platform we could imagine, which is just the agents. So I should say also that the agents, their personalities, we take the A and S, the American National Election Survey, which has very detailed information about US citizens, including their politics. And if they like to go fishing, we have a very detailed information. And basically we turn that into a description for persona. Because LLMs, they really love to impersonate people. And ask it to explain your microwave and the voice of Shakespeare and it will do excellent jobs. So just to be clear, so the LLMs that you let loose on the social media, they weren't all, you know, Adam and Eve. They weren't all tabular rosers from the start. You gave them a backstory. Exactly. And the point here isn't necessarily that, you know, we want them to be completely realistic encapsulations of, you know, of this particular individual that they're enacting. We just want to have kind of a diversity and want to capture a little bit of that cultural richness, you know. And so then they are allowed to interact on this platform. And the platform is very simple. So basically they can look at the, they see the most recent news, random selection of news from from the specific day that we're simulating. And then they can choose to post about their news, and they can write a post about whatever based on it. Or they can, they see the timeline and they can choose to repost what someone else has written, someone that they follow. And they can also see based on the post that they see, they can choose to follow someone that shows up in their feed. And if they follow someone, they kind of go into their timeline, they see their, a little presentation about them and then they see their, you know, most recent posts. Wow. And can they unfollow people too? No, we don't, we don't have a following. So it's really a bare bones. Okay, but there is some ability they can choose to follow someone or not. Exactly. And so what we're looking at the, at the outcomes here is the network structure that emerged just through this interaction. Yeah. Because it's, because we want to have simple measurable things and we don't want to just look at, you know, how they're talking or something that's like an immediate outcome of, of the large language model. We want to have something that's more an expression of the structure of the platform. And network structures are, are interesting in the sense that they are very much kind of emergent and they are produced through their structural outcomes. And so that's what we're focusing on. And so we're trying to identify these three attributes. And to our surprise, we didn't actually need to do anything more than provide this bare bone platform. And we got these three features that are, that widely considered the problematic aspects of social media. But I should say that that doesn't mean that engagement algorithms aren't problematic. That doesn't mean that, you know, that, you know, it might still be that they're making matters worse. But it does imply that removing them will not completely solve the problem. So how many users did your social network have? We run it with 500 users. 500, okay. Which is, it's a little bit of a limitation with the approach. It's quite expensive compared to running a conventional agent based model. And conventional agent based model have always been criticized for being very expensive to run computationally. So it is kind of a weakness of this approach. And you said that the bad outcomes happen. Remind us what the bad outcomes were. There's a list. Yeah. So basically you get echo chambers. So Democrats and Republicans end up not following each other. They're just talking to themselves. You get high levels of inequality, basically, you know, power all distributions of attention. So a few users kind of dominate the entire discourse. And then finally you get what Chris Bale has called the social media prism, which is that the more polarized, more extreme users tend to have more attention. Okay. Okay. And is there, I mean, I, it's very, very tempting to speak anthropomorphically about LLM's, right? And to ask about their motivations or something like that. But of course, that's, you know, illegitimate. They're just faking that. They don't really have motivations. But is what is the right way of phrasing the answer to the question? Why does an LLM want to follow people on its own political side of the spectrum? So, I mean, so, so where these outcomes are kind of stemming from, like, I think it's a little bit different outcomes each one. But so for instance, the, that we get the power low distribution that we get very unequal distribution of attention. I mean, that's kind of stems from preferential attachments, right? That the probability of you getting followers proportional to the many followers you already have. So that, that, I wasn't so surprised to certain degree that that emerged because it is a well-known feature of network, as I already mentioned earlier. But the other kind of dynamic that we identify that we haven't really seen studied before because we do kind of need to have this kind of special, large line with model network combination to be able to study it, which is the fact that basically we know that retweets, people repost or sharing is very effective, it's very emotional, it's very much reactive. So we see something that we're upset about or that we're really emotional reacting to. And those are the types of things that we tend to share. And that, you know, that is well known and it's been kind of argued that that shapes the content that we see on source media. But what we're adding to that is that it's not only shaping what culture, what kind of content you see, but it's also shaping the kind of construction, the gradual formation of the network structure. So you have a kind of feedback effect between what is shared, which is very much emotional, and who you follow. And that is what's creating this kind of dynamic where more polarized users tend to have more followers and get more attention because this kind of sharing is part of this kind of feedback process. I understand how an LLM can respond. to a query and say something, but did you have to cook up an extra set of instructions for when to follow somebody? We only we don't provide them with any you know it's just like this is your persona, how would you, based on this persona, how would you act in this situation and then just have them kind of respond on the basis of that? Okay and the persona includes their political orientation as well as whether they like fishing or whatever? Exactly so it contains their political affiliation and so in terms of the echo chambers it's also not maybe so surprising and we can also look at the kind of motivations that they'll I'm just giving for why the issues of all of them and it does like in terms of the echo chambers it's kind of like okay so I don't want it like since I'm Democrat and I feel strongly about it I don't want to engage with this person who's from the other side but I think that it is interesting that it goes beyond just that individual choice it's not like the dynamic and the emergence of the structural echo chambers is not just because individuals are choosing it but because their choices are impacting others because it shapes what messages are shared and how the structure of the network is gradually built up. And is there room for the individual agents as it were to be affected by their success on the network as their audience captured the LLM's learn to be more provocative so they get more retweets? So this is something that I think is really important and I think it's really central to how social media is reshaping politics because it's not just that we get more polarized week when we go on the platforms but the platforms are kind of shaping the incentive structures that are you know shaping society because they are defining who gets attention and who doesn't and so in that sense it's reshaping a politics built around being kind of outrageous and so on and but no. Okay, briefly put no but this is like a paper that I'm currently working on actually so. Okay good. Very much what I'm interested in. And of course someone is going to say look LLMs are not people right like what are what worries about limitations of mimicking human behavior should we have in the forefront of our minds when we're using these LLMs? Yeah so this is it's a really kind of big debate right now because there's a lot of kind of excitement around this kind of generative simulation. I've tended to be on the more skeptical side of this so me and my co-author Mike Loroi we also have another paper coming out where we were specifically trying to answer this question and like how much can we trust this and how much can we trust that they're valid as representations of human behavior and it is kind of it's a very interesting question to certain degree because the models are more realistic as representation of human behavior than just a list of rules but they're also much harder to validate right because you don't know exactly what is playing out and it's very hard to calibrate their behavior to match human behavior and so it becomes kind of question like a difficult question they're like ending up somewhere in between empirical methods that are you know have a high level of validity and formal models that are parsimonious that are used to understand and they're neither and so it's a little bit unclear how we can use them and the way that we think about that in this paper is precisely it's linked to the fact that we're not looking at the text that they're producing. We're trying to look at kind of structural features that are emerging as aspects of the platform not that you know we're trying to distance ourselves from their behavior and try to look at the kind of structural outcomes of that behavior and to certain degree the what we're interested in is the kind of the robustness of those of those outcomes so in the sense then what do I mean by robustness well basically similar to the Shelling Sargation model you can basically it's the kind of joke among models or whenever you build a geographical model that has you know any type of lattice in it you always get to say it Shelling Sargation effect because it's just it's just such a stable outcome and you have to really work hard to study any other effect and so that's the kind of effect that we're interested in we're interested in seeing how robust these emergent outcomes from the platform are and so if we were just looking at the kind of toxicity of their conversation I would feel less confident that these are you know something they're actually playing out in real world but given the robustness of these emergent patterns I have more confidence but that being said we definitely also need to you know do more studies on this well it does seem like the robustness from Shelling to your LLM's I mean these are these are sort of different individual small scale dynamics giving rise to similar large scale behavior it sounds like there should be a theorem like the second law of thermodynamics but about polarization or segregation or something like that being the state with lowest free energy I don't know I don't know what to call it but are there theorems like that in the statistical mechanics of social media so there are people doing these much more you know physicist approaches kind of opinion dynamics I have a colleague here who's just basically doing icing models wrapped into icing models in order to to understand the social media my approach tends to be a little bit more based on you know empirical data I'm trying to go closer to the to real to the system so I think it generally is hard to find unfortunately this kind of loss in the social world because it is such an open system and I mean linking back to the question of whether society is a complex system or a complicated system I would argue that it's neither or it's both to some degree and one of the features of polarization that is a little bit weird to me in the United States is how even it is like we've essentially always in the history of the USA had two political parties with roughly 50 percent representation each it never goes to like 70 30 or anything like that that's not something you can test your model right I mean you basically baked in into your initial conditions how many Democrats there were and how many Republicans yeah I know so this is not something I would capture in this model in general I'm thinking of these models as just trying to somehow capture you know one mechanism yeah and as soon as you start having a lot of mechanisms it gets very very difficult so in certain sense of like how the outcomes of this interaction feedback into society and how it drives polarization that's to certain degree outside the kind of bounds of this model so you you said that you were tracking like the statistical results not the individual words that the LLM's are putting into their posts or whatever but do you know that those backstories you gave them like you know this person lives in Boston you know they enjoy theater or whatever did that matter did that affect how they sorted or polarized or succeeded in the social media game I mean to certain degree yes because if I wouldn't have given them any sense of if they were Democrats Republicans they wouldn't be able to act as that and you wouldn't you know trigger this kind of feedback effect but if it matters that they that they know if they go fishing or not yeah it probably doesn't matter yeah I don't know I mean I would be I think I used to know I think of that more as a kind of a little bit of noise or a little bit of perturbation to to make them not only act on the basis of their political personality but that there's also like okay so this you know he likes fishing you know he's talking about fishing I like fishing I'm going to follow him so it adds a little bit of that kind of noise and the fact that you know our our lives are not just politics it's that's just a small part of of everything that we are we have much more rich identities than that and you injected news basically is that like you know there's an external source of perturbations it said like you know this news event happened or whatever yes exactly so so basically if you're just giving the agents if you're just letting them talk without you know having them something to talk giving them something to talk about it just becomes the most generic and non-interesting conversation and it also doesn't create this this kind of richness that you know also functions as a kind of noise you know and so we basically focus on a certain day and we got all the news from that day and then we present them with like a random selection of those news and then have them discuss it okay so you didn't totally make up the news you actually were inspired by real news exactly yeah we we got real news from from a particular day and so I guess the you mentioned the power law distribution of attention so some of these they're all LLMs but some of them get a lot more followers than others it is there any sense in which some of them are just better at social media than others or is it is it purely statistics and randomness so I would say that it's pretty much stochastic I mean I wouldn't say that it's just randomly one of them happens to be you know a great influencer as such but there I mean being more political and being more extreme does help does to become more more influential but is I would say pretty much stochastic and I mean that fits also there's been various like kind of experimental studies on on how on this power law distributions and how they can emerge on in systems just through the feedback effects and it doesn't need to be any difference between the things that are being selected you still get this power laws and it's just kind of random so did we learn anything about how to make the world a better place through doing this Is this a help that suggests any ways to make social media better? I mean, so I guess we didn't really mention the interventions that we tried out because we built this platform and then we saw these negative outcomes and that gave us the baseline where we could try, okay, can we fix this problem? We came the kind of next step. And so we looked at the literature and looked at what has been suggested, what are people kind of optimistic about in terms of trying to solve these problems. And basically we had the kind of wide variety of more or less sophisticated solutions that have been presented. One of them was kind of the bridging attributes, which Jigsaw has released, which is a subcompany of Google, which is basically they analyzed the content of the messages and then you can sort your newsfeed if you're a social media platform. You can use it to sort your newsfeed and you get the most kind of constructive comments, those are the ones that you show. So instead of showing the most upsetting comments, you can show the most constructive, the most partisan. So it's called the bridging attributes. So that's one example. And another example was like, is this smaller solutions? You're just hiding the biography, the description of the agents when they follow each other. So they don't know if the other person is Democrat or Republican for instance. And another is just swarming chronologically instead of showing the most shared posts. So basically we try out a bunch of those solutions that have been suggested. But I should also say that we are doing like fairly extreme versions of it. That wouldn't necessarily be realistic to implement on a platform. So for instance, we show the one algorithm where we show the least liked posts first. Sure. So you know where you would lead to a really awful platform if you implemented it in the real world. Because we want to see the most extreme solutions kind of. But unfortunately none of these solutions really fix the problems that we're observing. And some of them actually make matters worse. So for instance, the chronological timeline actually leads to more of a social media prism where you get more extreme users get even more attention. And so it seems what we take from this is that these kind of emergent phenomena seems to be very kind of rigorous to perturbations. That is basically a little bit like this shelling segregation effect. It's a very robust emergent phenomena. Yeah. And you don't want to have a social media network that just tells every user who to follow, right? You need to give them some agency there. Yeah. I mean, it's also the question of if people are going to use the platform or not. But basically, I mean, to me what this suggests is that these basic structure that we see across social media platforms where you have a network, you follow people and repost things, that tends to be linked to these problematic outcomes. Well, I guess that was where I was going to go. You already sort of said this is hard to study. But is it something truly new, these social media things? I would used to be happy just getting the conventional news on one of the three network stations on TV. And now we have a lot more variety in what we can listen in on. But has this had a big effect? Is it really like you can see that it has an effect? But I guess how much of our current political mess can we be tracing to this? I know it's a hard question to answer. Yeah. I mean, it's ultimately, you know, impossible to know. And to a certain degree, it's also a bit tricky. Treating social media is something that's like external to society and that happened to society. Because to me, that social media structure, the way it is, is very much an expression of coming back to Fordism, the kind of transition from an industrial society to post-Fordist society where the focus of capital is to is advertising and figuring out information about you that was not catering to a mass market where everyone, you're selling the same product to everyone, but that you're really trying to not only identify consumer niches, but even create consumer niches. And that basic fact of how the companies make money, the business model underlying social media and the internet, that has very much shaped what social media has become. Of course, it's also feeding back, you know, but it's very difficult to say how social media would be different if it wasn't that context. Well, I guess it's the feeding back. I was going to mention very briefly. It's not just that you have social media in addition to mainstream media, but the social media affects the mainstream media. They want those clicks too. Yeah, exactly. I mean, I think this is something that people often mention as like, okay, but I can just stop using social media and I won't be affected by these negative consequences. But of course, that's not the case, right? Because social media, as I mentioned, it's reshaping our politics and it's very much reshaping also mainstream media. A student of mine and a student project in my course last year, for instance, he looked at the New York Times headlines over time and then measured how clickbaited they are. And basically what he saw was that when social media and through the scene in like 2010 and so you saw a kind of jump and you saw that the New York Times also changed how they wrote their headlines. And I mean, that's just the kind of, there's one expression of it, but of course, it's reshaping, the incentives of attention produced by these platforms are reshaping our politics, our media and our culture overall. And what does clickbait mean? Is it a function of sort of giving less information and saying like, you won't believe what happened next? Yeah, there's like actually a bunch of kind of features of text that make them more or less clickbaity, but basically the way he did it was to just look at the databases of clickbait news articles and then non-clickbait news articles and then train a classifier on it. So I guess the last thing to talk about is there's polarization. So you had these LLMs, they sorted themselves in a shelling like way, et cetera. Can we say anything about the quality of the information, like the truthfulness versus misinformation, or social media helping us not just only talk to people like ourselves, but to get it wrong by sharing misinformation? So I would say, I mean, this is not something I'm looking at in this specific model, right? Because the, in part because the LLMs are under open AI, doesn't let them produce misinformation. So you can't really use them to study that. But I have looked at this in the context of using actual social media data. And I mean, what I would say broadly around it is that social media is by removing the kind of gatekeepers that we used to have from mainstream conventional media. They're also, and by creating kind of really strong incentives for gaining attention and shaping the kind of conditions, but that they're really producing conditions where not anchoring what you're saying to truth becomes a kind of beneficial strategy for gaining attention, right? Because you're both, it allows you to be kind of outrageous, it allows you to trigger people, and you're not really, you know, constrained by reality in the same way. And of course, that also becomes interconnected with politics. So I had a paper coming up with my co-author, Julianne Schwery, earlier this year, where we look at politicians across countries, and we look at their Twitter posts over a five, six year period. And we look at all the examples of when they've shared links, and we identify misinformation through that. And so we can link each politician to their likelihood of sharing this information. And then we can basically use that for kind of comparative model, so basically a statistical model to identify the conditions when politicians are spreading misinformation. And so this links to this broader question of the link between social media and the spread of misinformation, which has been a big debate around this, especially in the last few years, but whether it's just social media reducing the quality of information over all of them. And what we argue is basically, it's not just that, but that social media becomes intertwined with politics. It becomes intertwined with different political movements. And the result is that certain political movements are emerging shaped by the interests and incentives of social media in such a way that they use misinformation as a political strategy to gain advantages in political competition. And what we find in that study is basically that it's specifically the radical right populist parties that are driving this rise of misinformation. So it's not just a social media phenomenon itself, but it's social media intertwined with politics and political systems. So I'll let you give like a last big picture kind of thought here. Like, am I getting the impression that social media are just bad? That it was a mistake that their net effect is negative or can we have some spread of optimism to hold on to? I mean, I think that there are also some degree positive outcomes from it. And for certain communities can be beneficial. And to certain degree growing up, I love the internet. It was great. I grew up on this countryside on this little island in Sweden. And I made love nowhere. And the internet provided a social world for me and allowed me to connect to ideas and everything. And to certain degree what I would hope for is also going back to that innocent era of the internet of the 90s. I was using ICQ and you had this button where you could click and talk to a random person anywhere in the world. And I loved that. I spent my days talking to some random person in Arizona. And to certain degree, of course, that was a more innocent time. And maybe if we would try to bring that back, it would lead to something horrible these days. But I do think that we could create structures, we could create platforms and spaces that would actually be beneficial for us and that would actually be positive. We might need to re-think it in more fundamental ways than just this cosmetic change is to algorithms or designs. All right. That is something this homework out there for all the young people. Think about fundamental changes we can make because they're not going away. Even with social media or a net bad, they also are good and we're going to have to live with both of them. So better to remember, thanks very much for being on the Winescape podcast. Thank you so much. This was really great. [Music]