How to study “contemporary” news?

[MUSIC] What is it about? [MUSIC] Computational Communication Science? [MUSIC] Hello, hello, hello. Welcome back to another episode of what is it about computational communication science? My name is Mario Heim. I'm a professor for communication science, and particularly computational communication research at LMU Munich in Germany. >> Hi, welcome to today's episode. My name is Emma Schmittomreide. I'm a professor for computational communication science at Technische Universität in MNL in Germany. In today's episode, our guest is Josephin Lukito. Hi, Joe. Welcome to the CCS Board. >> Hi, it's so great to be here. >> Joe is an assistant professor at the University of Texas at Austin School of Journalism and Media. Topic wise, she focuses on malicious political language in the public sphere, on cost platform flows of messages and frames about global economic and political issues. Methods wise, Joe specializes in computational and quantitative methods, analyzing mass-communicated text, including corpus and computational linguistics, natural language processing, human and automated content analysis, and text as data strategies. Her ongoing work is on the multi-platform spread of misinformation, disinformation and unverified conspiracy theories in democracies, including the amplification of such messages, sometimes unintentionally by news media and political actors. >> We continue in this episode with our series on urgent societal problems. And today we ask the broader question of how to study contemporary news. We put the contemporary in the quotation marks because it contains a lot of, well, problem malicious or unexpected news as well. And then that we want to talk with Joe about analyzing contemporary, well quotation marks contemporary news environments. We want to talk about the hybrid media system and partisan amplification. We want to talk about gatekeeping, modern gatekeeping and multi-platform research. And of course we will talk about the role of CCS in all of this. >> So let me start with an opening question, Joe. Why do you care so much about political language in the news or in contemporary news? >> Well, thanks so much for having me on this podcast. And I love the use of the quotation marks around the word contemporary. Because for me, it's not just that news has become perhaps questionable. We have a large range of different types of content. But also that term contemporary. I'm referring to both contemporary modern digital news organizations or media organizations. But I also study older news organizations that have become digitized and now speak to an online audience. And for me, whenever I think about politics, language comes to the forefront of my mind. You know, people say a picture says a thousand words, but really, language is what we as humans use to create specificity. Right? If I'm in an argument with someone, I'm trying to use my words to persuade them. Or if I'm on this podcast, I'm using my words to communicate and talk about, you know, computational communication science. And so for me, understanding how we use language to understand with each other, to argue with each other. Those are, I think, really important parts of being human, but especially making sense of our political world. >> Yeah, thank you for this. I think that language is so very much in the center, right, of news and news organizations. And as well, maybe of our field of communication science. And later on, we might come as well to other options or how we can proceed with that. But maybe if we start talking about contemporary news environments, we should make some definitional work. Right? If I read your work or your papers, I very often encountered the term hybrid media systems and the specifics of hybrid media systems. So could you maybe explain what do we mean by a hybrid media system? >> Yeah, a lot of what I study, I use this kind of hybrid media system framework. And what I talk about something being hybrid, I'm really meaning that the media system now incorporates both traditional and digital media. And so this is true at the kind of news production level, journalists from traditional news organizations, certainly adapt digital techniques. They have online profiles. They're on different social media platforms and they promote their content online. But digital news organizations also adopt traditional media logics. They write in an inverted pyramid structure. They engage in the news gathering process. And they try to verify their information before publishing it online. And so for me, I really want to study how news interacts with the rest of the media ecology. That includes things like digital media and social media. Because what we find certainly in the US and in Europe, but also around the world, is that people are looking for much of their information, their news, their gathering online. They're either getting it from incidental exposure, kind of in the news science meway, or they're actively seeking out information online. Because it's much more accessible than going through a news paper or having a subscription to a broadcast news organization. And so as the audience has moved to this hybrid media consumption, I think it's really important that communication science researchers also study the media ecology from a hybrid format. So that kind of links, let's say, two different analytical levels, right? On the one hand, you're interested in the political language and the language that people and the news use. So we're very much on a micro level of message transmission. Whereas the hybrid media systems framework refers to a very high level of like a systematic overview of how this all works. And you just pointed out that there is something in between with some organizations doing stuff, maybe on a mezo level, between those two analytical perspectives. I'm wondering, particularly on this mezo level, you've already hinted at the fact that in this hybrid media system, there are also other actors involved, not just the news and the actors. And we've put quotation marks of our own contemporary. So what other actors are there? What other parts are missing in this hybrid media system? Oh, gosh, that's such a great question. So I think typically when we talk about news media, when we talk about the media system, the default actors that we think about are news media, politicians, political actors in public figures, and then the citizenry. But we know now, especially with the accessibility of digital media, that almost anyone can be a content creator, right? And so the media ecology today includes not just news organizations, but influencers, people who got famous. Wine, it includes digital experts, right? It includes more academics. I think one thing that I've seen from digital technology is a greater ability for academics to reach out to general audiences and communicate things about our research. And so for me, it's almost like anyone can be a content creator or better or for worse. And that's really good in some ways, right? I think it is important for citizens to be engaged. And I'm excited to see citizen journalists and academics reach out to a broader audience. But it also means that malicious actors, disinformation, digital campaigns, malicious actors, conspiracy theory organizers, also have access to these tools to communicate with a larger audience. We had this topic a lot during, let's say, a recent crisis, for example, COVID-19 or the Russian war. And we kind of perceived the situation as if such problems might be of increased relevance nowadays. So for example, for citizens to distinguish between trustworthy sources, right? Online and non-transvesty sources. And maybe we need as well to distinguish, right? That this actors might, or in this form of misodist information, might shift different political goals, for example. And I think your work deals a lot to be dead and how news media organization might as well be involved in such campaigns. And that's something I find very interesting, a very interesting perspective. Right? So what do you think, what are the different goals of such information campaigns? Yeah. And Mario alluded previously to me taking kind of different levels of analysis. And one of the reasons why I do study kind of micro level languages, but at this systemic level is because I'm really interested in the spread of these various forms of harmful content and the consequences of them. For me, the reason why I study misindist information or hate speech through the spread of that content from one platform to another is because I would like less of it in the hybrid media system, right? Whether that's coming from everyday citizens, but also kind of reducing the spread of it when it's unintentionally spread through news organizations. And as you noted, I do study harmful online content that tries to achieve different political goals. And I think oftentimes we tend to have a very narrow scoping of what it means when content is producing harm, right? Is disinformation fundamentally persuading people to vote in a specific way or to have an opinion about a political issue? But the reality is that a lot of communication goes beyond just persuasion, right? And we've seen disinformation campaigns that discourage voting or make people untrustworthy or they distrust the media system. And so I think in thinking about these harmful contents, I really want to expand this idea of what it means when a content is producing harm online and what that means for our political system is whole. Certainly when it comes to misinformation and disinformation and when I'm referring to misinformation, I mean content that is false but still spread online because people don't realize this false and when I talk about disinformation, I'm referring to content that is intentionally spread as false. So you know what false means spread it anyway. And for both of these kinds of content, they do increase distrust in the media ecology. And we don't need actually a lot of it to increase distrust in the media when people think that there is a lot of misinformation. disinformation or fake news, they tend to trust their own media system a lot less. And I think some skepticism is fairly healthy, right? That's a natural part of being media literate, but being wholly untrusting of the media system also makes you vulnerable to other forms of conspiracy theories and online harmful content. And so I think there has to be this sort of healthy balance in how we address, talk about, and correct misinformation and disinformation. Now trust in the media system has been a factor for decades, but has changed obviously, is that there's new actors in this hybrid media system. New actors and so much content, right? So much more content. Absolutely. And that is, I think a lot of it comes down to the marketplaces where these contents are negotiated platforms, specifically. And you talk about various platforms in your publications as well. Social media is one of them. Yeah. Can we pick, let's say social media and just look at what is it that makes these platforms such a good breeding ground for spread to spread of misinformation? Totally. So, unlike news organizations that have stricter forms of evaluation, I would say, if a journalist is ready to get a new story, there is a section editor that typically reviews it. The editor in chief might review it. It goes to a couple of copy editors and fact checkers. And so there is a very strong verification process when it comes to the production of news organizations, news content. That is not true for social media. Right? I can lie about something and go on a social media platform and just post whatever I want. And that's not necessarily saying a lot of people will see it, but there is the potential for many, many people to see it if I'm jouting it in a public social media platform. And so that verification process has changed very drastically, right? Whereas perhaps citizens have been able to trust news content previously, they can't necessarily do the same for social media content. And we put a lot more responsibility on audiences to do their own checking of whether something is true or false, right? One of the areas in which I think that is especially prolific is in partisan media content. And so, you know, with a rise of different social media platforms and the rise of digital platforms is cool. One of the things that we're seeing is a natural rise in partisan content, a greater variety of the ways that news is covered. And, you know, for all of these news organizations, both digital or traditional, when they're on social media, one of their main goals is to get other people to consume their content, right? If I am the journalist, a small news organization, whether traditional or digital, I want to have an audience for that content. And so I end up spending a lot of time on social media promoting my work, getting other people to see it and trying to strategize about how to make that content wildly popular. So I guess as you talk more about amplification, basically, right? In social media, I mean, this is at least a framework that I learned a lot about when I read through your work. Maybe you could help us out and tell us what is this actually? I mean, what does this concept refer to? That's super, super important because we have so many different terms talking about it and the the term that I use certainly in my literature is amplification, especially on social media. And I think one of the reasons why I study it in relations with social media is a matter of ease, right? Sometimes these amplification metrics are made more easily accessible. And so what I talk about social media amplification, I'm really referring to a set of strategies or mechanisms that try to influence public attention and increase the diffusion process through an online social network, thereby kind of making that content more available, more amplified it to a larger audience. And the goal of these processes is that they're supposed to increase the exposure to the social media content that a particular actor might be trying to advocate for. And so thereby, if they amplify that message, it also increases public attention to different ideas, issues, events that are going on individuals that we want to bring attention to. And we see that amplification, a lot of different actors, not just political actors, use amplification, right? Even historically, public relations experts also use amplification to try to get their content into these organizations. What we're seeing now is that those sort of amplification strategies, while they've traditionally relied on news media to amplify the content, that is kind of transferring and transitioning into a social media ecology. So even use organizations try to use social media to amplify their news content, right? That's why they're on so many different social media platforms. And by and large, this is not necessarily a terrible thing, right? We would like to amplify quality information about health crises or, you know, when to vote and good quality information. But because there are these kind of patterns to it, where, you know, if you're familiar with how a social media platform works, you can leverage amplification strategies for that platform, right? No, ample kind of content is more likely to get likes or shares or that sort of thing. And those sort of patterns can be and often are exploited by more malicious actors, people who are sharing harmful online content, including misinformation and disinformation. That's the part that I'm super concerned about is when they try to exploit these amplification strategies. So they're not doing a good faith, right? They're not trying to provide quality information. And oftentimes they will lie and cheat and scam their way into making something look much more popular than it actually is in words, increase amplification of that content. Okay. That makes sense for me, really totally makes sense for me. Actually, I like very much this concept. I think that's a great framework, right, for studying what is going on at the moment. And if I come back to partisan amplification of political contact, I understood that there are two extra types. For example, might be important. I mean, there's a lot more, but we could look at, let's say, some political actors, political influencers or so. And then the traditional media be digital or not partisan media. And that this might of course be problematic. The grocery might get exposed to one-sided news or even misinformation, disinformation. And if you come back now to the crisis that I mentioned before, very sort of, okay, for example, COVID-19, that was especially problematic. I mean, we are still about researching whether it was so problematic or not. But do you have any idea how communication was then affected by that crisis or happened during that crisis? Yeah. COVID-19, I think for a lot of people, the pandemic was a bit of a wake-up call that your media health, like the health of the media ecology matters as much as, maybe not as much. It matters a lot in relation to our physical health. So for trying to run a health crisis, such as a pandemic, it is important to get quality information out there. And we really ran into that issue, not just in the US, but internationally. How do we get quality information during the pandemic at a point in time where we do not know a lot about this disease, right? If we're thinking in the early to mid 2020s, there was a lot of misinformation during that time because there was a lot of gap in knowledge, right? We didn't know much about the pandemic or we didn't know much about COVID-19. And so these are situations that I think can be very readily exploited by missing disinformation actors, right? During, like, significant political moments or moments of crisis, misinformation and disinformation actors can exploit that to spread more misinformation through amplification processes. And, you know, in an ideal situation, would have media actors verify, validate and kind of push out that content or keep quality content. And this is something that we've often relied on journalists to do. But with the rise of social media, with the ability for a lot of individuals to produce their own content without any verification processes, a lot more of this, you know, harmful mis-indiscrimination is likely to spread, unfortunately. You mentioned, you know, political actors. And I think one of the things that I've been noticing is that at least for me, a lot of the political and that I study tends to have a partisan slance to it, right? We tend to believe content that aligns with our perspective more than we believe content that doesn't align with what we believe in politically. And so this has led to a very interesting to me, a very interesting asymmetry, at least in the United States, in terms of what kinds of mis-indiscrimination gets spread specifically by liberal and conservatives in the US media ecology. Now, you're referring to the asymmetry between the two partisan groups about what is being spread. But I happen to know that you also looked into the asymmetry about how it affects the differing partisan groups. Is that right? Yeah. Yes. So sometimes we think of effects. I think in the traditional sense, we often study effects in terms of running some sort of experimental design. And I would say that my quote-unquote effects research tends to be more looking at natural experiments, things that are happening on social media and things that perhaps impact the way that politicians or partisan news organizations talk or spread information. And one of the things that we find is not that there's just an asymmetry in the content that's produced. There's also an asymmetry in what's amplified. And then there's an asymmetry in terms of consumption. So we find that, for example, conservatives tended to be much more susceptible to COVID-19 misinformation. And that's not just because they naturally are because they're not. It's because there are political actors, news organizations, even malicious actors that are encouraging the spread of misinformation that makes it more adoptable by conservative audiences. We've seen, I also bring that up because we've seen similar effects also in the German media system, especially with this susceptibility because of the difference in how much misinformation is available geared to these specific groups. And it seems like a natural consequence of events that if there is more for group B then group B might be more susceptible to it as well And this group B also in in our cities shows that to be a portion of the conservatives rather than a portion of other Stances and not just that there is more content exposed to Conservatives liberals are different types of political ideologies But also the people who are advocating for those misinformation believes tends to be perceived as leaders within these communities Right is very different if I as a regular citizen said something but The interest in that and the believe in it is much more elevated if I were a politician that was talking about COVID-19 policies and then also had this very strong misinformation belief and so I think that you know the political leaders That spread misinformation help amplify it in a way that I Many many other political actors can not right even news organizations don't quite have that sort of command of attention The way that our political leaders do and so when a political leader goes on the social media platform and spreads misinformation about a serious Health crisis that has a very large impact on their constituency their supporters It kind of fits their strategy also in terms of Emphasizing this bus versus stem absolutely so it's us in a victim role versus an unknowingly Mass of them and then amplifying the news from sources that are not the mainstream news is somewhat aligned with that conception Totally yeah, and we see this especially when it comes to populist leaders right this kind of Leaders who leverage populist techniques kind of speaking as if they are Among the people are part of the people even if they necessarily are not and we definitely see that populist leaders across the political spectrum Really try to leverage social media amplification specifically and I think that's the case because social media amplification Doesn't cost nearly as much money is buying a media advertise Right, but you don't have to spend nearly as much money But you can have the potential of reaching just as large of an audience and so we see a lot of populist leaders we see a lot of Popular leaders particularly from conservative ideologies that spread us first them content not only leverage amplification But leverage misinformation and amplification spread in order to maintain a much more strongly held I would say audience or constituency Can we get the cookbook out and talk through the recipe of how to get amplified? What do we have to do? Yeah to to get via the Well who are the gatekeepers and how do we get past them in in the quote unquote contemporary? Yes News environment. I think that's a really important question And that's honestly what I spend a lot of my time studying is not just you know How do these platforms exist and how does amplification exist? But how does amplification and gatekeeping operate now in the hybrid media system? And so when I study this a lot of as I mentioned earlier a lot of the work that I study looks at how harmful online content tends to be spread and bypassing gatekeepers and one of the big areas of literature that I rely on as you alluded to is Dekeeping theory so this idea that journalists operate as a gatekeeper Allowing quality information to go through the gate of news and not allowing that information to go through the gate of news There's a scholar his name is Axel Bruins. He's a journalism researcher and he made this argument that at the point now with our hybrid media system Journalists are not as much engaging in gatekeeping practices so much as they are engaging in deep watching Practices because now anyone can post content online and oftentimes it is the onus or the responsibility of the journalist to verify or Talk about whether that is quality information or not quality information. They can't keep Things from going in and out of the gate But they do watch the gate and talk about what is quality or not quality information Of course now that everything has even for journalists move largely digital This has become a very very tricky thing to do and so one of the things that I am really interested in studying is how malicious actors might exploit This sort of gate watching practice to get themselves embedded in news content and some of my earlier work in 2018 or 2017 they got published in 2020 always with the timeline of academic work always I had a far times always Yeah, so you know during that time in 2017 and 2018 We had just recently learned about the presence of Russian trolls certainly in the US news media ecology But also in the European news media ecology particularly in Eastern European countries So I want to highlight that Russian trolls were not limited to the United States specifically But in the no we also and also not only to the Eastern Europe fear but also within the Brexit vote in the UK It was very prominent phenomena. Yeah, and I will say like coming from a little bit of an international relations background We know that Russia is particularly good at producing this and this information campaigns They have a long history of doing it and relative to other disinformation campaigns I would say Russia is among the most sophisticated. They leverage the hybrid media system They leverage amplification strategies at a rate that other countries simply do not And so one of the biggest things when this information came out active in 2016 election Was that Russian trolls were active across a variety of social media platforms They were pretending to be US citizens they were pretending to be US citizens of a variety of racial backgrounds They were pretending to be like conservative and liberal political officials as well as like teachers Doctors so on and so forth But one of the things that they were trying to do was to get news media attention through social media to amplify their own content So they were going on social media constantly trying to you know at mention or tag or reference and use organizations And thereby get their content Included as part of some sort of news content And so one of the things that I definitely noticed as I was looking at Russian soul content was that they were sometimes being accidentally quoted in US news media and the first time I had seen this it was in like a hopping-ton post piece And I was a little shocked. I was like how is this possible? How has this been you know gone through the news gate and presumably a journalist has put together this new story and yet there is a known Russian troll in here And one of the things that I realized was you know Russian trolls at that time were very much Exploiting and using news media content in order to amplify their own messages Whether those messages opinions were oftentimes there were messages meant to exploit that us them divide right really Making that us them divisions stronger than it has been historically So for example, there is a lot of unusual advocacy for like heterosexual pride day right things that we perceive as highly polarized content or highly polarized topics They were certainly Exploiting and there were not a lot of journalistic practices Train this sort of this information in and I realized like a lot of journalists some of their journalistic practices and hybrid media system Because they had changed had also become maybe these organizations more vulnerable to miss and disinformation spread So for example in this particular case of Russian disinformation what was happening was that Journalists was seeking out opinions of the public we oftentimes call this box popular So the kind of voice of the people and when journalists have done this for decades right we've always as journalists Why does it include the voice of the people in news coverage and when we often do that in a pre-digital sense We will stick a microphone in front of someone's face right we go out on this tree and a journalist will ask you questions about ongoing events Or if there's an election going on you ask people as they're coming out of perhaps a polling station But these processes are very very different for journalists now rather than going to that man on the street interview Many journalists will simply go online look for people posting on different social media platforms oftentimes at this point Twitter slash x Facebook and tiktok are three of the most common places where journalists will go to that platform All for information look for interesting opinions and then just post it as part of their new story and shockingly I realized through a lot of it's work that there are not a lot of verification practices for checking it that you know Social media post they just voted is actually from a real person let alone in this case a real American It's more or less a statistical question right? So if I in the early days as a journalist would go out to street Chances are I get kind of like a random sample of people but if I go online Chances are that I don't get a random sample but a sample of those individuals who wanted to show up in my sample Because they were able to leverage well as algorithmic or or social mechanisms of these of these platforms very well Yes, absolutely and I think this is I just Sorry, I just thought it's not a random sample maybe the question is I do get real people if I go out at least that I mean even if my sample is totally skewed because I'm I don't know going around lunchtime or whatever But in social media I might not even get real people right so so it's even worse I guess No, I totally agree like that's that's the thing is like it's not a random sample It's not real people and so we're getting a particular skew even if it's real humans we're getting Particularly partisan and vocal individuals and we're not getting real people necessarily We can't even verify or we don't have verification processes for checking whether they're real or not And I think it is important to remind ourselves of this too right and this is not only important for journalists But this is important for academics. I've seen so much Computational work coming from social scientists where they try to say like can we use the platform formally known as Twitter to study public opinion And I think that's interesting in kind of how you're conceptualizing it but the biggest problem I have with that is that only a small percentage of Americans, for example, are on Twitter. Only 10% of Americans are on Twitter. And so you're really not getting public opinions. This is not the same as conducting a random sample, a random representative sample for a survey. This is getting a particularly vocal audience. Here's a lot about politics and have very strong opinions on it, enough that they will be posting on social media. And so this is something that journalists are routinely lying on these sort of media practices. And if academics are relying on these social media practices, we're not getting a good sense of public opinion. We're getting a good sense of how people on a social media platform feel about a political issue. And maybe if you think about news factors and news values, and journalists might be interested to get the most polarized opinions, because they might produce, we have more clicks. And it's just actually quite sad if you think about this almost utopian idea of including the ordinary citizens in the news, right? And the idea that you could do this better in social media than we did before. And I was just wondering how much this is, as well, an economic problem, right? The question of, you know, the financing of such digital journalism or news platforms, and yeah, which is getting more and more problematic, or at least not that easy. I was wondering whether we have to do a podcast episode on this question, right? Because it's the new crisis and new topics that, of course, makes things more problematic, because the situation is newer, the situation is super hectic. So how do we get some opinions or who can say anything about what is happening right now? But the other thing might be that they don't have time, right, to go and find people, interview people. And let's take this quickly together on Twitter, or formerly on Twitter, now, X platform. And here also different stretches come. I mean, we've seen the platforms react to that. We have seen platforms establishing verification process, for example, Google or Facebook have implemented information boxes that say that, well, this information has been put to question by specific outlets. However, now Twitter/X has changed that recently to a box where citizens can question information not just like fact checking organizations. Community notes, I think what they call it, and so. Even adding to what Emesha said, there has been some change, but probably not in a way that we would expect to happen. Yeah, so so many changes have happened. I think like this, all the time, and you can y'all confront totally, you know, do easily like five or six podcasts about this topic, because I think. Are you need to come back? Whatever you want me, I'll be back. I, you know, to take this just a political economy component, I think that is something that we tend to gloss over, right, as computational researchers, partly because there's a question of data accessibility. Like, can we even access financial data about the media organizations that we study? And sometimes we can, like, that's something that is kind of exciting, I think, for me, about doing communication researches. Sometimes we can get financial data, and there are a lot of really quality media economy experts, and one of the things that they often highlight is how difficult it is to be a news organization in this fragmented media ecology. Right, and so you're not just, you know, one news organization competing against five or six other organizations. If you're a news organization, you're competing with hundreds, if not thousands of other news organizations, let alone media influencers or information influencers that are providing content of varying qualities. So I think for a lot of journalists and the for a lot of newsrooms, they really do struggle with how do I manage the cost of doing journalism. And so they might be inclined to take shortcuts, for example, relying on social media content, as opposed to doing man on the street interviews. They might engage in, for example, parachute journalism instead of having a bureau that is going to cover a conflict within area. And I think this is a major challenge that a lot of journalism scholars are really interested in trying to understand, like how do journalists do the work of journalism in a way that is cost efficient, given given how the media quality are now. And one of the things that I'm perhaps a little bit more optimistic about is ongoing relationships between newsrooms and media companies that could help benefit news coverage, fact checking, that sort of thing. And so one of the things that I've noticed recently is a lot of these organizations start to play the process of fact checking specifically social media content. Sometimes they will draw from things like community notes or other social media verification processes, but oftentimes they're combining this with other kind of additional journalism fact checking norms. And that gives me a little bit of optimism. Because that that's the kind of content I would like to see amplified in the hybrid media ecology. But as both of you mentioned, like this is an ever changing landscape and social media platforms can change their mind a lot about how they're going to spread content, how they're going to verify or validate information. And I see that the platform for me known as Twitter now X is certainly a platform in a serious transition right now. Right. And not just in terms of community notes and how people use them, but I think a lot about the verification icon that we had readily relied on on Twitter. And I think you know, for a long time, a lot of Twitter users relying on that particular verification status as verification for whether it's a real person, right. Do I know that is Barack Obama that is liking this tweet is there a verification check on his profile. And so, you know, that was a really important mechanism for Twitter specifically, but because the person who currently wants to put her now much more sought as some sort of way to amplify yourself. Right, like a status symbol as opposed to a verification symbol when he made it much more readily accessible to purchase without any verification as to whoever is purchasing it, we saw that the value of that particular verification check totally diminished right. That's it disappeared in the span of you know, a couple of weeks and we were at this point now when it comes to slash X that people are able to hide their verification check. They don't even want to show people that they spent eight dollars to get a verification check on Twitter. And that is astonishing in a couple of months. How much we originally relied on this verification check. So the point where we're at now where people are don't even want to show that they have it. If you take this together with what you said before, like Twitter as anyways a platform that incorporates only a small percentage of the population in the very specific population, let's say, politicians opinion leaders and so on, so we don't get the public opinion. And then on the other hand, Twitter is changing and becoming X and then changing again. So I mean, in order to find more general mechanisms that he could then use right for the future for for let's say designing interventions or so on to prepare for future crisis, future topics, we would need. Multi platform research, right somehow I mean, including. Absolutely different platforms. It seems necessary, but kind of impossible if I think about it because there might be a lot of problems. Totally, I would say what do you think about that topic? Yes, completely. And you know, I think over time we know that information doesn't stay on one platform and information changes over time. And certainly from 2016 till now, one of the things that I've noticed in the West is this transition from born based disinformation coming from a country such as Russia to domestic based this information oftentimes exasperated by extremists conservatives in particular. And for me when I study that sort of content, they're not only sharing it on mainstream social media platforms. They're sharing it on much smaller alternative media platforms. They're sharing it across different localities, not just text images and audio and a variety of other formats. And so for me, it is a challenge like, you know, whatever I think about multi platform research. And when I say multi platform research, I mean research studying more than one media platform inclusive of news and social media content. It is a challenge because the more platforms you add, the more complicated any project that really gets right it's when you think about certainly from a computational perspective, about what you need to do to clean, prepare, analyze, build classifiers for one platform to be it across four platforms makes it four times for work. But I think it's just absolutely essential because so much of this content is moving from one platform to another and that is absolutely amplification mechanism that I often see in my work. We typically call this a trading of the chain process and a trading of the chain process is when a piece of content begins on a smaller social media platform, perhaps somewhere like telegram or social it gets a little bit of attention. And then it gets picked up by either a more mainstream social media platform such as Reddit or it'll start to appear in smaller news organizations think each platform such as bright bar or more extremist partisan organization once it appears in a small news organization against further up the chain to more and more popular news organizations. And then it might appear in the Alex Jones show he's a fairly well known far right conservative podcaster talk show host and then once he talks about it, it might appear in a far right news content and then it'll appear in perhaps a hyper partisan news outlet that has a little bit more of an audience and then it'll appear in Fox News right. And then the content gets amplified further larger and larger news organizations will pick it up and something that we might see on telegram a week ago will somehow come. out of the mouth of a type or partisan conservative news channel a week later. That's a quite typical way of diffusion, right? News diffuses via various channels and gets picked up. And I would add to the list of challenges that you mentioned when we want to do multi-platform research. Two more things. One is the question of theoretical comparability. Can we is a share on Twitter X the same as a share on Blue Sky or the same as a share on truth social? That's the Trump never could. Too many platforms. Too many platforms. Absolutely. Too many platforms. So that's one thing to add. And the other thing that came to mind is we had an episode where we talked to Fabian Lind about multi-language research. And one thing that is like hinders multi-language research apart from the things you mentioned here as well is that it's not worth the effort. If I'm in an academic environment where I need to publish and I can get published with a study on one language, why do the hassle have to do the project in multiple languages? The same applies here. We have had a long history of communication research on Twitter or with Twitter data because it was so easy to get the data. So why have the hassle of also including Facebook, and truth social, and whatever. Now with the new owner of the platform, Poloninoa, Twitter, as you call it, this might change. It is becoming more difficult to get data on Twitter. So it's as difficult as it is for other platforms. What are your thoughts on that? Yes. This is a forever challenge. And I think, you know, Dean Freelon has a really excellent piece of the post-API age in the journal political communication. And one of the big things that he talks about is how we become so reliance on the permissions or the access that these platforms give us. And that is going to create challenges in our ability to replicate studies, in our ability to even know if we can continue doing this work. And you mentioned, you know, Twitter/X as a platform that we oftentimes studied and an overwhelming amount of research, especially in network analysis. For example, the light very heavily on Twitter because Twitter had been so open about the data that it gave to researchers. And one of the reasons why I ended up studying Russian trolls on Twitter is because they made a data set available to academics and to the general public. Right? And I think those sort of things are otherwise very, very difficult to study. It's already hard enough to access data because data access is limited and varied across platforms. It is even more challenging when you're trying to study mis- and disinformation. Because that content often does get removed from social media platforms. And we want to remove. Right? This is a good thing. But removing it also makes it harder for researchers, academic and non-academic to study it. And I think it's really worth emphasizing that not all researchers have accessed the same types of data sets. You know, I'm very fortunate to operate in a fairly prestigious university with funding and support for me to do this academic research. I think it's much, much more of a challenge when you are an early learner scholar or a graduate student or someone without as many research resources or funding in order to do this work. One thing that I've seen a lot of multi-platform researchers rely on, for example, are social listening tools, things like brand watchers, physio, that aggregate content from a lot of different platforms. The cost for that a license for a platform like that could be anywhere between 25 and $15,000 per year. You know, it's something that is just, you know, if you're a graduate student, that is not accessible research. Right? And so it really frees this unequal levels of access to data that I think is really, really problematic. And so a lot of my current work just by virtue of what I do has actually started to look more into data archival research. So because I study misinformation, I realize like I want to make this data more accessible to more people and I want more people to study hybrid media systems and multi-platform media qualities. But if I want people to do that, I should be working harder to make that data more accessible to people. And so it's been really interesting seeing a strain of my work starts to include data archival practices and thinking through what are the kind of legal ethical permissions about how we share multi-platform data. And you know, in kind of doing that work on archival practices, one thing that I realize is we really don't have a lot of social media archives. There are very, very few. I'd say Europe also has a handful of archival communities that are trying to also archive social media content, especially kind of in the closure of the Twitter API, a lot of archivists and data collectors were talking to each other about collaborating on data collection practices. And in kind of doing that work, I realize that a lot of the content that we're really focusing on happens to be text content, right? So that in addition to multi-langual challenges, in addition to multi-platform challenges and how do we even compare across them, we also have a tendency to focus very much on text and we struggle with multimodal collections, collections of not just text, but images, audio, video, so on and so forth. I think this is perhaps a silver lining for computational communication science researchers because we have so many more resources now to study multimodal content than we had five years ago, right? The techniques for computer vision or images data or video data has so dramatically improved. So, you know, I think for me, one of the most exciting things about computational communication research is the resources and the ability to study things beyond text at this point. We have so many more resources and tools now for studying image data and audio data and video data. And so one of the things I've definitely noticed for incoming computational social scientists, master students, PhD students, and early career scholars, is an increasing interest in non-text content and now the research and the ability to actually study that at scale. One of the areas for me and in my own research that I find really exciting is that I am starting to include audio data as part of the language that I studied. So many of the things that I had studied previously were very focused on text and written content. And I think that's fine, right? That is still really, really important. A lot of laws that get enacted still have to be written down, but there is so much communication that happens through audio, right? Through spoken language. This whole podcast is just spoken language, right? And so I realized that this is an important gap in our research that we really need to focus on. And so some of my work recently has been starting to look at audio as data. How do we study podcasts and top radio shows and the way people talk about politics on platforms like YouTube and TikTok? And if you think about the multi-platform media ecology, these video-based social media platforms are among the most popular. When I look at the URLs that are shared on Reddit or YouTube or on Reddit or Facebook or other platforms, YouTube is the most common social media platform that is hyperlinked to other social media platforms. And this is perhaps no surprise, right? YouTube is one of the most popular social media platforms period. And yet we don't study it because it tends to be video data. It's really large. It's hard to study. You need to transcribe it. It has all these sort of technical challenges. There are limits to what you can collect from the API. And so I think a lot of these challenges do sort of hinder the study of specific platforms that have data that's less accessible or harder to mean it. But we absolutely need people to research in that area because we should not be studying platforms just because they're easy to study. Because you are already in the how to improve a CCS, right? So I like that. Yeah, I wanted to. And you already touched on that by when we when you talked about the data that is not accessible to everyone, but might be accessible for some of us. And so I just want to quickly come to an article of yours that was published recently in political communication quite impressed me about scholarly solidarity. And I like it a lot that I think it was a perspective that we don't see that often and that made a connection and explicit connection to these problems. So what can we do as computational researchers to bring this scholarly solidarity hard for me to pronounce, but maybe easier to do something. That can be do. Yeah, it's like I love this question especially for computational researchers. I think for a lot of computational researchers, the work that we do is collaborative in nature. We need solidarity with each other to research together to collaborate on grants and projects and to share resources with each other. You know, I think about, for example, open science practices in the past and we talk about like needing to make research materials more accessible to each other. Well, that's something that computational researchers have been doing, right? We share code with each other. We share open source resources. We put things up on GitHub. And so a lot of those practices, I think, are just naturally really, really important. And in this scholarly solidarity piece that you mentioned, one of the things that I talk about is solidarity in responsible open science and data access. And that's something that matters a lot. It's very near and dear to my heart. I really, I think, you know, the most that I can train students is making sure that they think ethical and important data access questions. And how do you prepare data to share it? But also, what are the practices for sharing data so that other people can continue to research? One of the kind of research and dental work that I do is, as I mentioned, kind of in data archival work. And one of the things I'm trying to figure out is how do we present data to audiences or to other researchers that can kind of cross multiple social media platforms, right? And make this data more easily accessible to them. It's a lot easier said than done, right? One thing that I'm realizing in doing this data archival work is that it's like 50% technical knowledge and then 50% administration policy, talking to my university, getting data use agreements, that sort of thing. That's so, so important. And one thing that I'm trying to figure out is, can I share more resources like data use agreement templates or data cleaning practices? How do you engage in anonymization of digital media content so that when you share it, the user names are hashed and de-identified to the extent that we can? Or can we encourage academic publications to not only make data sets required to share, but also can we start to publish data papers, papers that are all focused about validating and checking and using a particular data set? And this is a trend that we've seen in other fields, right, natural language processing, for example, has a lot of benchmark data sets that are just fully available to their researchers. And I think that's something that communication researchers can also adopt in the work that we do. I'd love to see more of that sort of stuff. And I think in making this sort of data more accessible, we also make it easier to study that data across a variety of different ways. So I don't explicitly mention this in my scholarly solidarity paper, but one thing that does inform a lot of my research is that I actually come from a mixed methods background. I originally started as a qualitative scholar. And then I wanted to study content at scale in a representative way. And then I realized I did not want to manually code everything. And I was like, it has to be an easier way than to just like manually code 2000 tweets. And that's how I found my way into computational methods, right? Is that it was really accessible for me to study content at scale. But it also helped me realize that there is such a power of doing mixed methods research that combines both computational methods, qualitative and traditional quantitative methods, kind of in that vein of thinking one of the other areas that I think is really important is to do more research about non-US, non-Western countries. What we typically call weird countries. And for me, this is especially important because a lot of misinformation and disinformation spread does operate in the global South. It gets totally under recognized. There is, I'm originally from Indonesia. There's a lot of misinformation that is spread on WhatsApp in Indonesia that does not get recognized at all because Indonesian is a low resource language. We do not have a lot of NLP tools to analyze that particular language. We've seen very similar challenges with Arabic, with Zikalog, with a whole variety of, is it indigenous languages? It is really hard to study those things computationally because those are low resource languages. I'm optimistic that it's changing. I don't know if I should be optimistic, but I want to be, right? I think this is something that even NLP researchers are science and realize is super, super important. And my hope is that that sort of work will make its way that machine translation work will make its way into the field of communication and make NLP research more accessible across a variety of different languages. And then the last thing that I'll note. And this is more of a personal thing because I do study language a lot, whether it's spoken language or it's written language. For me, I think it's really, really important for communication researchers to start to develop our own theories that relate to language specifically. And I say that because in a lot of my early work, I was trying to find theories that talked about political language, that talked about why people choose certain words versus other words. And we realized so much on framing theory. In order to do that work, at least in communication research, a lot of our media effects or our language effects studies, oftentimes are done in the framework of framing theory, which is great. Like I think that framing theory has gotten us here, right? It's done a lot of good for us in the field. But I think there's a more complex and empirically grounded way to study language that goes beyond whether a frame exists in content or not. And so some of the work that I've been doing and writing about has been thinking about how do we study language across a variety of different layers. And so when I say a variety of different layers, what I really mean is, we can study language and individual words, right? I can say if I use a particular hashtag on a post in my signal that I'm politically inclined towards one party or another, but I can also study things at the sentence level or the paragraph level. And those sort of layering of language analyses allow us to not only study, you know, or analyze the content more robustly, but it also helps with a lot of our pre-processing steps. And certainly, you know, I feel free to stop me at any time because I can ramble for like two hours about data processing. Like we spend so much time talking about these fancy classifiers and LLNs and all of those things, I think are super important, right? Like I want to make sure I build as NLAs Supervised Machine Learning classifier. But I always tell my students, like if you have junk data, it does not matter how fancy your model is, right? You're not going to get quality results. Really thinking about pre-processing steps, especially for language, I think is really, really essential. And there's a way to do language processing in a layered approach that takes into account morphological structures, right? Am I going to shorten economic and economy down to its root word? Those sort of little decision-making processes really impact what our language data looks like when we're actually running it through a classifier, right? And while CCS certainly helps us with that, I think the outlook that you gave already hints at the problems is when we get there. So as soon as we start to look at audiovisual data, visual data, for example, how do we break things down there and what are the pre-processing decisions there is something we haven't touched a lot yet? Yeah, I think there's a lot to do. There is a lot to do. But I think that's exciting, right? For me, the fact that there's a lot to do gives a lot of opportunities to junior scholars for like, here are all of the pressing challenges we have. And if you learn computational methods, you can really contribute to the field in really, really meaningful ways. And I think that's so important, especially when students are thinking about like, where can I contribute most to the field and most to society? I think computational methods is just such a natural way to do so because you're able to study audiovisual data more at scale. You can actually tackle these challenges if you spend enough time coding in Python or R1 hosts. And you can even contribute as a qualitative researcher as you outlined, right? Because I believe that this mixed methods, if you really want to incorporate them, we need, of course, a lot of collaborations because I already feel overwhelmed by the possibilities of computational methods. And then it can't even think about not learning everything about qualitative methods. Right? This is a very natural or normal process, I would say. So, yeah, so the community here is welcome. We were entirely inclusive, you're quantitative quality, and even the non-imperial ones they should come in and provide theory. So, we're everybody's welcome. Everyone's welcome, right? And I think that is a solidaristic approach. Like when we take a field and we think about how can we communally work together? It's a build on research from one another, right? Like if I'm thinking, so, you know, a lot of what I study ends up being so US contextualized because I am American researcher born in the US. If I want to study a country such as Japan, I don't have the cultural context for it. Right? And so I do think that this is where qualitative scholars, non-competational scholars, area specialists are really, really important for collaborating with, because they can oftentimes provide a lot of cultural context for why something is being framed as a certain way, or why a specific pun is used. Right? I think a lot about word embeddings and our ability to use word embeddings to identify different meetings for the same word. Right? And that sort of stuff, even if a computationally, you can do it in Illustrate that they have different semantic interpretations. You still need someone from that cultural context to be able to understand and explain what that difference is. Right? Perhaps there's a historical background behind it. And so I think these sort of collaborations don't just need to be among computational researchers. They need to be among researchers from a variety of communities. They need to be researchers using a variety of methods. Sounds like a good. A good finishing sentence. Sounds like a wrap. So these collaborations, of course, we can include as well journalists and citizens or make sure that our outreach is a bit improved so we can inform them. Right? About these kind of problems, these kind of techniques, for example, for amplification, but as well, maybe to help them to develop some strategies or more resilience against such actors. And we're also collaborating with the legal side of things. We've talked in this episode with Natalie Helberg about legal perspectives about how to enable researchers to collaborate more thoroughly across platforms to get data more easily. So I think there is a lot of doors opening on the routes ahead. And a lot of work that we need to do. Thank you so much, Jo, for talking us through all these, let's say, findings and challenges, but as well, to have this optimistic view on what we can do in the future. I think this is really important and really helps us to contribute to solve some of these issues. So thank you for being with us. Thank you so much for having me. Thanks a lot, Jo, this has been great. And thanks to you for listening, we, as always, are looking for suggestions also for future questions and topics we should discuss or could discuss and also suggestions for future guests. We have, Jo, you have named Axel Rooms and Dean Freeland, two individuals who have not been on this podcast yet. So if they're listening, feel free to drop us a line. And we're happy to come back to you. But until then, we look forward to hearing you next time. Bye-bye. Bye. [MUSIC]