Conformity in the Digital Age: Jonathan Penney

Welcome to the Polytech Report, the podcast where we explore questions surrounding politics and technology in this new digital age. This episode, conformity, I'm joined with Jonathan Panney, a leading expert in the intersection between law, technology, and human rights. His research on privacy, surveillance, and emerging technology has been widely recognized and published across leading outlets. His new book, Chilling Effects, Repression, Conformity, and Power in the Digital Age, explores how surveillance and digital systems shape human behavior and threaten democracy. So Jonathan, what started you down this path? I was always, you know, since I've been, you know, growing up, I was always a bit of a nerdy tech kid. I recall my friend Aaron was among the first to have a, you know, like a high speed modem. I was an early adopter at the internet, I spent time in, like, the late 90s and early 2000s on, places like IRC and in a relay chat and, you know, different parts explored the internet early on. And I recall conversations I would have with friends offline and with friends online was, you know, how, how fascinated and excited we were about the internet back then and how its capacity to connect people and to amplify voices and to do, you know, great things for freedom and privacy and democracy and, you know, global connectivity. But over time, I also recall those same conversations we always had concerns about inevitably whether, you know, big government and, you know, industry and bad actors would at some point figure out a way to, you know, threaten and to weaponize these same kind of online and offline tools. And so I was always interested in these kinds of questions around freedom of speech and activities online and concerns around privacy. Read from, you know, my, you know, growing up to times later when I continued my interest in law school and then later with graduate studies ended up doing research as a doctoral student on this very topic. So there's always a personal side of this. But the timing for this particular book and why I wrote it now was that I really think that we've entered a new era where the threats that I talk about in the book, threats that have this chilling or discouraging effect on our speech and activities online and offline. They're being weaponized in unprecedented ways in this new era to chiller activities and to shape our behavior at scale. And I think the way that we think about this public policy challenge is deeply flawed and we need a new way of thinking about it. And so I think we've, while I've been thinking about these issues for a long time, it seemed like a pretty timely topic and an urgent topic and the urgency and the importance seem to be increasingly important to me. So I, that is the broader context for the book today. So for people who have never heard the term chilling effect before, can you just briefly describe what this actually means? Absolutely. So this concept of chilling effects has a kind of dual meaning or dual character to it. On the one hand, it's a legal concept whose meaning and prominence emerges in the post-war period in the United States and U.S. constitutional law. So you have, in the late 1940s and the 1950s, you have what's known as the McCarthy era, which is now called of the Red Square. You have this period in the United States and across the West where there's this real concern during these years of a calmness threat to the national security of the United States and also governments across the West. And the policy response was the governments in the United States enacting a series of laws that targeted the supposed communist threat and many of these laws targeted speech, targeted freedom of assembly, targeted protests and other kinds of activities in the, and often these laws were very overreaching. And so the U.S. Supreme Court invokes this concept of a chilling effect in a series of judgments, many of these laws were challenged in court. And the U.S. Supreme Court invokes this concept of a chilling effect because these laws had a chilling or discouraging effect on the free exercise of people's speech, which is protected by the U.S. First Amendment in the U.S. Constitution and struck down many of these laws. And so the term chilling effects comes to prominence in U.S. constitutional law. And really that legal meaning has carried with it to Canada, to the United Kingdom, to Europe and Asia and beyond. This term has traveled within legal jurisdictions, hence invoked by lawyers and judges and policy makers around the world. And this legalistic meaning has traveled with it. But underlying the legal concept and underlying the legal doctrine is a theory of human behavior. And that's what I talk a lot about in the book. That is, in the face of certain threats, a privacy threat, legal threat, even a threat of personal violence, these threats, we have a tendency to self-censor and self-restrain our behavior for self-protective reasons. And so there's two meanings, everyone. It's a legal concept that speaks to how laws have a chilling or discouraging effect on protected legal rights. But chilling effects also now refers to a theory of human behavior underlying that legal concept, but how we self-restrain, how we self-censor in the face of threats. And I talk about both in the book, but really a key focus of the book is that underlying theory of human behavior. When you talk about this term, is there a misconception of it by the public, at all or is it kind of understood? In my view, there is a lot of misconceptions about the terminology, the term chilling effects. Now, that's due to a few different factors. Main target in the book is what I call conventional understanding of chilling effects and the public policy challenge that they pose to individual rights, but also the health of broader democratic society. And conventional understanding has largely shaped legal understanding, but the understanding of journalists, policy makers, and by extension through media coverage and how it's been written about by journalists and even by writers in popular culture like George Orwell and Aldous Huxley, for example, you know, 1984 in similar kinds of texts and Brave New World of Huxley's famous text. This conventional understanding comes, it's been largely shaped by this legalistic origins. So it's very individualistic. It's very legalistic, focused on laws to the neglect of other kinds of chilling effects, but most importantly that I highlighted in the book, conventional understanding has largely been discredited by social and behavioral research. That is, underlying conventional understanding is a deterrence theory of human behavior that social science research has really shown requires very specific conditions to be met. Of course, certain laws and other kinds of threats can have a deterrent effect, but often you need very unique sets of circumstances that aren't usually present in day-to-day contexts. So the research shows that the theory underlying conventional understanding is weak, has a weak empirical basis in research and understanding. And lastly, conventional understanding really cannot explain chilling effects in a wide variety of circumstances. It can explain it in relation to the law and some traditional privacy threats. And so the general public, to go back to your question, but also lawyers and judges and even privacy scholars have, in my view, all gotten this wrong, have been rely on conventional understanding. We need to move past that and we need a new, better informed understanding that's one that's informed by social theory and behavioral research and science. I think that there's another sort of public misconception that's more foundational saying that. Why are you worried about being surveilled if you have nothing to hide? If someone said this to you, what's sort of the best argument against this point of view? Great. So this is one of those classic arguments that you often hear raised in relation to forms of government, police, and national security surveillance. That is, if you aren't doing anything wrong, if you aren't up to any mischief, then you have nothing really to hide from the government or from the police or from national security agencies, then why should you care that they're watching or they're listening or they're tracking and gathering data about what you're doing online and offline and profiling you as a result? So baked into this nowhere to hide argument, there are a number of different assumptions. And I think once you understand some of those assumptions, you can understand why it's a deeply flawed claim to make. On the one hand, what is assumed there is that there is a real black and white way of thinking about how we live, what we read, what we speak about, and what we do. Either something is criminal or something we should be ashamed of and that with something we would want to hide on the one hand. And on the other hand, you're only doing legal things, socially desirable things, things that would be approved by everyone around us. And the nowhere to hide argument assumes that if you're not doing any of those bad things that you really shouldn't care, that the government is watching or surveilling you. But the reality is how we understand personal development, how we understand forms of intellectual privacy, how we understand personal relationships. This is insights from social psychology, insights from privacy, insights from scholars like Neil Richards who and Julie Cohen have written about intellectual privacy in order for us to develop as people to develop our identities in order to forge long-lasting personal friendships and relationships with ourselves. We need a space of privacy where we can experiment. We can talk about embarrassing subjects where we can read about and search about embarrassing, shameful or even controversial subjects. And if we don't have that space, or we avoid these kinds of conversations or reading about these kinds of topics in subject matter, then we really can't develop as individuals and as people in our growth and identities are deeply stumped. And that's just on an individual level, right? On a societal level, right, we also know that innovation, creativity and democracy and descent operate on this assumption that to move society forward, to have social progress, to have creative endeavors to innovate, to move us beyond often conventional thinking. We need a marketplace of ideas, which itself is a bit of a fiction because we know that when people are speaking, not everyone has an equal platform, not everyone has an equal voice, but at least the running assumption that we know with and healthy democracies, you need to have people who are willing to descent from group think. You need inventors and creators and innovators to descend from common or conventional ways of doing things, right? And if government is watching what you're doing and tracking what you're doing, and you avoid all of those sort of more marginal ways of doing things, if you avoid challenging group think, if you avoid being creative in your solutions, because you don't want to have anything to hide, right? Then again, there's going to be real trouble and dangers, not just on an individual level, as I mentioned, but in society itself, right? And the research, and I've talked about in this book, these are one of the dangers of chili effects, and like this, and then maybe this is the third problem with this argument about nowhere to hide in this misconception of the public, that it also assumes a kind of rational understanding of how we behave when we know we're being watched, right? It assumes there's nowhere to hide that everything we do is deliberate, conscious, and rational. But the reality is, when we know we're being watched, when we know we're under observation, that in and of itself has a chilling and conforming effect in our behavior. Not just on a deliberate level, but on a subconscious level. So in moments where we know we're being watched, and if we feel like I've got nowhere to hide, so I'm happy to just do what I do and let the government watch. We are not even aware, and this is what social psychological research shows, and I talk about a lot of this research in the book. Even the knowledge or awareness that you're being watched has an impact on our behavior. Makes us more compliant with group think, and less willing to challenge conventional ways of thinking and doing, and that's, again, deeply problematic. And so there's just a few, I mean, there's others who have critiqued this argument before, but those are, I think, some key arguments that I think come from the book that show that this kind of argument is deeply flawed. And is there even a real difference in the way that the public acts when comparing knowing that the government is watching you and being paranoid that they might be watching you? So that reminds me of sort of a famous line from Kurt Cobain, the late singer and songwriter for Nirvana, who families he said, just because you're paranoid doesn't mean they're not following you. On the one hand, there's going to be certain groups and communities in society that have very good reason to think that they are being watched, or they've been put on notice that they've actually been surveilled and targeted under watch. And where you know for a certainty that you or the group or community that you're associated with has been targeted for surveillance, for prosecution, for other kinds of persecution, the Chilean effects and the impact of that threat, the impact of that awareness is profound. And I talk about this a little bit in the book in chapter 8, something I call the Chilean effects curve, where the more targeted and personalized the threat, be it a surveillance threat, be it a targeted legal threat, be it a personal threat of violence, for example, the more personalized the threat, the greater the Chilean effect on the target of the victim. So when a person, and let me give you an example, is that I, one example that I talk about in the book, for example, Muslim American communities after 9/11. So after the al-Qaeda terrorist attacks on the towers in New York City, the World Trade Center in New York City in September 11th, 2001, you had U.S. Security Services, National Security Surveillance, the FBI and other anti-terrorist sort of task force, targeting often Muslim American communities for surveillance. And they were made aware of this surveillance, many of their communities were targeted and singled out for surveillance, and there's been a lot of research done how this had a really chilling and corrosive effect on those communities, how difficult it was when you knew that your communities would be targeted by state and law enforcement for potentially being a national security threat to the country. However, there are also cases to come back to your question where you do not know for certain that you're being watched, but maybe you're simply paranoid about that possibility. And what I think research shows, including my own research, like my Wikipedia study, even where you just are aware about the possibility of mass government online surveillance and offline surveillance, and that you're not necessarily targeted, but you're aware of the possibility that the government might be tracking what people are reading on Wikipedia, what people are searching for on Google search, or what you're reading in other parts of the internet. If you're aware of that, that too can have a chilling effect on what you're willing to seek out, read, and do. And that's what I found in my Wikipedia study, pre-imposed Snowden revelations, and what other researchers have found who've reproduced a similar kind of chilling effect on it. surveillance effect, pre-imposed note, in other contexts, replicating my findings and other contexts. And I think that's a different kind of chill, because in that context, reading Wikipedia articles isn't illegal. It's just content on there that if you knew the government is looking over your shoulder or tracking what you're doing, or maybe tracking what you're doing, you might say, "Let me avoid reading about Alcada," because I don't want to get caught up in a mass government drag net, and then maybe I end up in a no-fi list at some point down the future. Intentionally, unintentionally, just people want to avoid trouble, so they say, "Let me just avoid these really privacy-sensitive articles, and avoid searching for them on Google." Because who knows who's tracking and collecting my browser history out there? Let me avoid them on Wikipedia, because who knows who's tracking that? And I'll just avoid those more controversial topics, even where you have no good reason to think that you're being watched, you're just a citizen, and you're avoiding these topics on one. That's, I think, a case where people might be paranoid, you might explain that, but in other cases, you may say not paranoid, just being more cautious with the possibility that you're being watched. And that's an example of a broader chill that I'd talk about in the book, but even when it's not hard, you'd have an impact on behavior at a great even society level scale. And one interesting idea that you described in your book was the surveillance inequality between the poorer and wealthier neighborhoods, with the poorer being significantly more exposed to drones. How does this even begin to manifest? Yeah, so this is a great point, and it's what I, when I talk about in the book, I go through what I call the dangers of chilling effects, understanding how the impact on individuals and how they impact on broader society, and one of the real deeply troubling implications of chilling effects that I've found in my own research on chilling effects, and what other research have similarly found is a few things. On the one hand, and this is to go back to the example that you mentioned, drone surveillance, and places, so there's, there's two angles to this, or two sides of this equality problem. Where chilling effects perpetuates inequalities and inequities within society, and there's two main ways that this happens. On the one hand, the threats that cause or lead to chilling effects, like surveillance threats, like targeted law enforcement, or targeted legal prosecution threats, personal threats of violence, or the risk thereof. These threats disproportionately typically victimize women and minority populations. That is, people who are poorer, or come from more marginal communities, immigrant communities, working class communities, poorer migrant communities, disparate communities, all these, they tend to be victimized by police surveillance and targeted prosecution disproportionately. They're disproportionately targeted by all forms of online harassment, abuse, stalking and harassment, both in the public and private sector actors. They're disproportionately victimized by police usage of new kinds of emerging tech, including AI, like in this, the McDonald's case that I talk about in the book, but also new kinds of police drone surveillance programs. drones are being increasingly used in certain communities to police those communities, and they tend to be established with flight paths and surveillance operations, which disproportionately target poorer communities, poorer regions and poorer areas. The threats that cause chill and the impacts that I talk about, those threats disproportionately target these communities or poorer individuals and poorer groups. And that, of course, perpetuates the same kind of inequities in society, because if you're more likely to be affected by these threats, you're more likely to be chilled and all the attendant corrosive impacts that it has on who you are as a person and your personal relationships, your economic opportunities, all of that. It's not just that, the threats, what I also found in my own research is that people from more marginal backgrounds, women, minorities and impoverished communities are disproportionately chilled or impacted by these very threats. That is, we have two individuals, one who comes from a poorer background or come from a more marginalized community, and somebody who is surveilled or targeted by the same threat that comes from a more established community that is more wealth and more family community or even governmental support. The person from the poor or marginal community is going to be disproportionately chilled. That is, their life, their speech, they're going to experience greater forms of self-sensorship, greater profound impacts on their lives and they'll experience other kinds of more broader forms of chill, like for example, systems of violence, kind of an institutional chill that I talk about in chapter five in my taxonomy. That's more likely to affect those groups, so its chilling effects and the threats that cause them can perpetuate inequality in society, both by having threats disproportionately target these same populations over time, but also chilling effects disproportionately affect them. And there's different explanations for that, it's because they don't have the same resources, they can't seek legal advice when needed, they don't have the same family support, community support, broader governmental services support, but also they might have experienced forms of surveillance, harassment, personal abuse in the past, which reaches also shows, makes it more vulnerable to chill and the same kind of curse and impacts to threats in the future. So if you've experienced personal abuse in the past, you're going to have experienced chilling effects in more profound ways, that's also what the research shows. To me, there's levels to surveillance, whereas the most simplest form of this is just setting some prison guard outside of your cell and just watching you what you do, a few rungs up that would be the government sort of checking your activity, seeing all that you're doing online, and one application of that is as you said before the facial recognition technology, which is arguably more invasive than anything that's came before it. To me, it seems like we're at the peak of this sort of surveillance ladder, is I can't really think of anything that can go any farther than it has now. Yeah, that's a great point. Have we reached sort of apex surveillance, are we at peak surveillance? I go back and forth on this question, but what I tend to think is, is that there's even further we could go, and whether you view that as greater heights to go in terms of the capacity for surveillance, or maybe we could sink even further. I try to speak to some of that potentially darker future in chapter 10, which I get into what I call the future chilling effects, and how to stop it, and I speak to it, I use as new term supervalence, which, shorthand you might say, is a way of speaking to forms of AI-enhanced surveillance, but even more than that, AI-enhanced surveillance that incorporates forms of facial and bodily tech identification, uses mass forms of data-based saying, powerful artificial AI-driven engine that's powering these systems, but also highly personalized enforcement. And today increasingly, you already have societies where surveillance is sort of accepted, right? and China has hundreds of millions of dollars. of surveillance cameras through the country, but they are already increasingly deploying AI to enhance and to power that already existing surveillance system. Meaning you might look at China and say China has achieved peak surveillance, but there is still further that they are going to be able to go. And where I can see this going is where you end up with a more totalizing system that combines both public, space, facial and bodily recognition, surveillance, power by AI. And that ends up being combined with forms of online data-driven surveillance, but also forms a personal surveillance of any kind of like digital activities you're done in the province of your own home. If you're doing them online, and you can even imagine a future where the internet of things, right, that we have smart appliances around our home, even those will be part of a broader surveillance apparatus. And while I've mentioned China, the reality is these systems are increasingly being deployed in the United States, Canada, and across the West. Certainly AI-enhanced surveillance is already being deployed and experimented with at home and abroad, not just in China, not just in the East, but across the West. And so while already we are in a darkened spot presently, unfortunately, I feel like things could get worse if we don't act quickly. I think the aim, one of the messages that I try to get across in the book, is we need to end our typical complacency about these technologies because, for example, when it came to social media and the internet, we took a very social media, in particular, a very hands-off approach, and now a lot of the attendant problems are very difficult in ways you might argue socially intractable. So I think it's attendant upon us to be more thoughtful and proactive about AI surveillance and other kinds of privacy threats before things get any worse because as bad as things seem, I still think they can and will get considerably worse. And I suppose it's only getting more opportunity to get worse daily as more data is created. Once the statistics in your book stated that I believe in 2003, human history had created something like five exabytes of data, and nearly ten years later, that amount of data was being created every two days. What does that scale mean for privacy? I mean, a pessimist might say like, how is privacy even possible in a world with that scale of big data, but I think what it really means is not, as always, I mean, this is something we often do. With every new technology, new form of data that comes online, people like declaring the death or end of privacy, which to me is a deeply problematic trope because it just creates a card flash for those who have an interest in demolishing privacy, just collecting information about us and profiling us without constraints or with restraint. What it, in my view, means for privacy. It means that we need to get off our hands. It means, first and foremost, that this is a complex problem that touches every facet of our lives, meaning that if we don't get off our hands and we don't get more robust in our both personal habits when it comes to managing our own choices about our information and data, but it also means that we, as a society, need more robust public policy responses to big data, its generation, collection, and retention over time, and we need a more robust public policy response to that. If we don't, then we're going to end up in a scenario where forms of even more creeping mass surveillance that touches all aspects of our lives becomes a reality. And I think that's a life that's a reality we don't want to be living in. We think we understand what that looks like, and that will just become normalized to it. But I think no one is ready for that future. If we don't want to avoid it, we need to act now. And I think that the main issue with the policy surrounding this is that there's a misunderstanding of what we should be doing. I feel like there's an idea right now that when you understand a lot of something at its simplest state that can translate into something as it advances, which may have worked in the past, but now is something that's changing as quickly as AI and even the internet, I feel like we need to change our framework of analyzing this. Yeah. I couldn't agree more. I mean, first and foremost, the way that we respond to the public policy challenge posed by chilling effects and the threats like mass surveillance is available as data collection of attention and algorithmic profiling that's being done by the public and private sector every single day. We need new understanding of the threats and the public policy challenge. We need to discard conventional thinking or conventional understanding as I talk about in the book. That's the first step. And that's a key claim that I make in the book and it's a key thing that I tend to achieve by advancing this new understanding. Because without that, the reforms that I talk about or that I recommend, if you're using the only way, I think you're going to think this is necessary, you need a more singular, more modest, even a more hands-off, ex-post-factor response is going to be fine until you understand the scale and the scope of the public policy challenge, you're not going to get anywhere with some of the legal reforms that I'm recommending. Before we talked about the difference in believing you're being surveilled and sort of seeing this happening, if these chilling effects are fundamentally about self-sensorship driven by uncertainty, does that mean the actual surveillance infrastructure matters less than the belief in it? What does this actually say about our ability to ever truly solve the problem if it doesn't matter if the government is watching us or not, if we're all paranoid no matter what? Right. So a few points in this, and this is actually something that came up as a debate in relation to some of my findings in the Wikipedia study, but other findings, where Edward Snowden, June 2013, leaks information about national security, the NSA, National Security Agency, the United States, US government, governments across the West are engaged in forms of mass online and telecommunications surveillance for national security reasons. And my study shows that post those Snowden revelations and due to those revelations being covered intensely by mainstream media, you have a new public awareness about these government processes and that newfound awareness has a chilling effect on what people were willing to read and search for online. And some people reading that study came away with the conclusion that, "Oh, who is the blame for this? Is it the government? Or is it really Edward Snowden?" It's Edward Snowden's fault because he's the one that brought information to light about this abusive and minicases off an unconstitutional and legal forms of mass surveillance. We did lead to the intense media coverage in this newfound public awareness, which in turn had this negative effect on people's behavior. Do we blame the government or do we blame Edward Snowden? And the way I always thought about that question and that debate was, is that it's a real red herring, to the extent that the cause of the surveillance chill and other effects there isn't journalism, isn't disclosures about illegal and unconstitutional government surveillance programs. It's the surveillance programs itself if you ask people why they're avoiding sensitive topics They're not gonna say I'm avoiding them because of Edward Snowden. They're saying I'm avoiding them because of the potential that government's watching me and It's not like there wasn't already some awareness of these secretive National Security programs pre-Snowden people knew they existed There's just less information about the nature scope and and and the details of some of those programs But there are certainly groups who knew they're being targeted had a basis for it because They were prosecuted and some details would come out in some of the Government disclosures in the in the context of various government cases or it would be released by government reports all of that So the point is here uncertainty Which can be bred by secrecy right is only one factor in Causing chilling effects in the book. I talk about three other key chilling effect factors. One is Observation or surveillance or just awareness of the possibility you're being watched itself awareness of being watched or even the possibility of being watched has a chilling and conforming effect on your behavior Second, these are uncertainty is certainly a factor but not the only one Thirdly, personalization the more personal the threat the greater the chill and lastly Entities with greater power and authority have a greater chilling effect or have a more chilling effect on On people's behavior due to that power and authority so these are all key factors that can lead to chilling effects And so in the context of a government surveillance program you might have some people who are paranoid about being watched and Might be paranoid about it so therefore they shift their behavior accordingly Because they have uncertainty about what the government might be doing But even where you have cases where there is less secrecy and less uncertainty you can still have a chilling effect Even where there's no uncertainty but what's happening if you know that the government is targeting certain groups and you think okay They're not targeting me, right? They're targeting Muslim Americans Even the possibility that you could be caught up in that mass kind of surveillance can have a chilling effect If you know that or if you're a part of a group you're an activist or a protester and you know that there's a Possibility that you could be watched by those same government programs because you'd be viewed as a threat to order instability All of those factors can impact your behavior as I argue in the book so even though There is an impact from awareness right Whether government is surveilling you or not surveilling you. Yes, you can be chilled Even when you're not being watched and that's sort of the point of Foucault's Panopticon and the Panopticon jail that Jeremy Bentham famously designed in the 19th century and Mikaou Foucault the French philosopher You you know used as a metaphor for surveillance in modern society this notion that we're all potentially being watched and that on certainty of being potentially under surveillance has a Disciplining effect on our behavior and he took that as a metaphor for Surveillance in modern society. I think that's right But I think the fact that we know for certain that government and private sector entities are engaged in targeted surveillance do Target protesters do target students in a lot of cases. We're seeing in the United States and Target people who are dissenters Target minority populations Or even there's the mass population itself for tracking for national security reasons and to keep people in check to Shape and make entire populations more easy to manage right That should be the focus of a concern. So it does matter Even though you can have a chilling effect when the government's not even watching you just the possibility of it The fact that they are and the fact that the private sector is watching the fact that extremist bad actors are weaponizing stalker wear weaponizing trolling and other kinds of personal threats to genuinely Target people and we're aware of it because of the stories from victims and from important reporting from media and journalists When you combine those two it does matter that the programs exist. It does matter that people are being targeted Because it all increases the chill and the impact in the long run certainly for those who are in fact being targeted, but also for the rest of us as well And I suppose that it's easy to look at this sort of idea and blame it fully on the government Don't get me wrong there a foundational force to this But I feel social media as you've touched on is also a significant player in this if you Publish something on social media it gets a million views and it's contrasting a bunch of people's opinion people are going to start sending you death threats So when you just sort of approach this topic from multiple Sides absolutely absolutely in fact one of the The key points. I try to get across in the book is you know, it's It weighs my own research in the space, you know begins with a focus on You know government as being you know the the the threat actor But in the time from the Wikipedia study to the present the threat is clearly become more diversified and far more complex I think it was always that way, but developments over time have shown that it's not just government but Surveillance capitalist companies or companies who are weaponizing social media and Make money and profit off of engagement trolling threats abuse In examples like you just mentioned if you know a post goes Viral online you get a lot of engagement it keeps people on the platform and People are often arguing they are threatening and a lot of that abuse is part of the engagement that That is monetized by the platforms themselves so private sector monetization models incentivize the same Surveillance and the same kinds of amplification of threats that impact that our speech and behavior that impact and show victims Surveillance capital is another part of it But I think we've also entered a new era where you have a unique synthesis of state and tech power You have new companies like Palantir oracle And you know name any of the Elon Musk companies which represent a unique synthesis between Weaponization of state power with technology power and we're using it to Target and victimize anyone who would descent criticize any perceived enemy of the state any perceived enemy of the company or a private powerful private individual and So if you're just focused on government you're going to miss all of this all of these other threats social media New emerging forms of technology Joan surveillance the kinds of online harassment and abuse that are carried out often not by private sector companies not by government but by individuals or groups who enjoy engaging in trolling threatening Stocking behavior and victimizing people often women of minorities, right? So if you're only focused on government You're going to miss the complexity of the problem. You're going to miss the broader impacts and that's a key message of the book That's why conventional understanding which so focused on government? So focused only on overreaching lies, Mrs. privacy threats, Mrs. personal threats Mrs. A whole host of other threats that can have the same chilling and repressive impact on our behavior and deeply troubling corrosive implications for democracy Say this does sort of go farther than it has right now in the world keeps Getting more convoluted and like crazy Do you see the government utilizing fear more as an elementary factor to control the population even more than now? Yeah, unfortunately I do Because what the research shows when it comes to chilling effects is you know, it's all about threat perception Right people disengaging people self-sensituring people Leaving their jobs leaving their homes silenced Victimized all of this is due to to perceptions of threats. And a way of keeping people, keeping people compliant, silent and obedient is to ratchet up perceptions of threats, especially personal threats. Or if you can show it's not just that, there's this threat out there, but instead you or your family or your community is specifically targeted that increases the fear and increases the chill, right, of that. And unfortunately with technology today, the capacity to weaponize fear and related threats is unprecedented compared to other times, right. It's not just having platforms like social media that can amplify threats, amplify propaganda at untold scales, right. You know when a lie travels around the world like eight times before the truth is still trying to catch up, right. So the line goes. It's not just that it's also gender to AI today where you have now very sophisticated automated text generation systems and platforms that can easily mimic human interaction, speech, language production that can again produce lies, propaganda and therefore amplify and spread fear with very little scarcity of resources, right. Online it can be done cheaply and at mass global scale. And because it's so easy, because it's so cheap and because of the impact on populations and people in psychology is so profound, I can easily see it being over time increasing weaponize. I mean that's really what a lot of these far right populist movements are all about, whether it's Trump as in the United States and there's similarly far right populist movements growing in Canada, there's certainly gaining strength and momentum across Europe, Asia and in other places around the world. A lot of the politics of those is based on fear of certain threats to community immigrants. And that's when nationalism and populist, nationalist reactionary politics is creating threats or creating political movements based on fear of the other fear of certain other communities or other people who are threatening them, other families or the communities or their identities. And that can lead to extremism, it can lead to more hate and polarization, fragmentation within society, due to conformity to those kinds of norms and can also lead to greater groupthink and greater conformity with broader society. So you have this sort of double edged sword of both greater groupthink within society but also potentially great tribalism and extremism as well. And they're all a product of the same fear and threats that lead to and cause chilling effects. I am under the view that big tech companies basically run the world. If Mark Zuckerberg wanted anything done, he could get it done with Instagram just by provoking stuff through there. - What does this mean when these tech companies start working with governments? - Yeah, so it's deeply, deeply troubling. Because when I come back to what I call in the book the chilling effect factors, when you combine the power of the state which has law-making powers, policy-making powers in some cases unlimited resources, ways of wielding different threats to chill and shape behavior at population level scale. When you combine those state powers with the immense powers of some of these tech companies which has data about us, which profile us, which reach into the most intimate parts of our lives, our intimate conversations in our homes online and offline with our significant others. And that kind of data about us and our running platforms that concern so many parts of our day-to-day lives. When you combine the power and data access and profiling capacity of big tech with state power, you create this deeply, deeply troubling cauldron, unique synthesis of state and tech power that ratchet up each of those different threats. So there's sort of two sides of this. On the one hand, states and big tech have so many more resources in order that they can wield and weaponize to chill. So they can weaponize surveillance and observation and data collection. They can make things more personalized with data personalization and algorithmic profiling to make the threat more personal, that can track you in greater numbers of ways, that can watch and observe your number of ways. And both government and big tech both have immense power and authority in modern advanced information tech in the digital age. And so when you synthesize these two powerful actors that synthesized outcome or this combined outcome, the impact on us, individuals and on society, right, and on citizens, in my view, is profound. And that's one of the key things that we need to guard against is these tech state cartels that are already consolidating power and converging power and gathering power to themselves and increasingly collaborating and doing joint ventures, volunteer, Oracle, XAI, all of these companies, Amazon, they're all interested in the same. And they all have a collective interest in a weakened regulatory state. We can privacy protections. We can regulations and we can rule of law and we can democracies. Because these are the only institutions that can hold them to account, right? So when I mentioned the state in ways, yes, the state and governments have these great powers, but they also are one of the only ways within democracies that we can hold these other actors accountable, whether it's political actors or governmental actors or it's big tech in the private sector, having robust regulations, robust law enforcement, loco, robust prosecution, civil and criminal for abuses of the tech, for excessive, for excesses. So we need healthy democratic governments who are responsive to citizen rights and concerns to take action. And that's the only way we're going to address these combined state and governmental cartels. And we need to do it quicker because as we speak, they are synthesizing, combining and collaborating and consolidating their power and their capacities. And the longer we wait, the harder it's going to go. - Thanks to Jonathan Panney for joining me for this episode. His book, "Chilling Effects" will be linked in the description of the episode. If you're interested in further conversations like these, subscribe wherever you listen. Thank you.