The Facebook Files, Part 4: The Outrage Algorithm

This is the Facebook files, a series from the journal. We're looking deep inside Facebook through its own internal documents. If you haven't already heard parts one, two, and three, they're in your feed. Facebook's algorithm is something of a black box. It's a complex set of mathematical equations, all adding up to a mysterious calculation that ultimately decides what you see when you log on or open the app. And in early 2018, Facebook said it was making a big change to that algorithm. So Facebook's algorithm changes all the time. They're constantly filling with it. But this change was a paradigm shift more than just a tweak. That's our colleague, Ki Cheggy. It was a completely different emphasis for what was going into your news feed. And they actually came forward and discussed it, which is not normal for an algorithm change. The major announcement from Facebook, if you didn't notice the overhaul of your news feed overnight, they're trying to enhance your connections with family and friends. Zacharyk writing on Facebook, of course, quote, "Facebook is a lot of work to do, whether it's protecting our community from abuse and hate, defending against interference by nation-states, or making sure that the time spent on Facebook is time well spent." At a congressional hearing after the company overhauled the algorithm, Facebook CEO Mark Zuckerberg said the change would clean up the platform and make it a healthier place. It's not enough to just connect people. We have to make sure that those connections are positive. It's not enough to just give people a voice. We need to make sure that people aren't using it to harm other people or to spread misinformation. We need to now take a more active view in policing the ecosystem, but I'm committed to getting this right. And I believe that people will see real real differences. When Mark Zuckerberg came out and explained this, he described it as something of a sacrifice that the company was going to make. He said it was the right thing to do for the good of humanity and for the good of the mental health of their users, perhaps at the at least in short-term expense of the business. Was Mark Zuckerberg telling the full story about the reasoning behind this change? According to what we saw in the documents, no. An array of internal documents reviewed by the Wall Street Journal reveal an entirely different story behind Facebook's algorithm overhaul. From the documents that we have seen, there was a panic going on inside the company. The panic wasn't over misinformation or harm, stemming from the platform. It was about a troubling trend with Facebook's business. The company was noticing a steep decline in user engagement. Facebook hoped the algorithm change would reverse that decline. It did that and more. The most explosive finding was just how harmful certain aspects of this algorithm change were and how much Facebook knew it. Welcome to the journal, our show about money, business, and power. I'm Ryan Knutson. This is part four of the Facebook files. Coming up on the show, what really drove Facebook to rewire its algorithm? How it made the platform an angrier place, and the team inside the company that tried to fix it. This episode of the journal is presented to you by Accenture. Behind this podcast and the music you listen to, Spotify has a sophisticated advertising business. Accenture is deeply embedded in that business, working with Spotify to reinvent the rhythm of ad sales, building, maintaining, and optimizing. Accenture finds the friction and builds the systems that make things run smoother and faster. The result, less time spent on operations, more time connecting brands with the moments and fandoms that matter most. Learn more at Accenture.com/spotify. People disagree about a lot in this country. Taxes, healthcare, education, but wherever you land politically, I think most of us agree on this. Better information leads to better decisions. That's the mission of USAFAX to make government data clear and accessible, because public data should be easy to access, trust, and understand, so we can use it to make informed decisions. Read and sign our open letter to support public data at USAFAX.org. 2017 was a difficult year for Facebook. So based on the documents that we've had a look at, 2017 was what one Facebook employee referred to as an Anis Horribilis. Anis Horribilis, Latin for a horrible year. The reason the employee called 2017 a horrible year was because Facebook was experiencing a distressing shift in user behavior. Engagement metrics were declining alarmingly throughout 2017 in a bunch of different ways. And what are engagement metrics? Like how much are people commenting? How much are they resharing? How much are they posting original posts about their lives? And that one, original broadcast posts is how they refer to it. That one had been going down for a long time. Was the world aware that Facebook was having these kinds of challenges internally with their engagement dropping? No, it was not known that they had internal alarms ringing about how the engagement was dropping. Interestingly, the documents show that the amount of time users are spending on Facebook wasn't going down. It's just that when they were using the app, they weren't really engaging with it. One presentation, Keach and her colleagues reviewed included side-by-side graphs that show that in 2017, there was a clear downturn in things like likes and reshairs. Why did they think that fewer people were posting and commenting? What was fascinating in the documents is that they admit they couldn't figure out why. They didn't know why. The best they could figure out is that it had something to do with video and something to do with the amount of sort of professionally produced content. And so the best they could do is tweak things sort of away from video, away from professionally produced content. But it really was kind of based on a guess from the documents that we've seen. Why does that matter? I mean, why does it matter if I'm commenting on posts or sharing new information if I'm still spending an hour a day scrolling through the feed and seeing ads? Their fear was that if the video-heavy Facebook old algorithm was basically turning people into zombies who were just passively sitting there spacing out, watching Facebook, but not like doing anything that they would sort of snap out of it. And then stop using Facebook. In one internal training video, Keach saw, an employee said the company had an ethical duty not to turn users into zombies. But it wasn't just an ethical duty. The employee also said there was a business concern. They feared that users would leave Facebook if they thought it was bad for them. It's fascinating to see inside how they reacted to this problem. In general, the whole machine was like humming along pretty much fine. But one team had noticed it and they started sort of a frantic effort to get to the bottom of why this was happening. Once Facebook data scientists identified this engagement problem, the company came up with a plan. To get users to post, comment, and interact more during the time they spent on Facebook, the company would make a major change to what users saw in their news feeds. They would overhaul the algorithm. Over time, we believe people will see more posts from people they're connected to, and less content from publishers. In a Facebook video announcing the change, the company said the pivot was all about making the platform healthier and improving its users' well-being. And over the next year, our team's mission is to help you have deeper, more meaningful interactions with people you care about. Our goal is that the time people do spend on Facebook will be better. The algorithm change changed on a new formula, that the company internally called, "Meaningful Social Interactions." Meaningful Social Interactions, or MSI, as it became known inside the acronym Happy Place that is Facebook. MSI. Sounds catchy. Yes, MSI. What exactly is MSI? It is a number that measures how much a post is interacted with by people who you are close to. The interactions can be things like comments, likes, reshairs, emojis, and then there's another mathematical part of it that's measuring how close the people who are doing that are to you. So it both measures the interactions and the closeness of the people who are doing the interacting, and that has an impact on the number. Facebook used the concept of MSI to create a scoring system. The more likes, comments, and reshairs, and the more those happen among people who are close to each other, the higher the MSI score. And in the very beginning, the goal was simply to get as much MSI as possible. If MSI is high, that means you're not just a zombie, passively scrolling and watching videos. You're interacting, you're engaged. You are more likely to post something to share a little tidbit about your life if you are more likely to get a comment or a like about it. more humiliating than sharing something on Facebook or Instagram and getting no response from anybody. Exactly. The documents that Keith reviewed actually break this MSI formula down. It provides a rare glimpse into the inner workings of the algorithm. So it's actually a pretty simple formula when they rolled it out. A like was worth one point, a reaction or reshare was worth five points. A significant comment was worth 30 points. And then they would add or subtract based on how close the the people who were commenting or interacting were. So you know whether it was a group or a friend or a stranger. For example, an RSVP to an event was only significant if it was a yes. That would be worth 30 points. But if you RSVPed maybe or no, it was only worth five points. Facebook would show these significant interactions to more people with the goal of spurring even more engagement. So did this algorithm tweak solve Facebook's engagement problem? From Facebook's top level perspective, it did work according to what we saw in the documents. It neutralized the decline in commenting and it boosted reshairs and it increased engagement. And it still like had a pretty healthy impact on how many daily users they had. So from with the top line Facebook metrics, it was a success. One slide in an internal presentation explains that original posts which Facebook considers the ultimate metric of success were up following the MSI change after years of continuous decline. But from the perspective of what they said it was for, which was increasing well-being, it was a failure. Facebook's own internal research, which the journal looked at, found that although engagement increased, there was actually no evidence that users felt better about the time they spent on the platform. People actually like what was in their news feed a little less than they did before the change. So people were less happy and those studies persisted throughout 2018 and pretty much consistently showed that same thing. MSI was not that good at showing people what they wanted to see. But it was good at showing them things that might spark a feeling in them that made them want to comment. Exactly. So people inside Facebook started to notice that this was effectively highlighting the very worst kind of content, stuff that was divisive, really negative, and just kind of represented the worst parts of humanity. One reason this happened is because MSI was built around two competing aspects, optimizing for closeness and for engagement. And oftentimes it was engagement that went out. We spoke to some folks inside Facebook and they describe a system kind of like this. So let's say you have two friends. One person is someone you are legitimately good friends with, and you have let's say 500 other friends in common, right? So the closeness of your relationship would be obvious. But this person just posts kind of boring updates about the school board and you know, like when garbage pick up is that week in your neighborhood. Maybe what they had for breakfast. Right. And then you have another friend, some random person from high school you barely ever talked to that you have maybe only 10 other connections in common with. But this person is blasting out nonstop, controversial political content, things about COVID, things about vaccines and masks, and that person's stuff, you know, tends to wreck up lots of comments. In your feed, even though you are better friends with the other person, you're probably going to see a lot of stuff from your random sort of high school friend because what they are posting creates so much engagement because it is so divisive. Keach says there's a specific part of Facebook's algorithm change discussed in the documents, which helps explain that. It's a part of MSI that predicts when something might go viral and then actively pushes that content out to more users. It's called downstream MSI. So downstream MSI is basically the element of this formula that makes it more likely that you would see something in your news feed if Facebook's algorithm has determined a guest that other people that you share with or that see it are going to be more likely to comment on it. In other words, downstream MSI is trying to predict what content is most likely to be repeatedly reshared or commented on and then show that content to more users. So that's like a really different way of filtering what you see, right? It's not based on what you would actually most like to see. Oh, it's most relevant to you or what's highest quality. It's what will get the most comments. And the result of that, it turns out that what gets the most comments is really divisive, outrageous stuff, especially stuff that provokes political anger. Individual users might not have noticed these changes right away, but businesses whose bottom lines depend on being seen on Facebook were paying close attention. One industry paying particularly close attention was publishing, like newspapers and magazines. And in the documents, Keach found that some of them were letting their concerns be known. So in October of 2018, Jonah Peretti, the CEO of Buzzfeed, sent an email to a high ranking Facebook executive saying that MSI was basically backfiring, that this thing that was created theoretically to encourage meaningful social interactions was not creating meaningful social interactions. And in fact, was incentivizing publishers like Buzzfeed and others to make the very worst kind of content, the most toxic kind of content. And he actually provided an example from Buzzfeed of what he was talking about. He said, you know, we do all this great stuff about animals and self-care. It's high quality and should be going viral, but it's not. Instead, what's going viral are posts like this one post that we did called 21 things that almost all white people are guilty of saying, which racked up 13,000 reshairs and 16,000 comments on Facebook and it went completely viral. Mostly of people just yelling at each other in the comments about race and yelling at Buzzfeed for having written it in the first place. And it was that angry comment frenzy that propelled the virality. Buzzfeed declined a comment on Peretti's email to Facebook, but the documents show that Facebook employees took Peretti's criticism seriously. And again and again in the months after that, data scientists keep designing experiments saying, hey, people think that this algorithm change is making Facebook more divisive and sensationalistic. News publishers are saying it's forcing them to make bad content. This is a problem. We need to do something about this. We need to figure out how to change the algorithm to take away these incentives, to create so much as one person called it outrage bait. The documents show that after the algorithm change, divisive content ramped up on Facebook. Sensational extreme posts and stuff like misinformation was now going more viral on the platform. When asked for a comment about the impact of the algorithm change, a Facebook vice president said that any algorithm risks promoting content that is objectionable or harmful to some users. He also said that that's why Facebook has an internal team dedicated to combating that content, the integrity team. After the break, inside that team, and what happened to the changes they propose to fix the problems with the algorithm. This episode is brought to you by Indeed. Running a business means dealing with the dreaded hiring process. And the stakes are high when the right hire could make or break your business strategy. But good news. Indeed, sponsor jobs is here to make things less painful and nerve-wracking. It gives your job post a boost to reach quality talent faster. Plus, it can help put your job in front of exactly who you're looking for based on criteria you set. Like if you need certain skills or certifications. And it works. Sponsored jobs posted directly on Indeed are 95% more likely to report a hire than non-sponsored jobs. So spend less time searching and more time actually interviewing candidates who check all your boxes. listeners of this show will get a $75 sponsor job credit at indy.com/podcast. That's indy.com/podcast. Terms and conditions apply. Need a hiring hero? This is a job for Indeed Sponsored jobs. AI is moving fast across the enterprise. But without visibility, it's just chaos, different tools, different models, different teams using AI in completely different ways. ServiceNow turns that chaos into control. With the AI Control Tower, you see all your AI across the business in one place. What it's doing, what it's done, and what it's about to do. So you stay in control. To put AI to work for people, visit ServiceNow.com. there's something called the Integrity Team. Its job is to improve the quality and trustworthiness of content on the platform. And a few years ago, Facebook assigned a smaller group within the Integrity Team to focus on political content and misinformation specifically. It was known as the Civic Team. And when Facebook made this MSI algorithm change in 2018, the staffers on the Civic Team noticed it was creating a lot of problems. Here's our colleague, Jeff Horwitz, the lead reporter in this investigation, who you heard in the first episode. One of the things that came with sort of the adjustment to meaningful social interaction, which the idea was that passive scrolling with bad and the active engagement with content was good. And one thing that Facebook civic researchers found is that that weighting actually encouraged people to get into fights because nothing produces more comments and flame more. The Civic Team found that the new algorithm was actually pushing misinformation and toxic content. And not only that, but according to the documents, the more times something was shared, the more likely it was to be false. It sounds like almost too simple, but literally every single hop of a reshare, it gets worse. So if a thing's been reshared 20 times in a row, it's going to be 10x or more likely to contain nudity violence, hate speech, misinformation, than a thing that has just been not reshared at all. The documents show that integrity researchers started to propose fixes that could potentially prevent so much negative content from going viral. One idea they had was to dial back downstream MSI, the part of the new formula that made predictions about what content was most likely to go viral and showed it to more users. In practical terms, that means that Facebook might recommend something to you that would go viral, even if they thought you would like a different piece of content more. This was something that the integrity people were like, this is crazy, you're just making everything worse, like this is unstable. The Civic team did some tests and found that if Facebook dialed back downstream MSI, it could reduce the spread of misinformation. So the company agreed to do it, but only for certain sensitive topics, like civic and health information, and only in certain countries, like Myanmar and Ethiopia. So basically this was a toe in the pool. The documents show that in early 2020, the team presented a proposal to Zuckerberg. They were advocating to dial back downstream MSI more and in more places. Zuckerberg was open to testing this, but the company didn't do it. The documents show that Zuckerberg was worried it would hurt viral content that was positive. So the integrity researchers looked for other solutions to tamp down on viral misinformation. The Civic team folks knew they had a viral problem on the platform and just simply, you know, piecemeal enforcing it their way through it wasn't going to work. And so at that point, they started looking at what people titled "Big Levers." Big levers, meaning major overhauls to the platform, changing things that were central to what made Facebook, Facebook. One area the researchers tried to target was the platform's overall speed, and how easy it was for information to spread quickly. They were like trying to figure out ways to like, not stop people from saying what they want to say, right? But just make them pause for just a split second if they wanted to say it. And make it a little harder for that thing that they say to spread rapidly around Facebook. Yeah, exactly. Because if everyone has to pause for a split second, one that adds a lot of time right there, and two, it means that exponential growth curves are going to change. And that's the thing that we're talking about, right? It's like these minor tweaks. If they're at the beginning of an exponential growth curve, kitten might basically change what is the most successful content on the platform. Sources told Jeff that the integrity team came up with different ideas that could slow things down on the platform. And the biggest way they came up with to reduce the platform's speed was to do something radical. Get rid of the reshare button completely. So senior data scientist just went to the executives of news feed and gave a presentation that was like, hey, really an idea for changing this product and addressing all of our integrity problems, kill the reshare button. Just kill it. Kill the reshare button. Just like remove reshairs. It's done. Wow. And you know what, like the data backed up that if you wanted to address problems on Facebook integrity problems, this was the way to do it. It was like just huge gains off the bat. And that proposal from people who were aware of hearing it on the news feed side was like, I mean, it was not going to happen. Facebook was not going to do this. It was so clearly going to hurt the business badly in a way that Facebook was just not going to tolerate. So like it was done in arrival. Jeff found that the integrity team proposed other solutions, like reducing the number of group invitations someone could send in a day. The current limit is 2,250, or capping the number of comments someone could leave. The current limit is 300 per hour. Some of these things were like actually pretty minor on the margins. Like this wasn't going to affect Facebook's annual earnings in any meaningful fashion, even if it was going to tank misinformation in some of these instances. And I think they truly believe that anything that results in less Facebook usage is like kind of a de facto harm to the product. Facebook decided not to adopt these changes either. Again, it was because the company determined they would hurt user engagement. I mean, the experience of being kind of an integrity researcher on this stuff was that you come up with something that seems like it would reduce harmful content or bad behavior in some area of the platform. And then, you know, they'd run the numbers and find that it was not good for user metrics, the standard growth metrics. And then at that point, we'd get shelved. But shelved doesn't mean like totally forgotten about. Shelved means available for use in an emergency. Instead of killing these other proposals outright, the company decided to make them part of what became known as break glass measures. And the idea is you just kind of have it in your back pocket, right? So like these tools worked. They knew they were. They didn't want to use them. Build a tool that they could deploy rather than just changing how they algorithm functioned. Exactly, exactly. It's like, it's like, okay, great. Thank you guys for figuring out that if you limit reshairs after they've been reshared twice in a row, misinformation just plummets. That's a great thing to know. But that in your back pocket, we might need it someday. So the 2018 algorithm change remains largely intact. But about a month ago, nearly a year and a half after the company didn't expand changes to MSI, beyond countries where it had already done so, like Ethiopia and Myanmar. Facebook said it would start doing more tests that deemphasize parts of MSI. A Facebook official said the company is running those tests because when it comes to things like political content, the downsides may sometimes outweigh the benefits. And as for the Civic team itself, right around the 2020 election, it was disbanded. The man who ran it was Samid Chakrabarti. In the final days of the 2020 election, I have been told, this is people, not documents here. I've been told that Samid Chakrabarti, the head of the Civic team was basically pulled aside by management and was told that they would be wanting to now and after the election. And Samid in his kind of going away speech said that, you know, he thought it was for the best. Hopefully it would all be worked out and that everyone was saying this was going to be a good thing for the Civic team and for the work overall. And then he basically said that he hoped that people kept fighting and that they kept raising their voice when they saw the company doing things that struck them as being wrong. Samid Chakrabarti is no longer a Facebook employee and he didn't respond to our requests for an interview. But since this reporting first published, he has spoken out about the 2018 algorithm change. In a series of tweets, he said, treating all engagement equally, irrespective of content, will quote, invariably amplify misinfo, sensationalism, hate, and other societal harms. I wish this weren't the case, but it is so predictable that it is perhaps a natural law of social networks. He also said the challenge is almost a philosophical one. One that Facebook can't solve alone. [Music] Our daily show is back on Monday, but there will be more episodes of the Facebook files coming soon. And on Monday at 1 p.m. Eastern, there will be a live event where you can ask Wall Street Journal reporters and editors about the Facebook series. Sign up at wsj.com/live-QA. That's wsj.com/live-QA. The series is part of the journal podcast, a co-production of Gimlet and the Wall Street Journal. Your hosts are Kate Leinbaugh and me, Ryan Knutson. The series was produced by Pia Gadkari, Max Green, and Martin Kessler, with production help from Enrique Perez de La Rosa. This episode was edited by Catherine Brewer, Gerard Cole, and Annie Roastrasser. Special thanks to Colin Campbell, Anthony Galloway, Mitchell Paselli, Lydia Pole Green, Brad Reagan, Matthew Rose, and Rob Rossi. Our engineers are Griffin Tanner and Nathan Singapac. Our theme music is by so wily and remixed by Peter Leonard, additional music in this episode from Emma Munger, Nathan Stingapok, and Blue Dot Sessions, fact-checking by Nicole Pasolka. Also, thanks to the whole journal team, Priscilla Alabi, Sam Bear, Annie Minoff, Laura Morris, Afifnissouli, Ricky Niveski, Sarah Platt, Willa Rubin, Matthew Sherman, and Matthew Shilts. Thanks for listening. See you Monday. [BLANK_AUDIO]