Content Moderation At Scale Is Impossible: Twitter Locks Accounts For Fact Checking The President

from the same-shit,-new-day dept

Another day in which we get to explain how content moderation is impossible to do well at scale. On Wednesday, Twitter (and Facebook) chose to lock the Trump campaign's account after it aired a dangerous and misleading clip from Fox News' "Fox & Friends" in which the President falsely claimed that children are "almost immune" from COVID-19.

People can debate whether it was appropriate or not for Twitter (and Facebook) to make those content moderation decisions, but it seems perfectly defensible. Claiming that kids are "almost immune" is insane and dangerous. However, where things get sketchy on the content moderation front is that Twitter also then ended up freezing the accounts of journalists and activists who fact checked that "Fox & Friends" nonsense:

Or in the case of Bobby Lewis from Media Matters, Twitter suspended his account for simply mocking part of the Fox & Friends clip, noting that when a host asked the President to "say something to heal the racial divisions in America" Trump couldn't do it and could only brag about himself:

Now, tons of people are reasonably pointing out that this is ridiculous, and arguing that Twitter is "bad" at content moderation. But, again, this all comes just a few weeks (has it been a few weeks? time has no meaning) since Facebook, Twitter, and YouTube all received tremendous criticism from people for not being fast enough in pulling down another nonsense video -- one that Breitbart livestreamed of "doctors" spewing utter nonsense about COVID-19 in front of the Supreme Court. Indeed, at least week's Congressional anti-trust hearing, Rep. David Cicilline lit into Facebook for leaving that video up for five hours, allowing it to get 20 million views (meanwhile, multiple Republican representatives yelled at Zuckerberg for taking down the video).

So, if you have some politicians screaming about how any clip of disinformation about COVID-19 must be taken down, it's no surprise that social media platforms are going to rush to take that content down -- and the easiest way to do that is to take down any of the clips, even the clips that are people debunking, criticizing, or mocking the speech. Would it be nice if content moderation systems could figure out which one is which? Yes, absolutely it would. But doing so would mean taking extra time to understand context (which isn't always so easy to understand), and in the process also allowing the videos that some say are dangerous by themselves to remain online.

In fact, if Twitter said to keep up the videos that are people fact checking or criticizing the videos, you create a new dilemma -- in that those who want the dangerous nonsense to spread can, themselves, retweet the videos criticizing the content, and add their own commentary in support of the video. And then what should Twitter do?

Part of the issue here is that there are always these difficult trade-offs in making these decisions, and even if you think it's an easy call, the reality is that it's going to be more complex than you think.

Filed Under: como, content moderation, content moderation at scale, covid, donald trump, fact checking, impossible, journalism, reporting
Companies: twitter

There Is No Magic Bullet For Moderating A Social Media Platform

from the it's-not-so-easy dept

It's kind of incredible how frequently we see people who seem to think that the fact that social media platforms are so bad at moderating the content on those platforms is because they just don't care or don't try hard enough. While it is true that these platforms can absolutely do a much better job (which we believe often involves providing the end user more tools themselves), it's still amazing at how many people think that deciding what content "belongs" and what content doesn't belong is somehow easy. Earlier this month, in Washington DC there was the Content Moderation at Scale "COMO" conference. It was a one day event in which a bunch of companies revealed (sometimes for the first time) how they go about handling questions around content moderation. It was a followup to a similar event at Santa Clara University held back in February (for which we published a bunch of the papers that came out of the event).

For the DC event, we teamed up with the Center for Democracy and Technology to produce a live game for everyone at the event to play -- turning them all into a trust & safety team, tasked with responding to "reported" content on a fictional social media platform. Emma Llanso from CDT and I ran the hour-long session, which included discussions of why people chose their decisions. The video of our session has now been posted which helpfully edits out the "thinking/discuss amongst yourselves" part of the process:

Obviously, many of the examples we chose were designed to be challenging (many based on real situations). But the process was useful and instructive. With each question there were four potential actions that the "trust & safety" team could take and on every single example at least one person chose each option. In other words, even when there was a pretty strong agreement on the course of action to take, there was still at least some disagreement.

Now, imagine (1) having to do that at scale, with hundreds, thousands, hundreds of thousands or even millions of pieces of "flagged" content showing up, (2) having to do it when you're not someone who is so interested in content moderation that you spent an entire day at a content moderation summit, and (3) having to do it quickly where there are trade-offs and consequences to each choice -- including possible legal liability -- and no matter which option you make, someone (or perhaps lots of someones) are going to get very upset.

Again, this is not to say that internet platforms shouldn't strive to do better -- they should. But one of the great things about attending both of these events is that it demonstrated how each internet platform is experimenting in very, very different ways on how to tackle these problems. Google and Facebook are trying to throw a combination of lots and lots of people plus artificial intelligence at the problem. Wikipedia and Reddit are trying to leverage their own communities to deal with these issues. Smaller platforms are taking different approaches. Some are much more proactive, others are reactive. And out of all that experimentation, even if mistakes are being made, we're finally starting to get some ideas on things that work for this community or that community (and remember, not all communities work the same way).

As I mentioned at the event, we're looking to do a lot more with this concept of getting people to understand the deeper questions involved in the tradeoffs around moderating content. Setting it up as something of a game made it both fun and educational and we'd love some feedback as we look to do more with this concept.

Filed Under: como, content moderation, game, tough choices, trade offs