Content Moderation At Scale Is Impossible: Twitter Locks Accounts For Fact Checking The President

from the same-shit,-new-day dept

Another day in which we get to explain how content moderation is impossible to do well at scale. On Wednesday, Twitter (and Facebook) chose to lock the Trump campaign's account after it aired a dangerous and misleading clip from Fox News' "Fox & Friends" in which the President falsely claimed that children are "almost immune" from COVID-19.

People can debate whether it was appropriate or not for Twitter (and Facebook) to make those content moderation decisions, but it seems perfectly defensible. Claiming that kids are "almost immune" is insane and dangerous. However, where things get sketchy on the content moderation front is that Twitter also then ended up freezing the accounts of journalists and activists who fact checked that "Fox & Friends" nonsense:

Or in the case of Bobby Lewis from Media Matters, Twitter suspended his account for simply mocking part of the Fox & Friends clip, noting that when a host asked the President to "say something to heal the racial divisions in America" Trump couldn't do it and could only brag about himself:

Now, tons of people are reasonably pointing out that this is ridiculous, and arguing that Twitter is "bad" at content moderation. But, again, this all comes just a few weeks (has it been a few weeks? time has no meaning) since Facebook, Twitter, and YouTube all received tremendous criticism from people for not being fast enough in pulling down another nonsense video -- one that Breitbart livestreamed of "doctors" spewing utter nonsense about COVID-19 in front of the Supreme Court. Indeed, at least week's Congressional anti-trust hearing, Rep. David Cicilline lit into Facebook for leaving that video up for five hours, allowing it to get 20 million views (meanwhile, multiple Republican representatives yelled at Zuckerberg for taking down the video).

So, if you have some politicians screaming about how any clip of disinformation about COVID-19 must be taken down, it's no surprise that social media platforms are going to rush to take that content down -- and the easiest way to do that is to take down any of the clips, even the clips that are people debunking, criticizing, or mocking the speech. Would it be nice if content moderation systems could figure out which one is which? Yes, absolutely it would. But doing so would mean taking extra time to understand context (which isn't always so easy to understand), and in the process also allowing the videos that some say are dangerous by themselves to remain online.

In fact, if Twitter said to keep up the videos that are people fact checking or criticizing the videos, you create a new dilemma -- in that those who want the dangerous nonsense to spread can, themselves, retweet the videos criticizing the content, and add their own commentary in support of the video. And then what should Twitter do?

Part of the issue here is that there are always these difficult trade-offs in making these decisions, and even if you think it's an easy call, the reality is that it's going to be more complex than you think.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: como, content moderation, content moderation at scale, covid, donald trump, fact checking, impossible, journalism, reporting
Companies: twitter


Reader Comments

Subscribe: RSS

View by: Time | Thread


  1. identicon
    Jeff, 6 Aug 2020 @ 4:06pm

    Serveral big issues here

    link to this | view in thread ]

  2. identicon
    TechDirt'sCow, 6 Aug 2020 @ 4:11pm

    moo.

    You made an udder typo!

    "spewing under nonsense"

    Moo.

    link to this | view in thread ]

  3. icon
    Stephen T. Stone (profile), 6 Aug 2020 @ 4:12pm

    Things will always get messy when you need a scalpel and all you have is a sledgehammer.

    link to this | view in thread ]

  4. identicon
    Anonymous Coward, 6 Aug 2020 @ 4:57pm

    Fals positives are inevitable....

    ....and moreso the more effort being put into finding the genuine content being moderated.

    I still remember the time google delisted my site which offered a fix for a common browser hijack which loads of my friends' parents kept getting infected with. I got tired of manually wlaking them through fixing their systems every time, and so created it to streamline the process to just sending them the url. My site ran a simple script which unlocked the registry for editing, then patched out the malicious registry entries it put there preventing any antimalware software from running, then downloaded and ran such a piece of software to clean it out.

    But my site got flagged as a malicious attack site and so was delisted. Because it had enough that was comparable to the very things it existed to fix. Because they were looking for those kinds of sites. Which is what we want them doing.

    link to this | view in thread ]

  5. identicon
    William Null, 6 Aug 2020 @ 5:25pm

    If it's impossible...

    ...why even try? Are they and Mike fans of Sisyphus by any chance?

    link to this | view in thread ]

  6. icon
    Stephen T. Stone (profile), 6 Aug 2020 @ 6:00pm

    why even try?

    Because “not trying” gets us 8chan.

    link to this | view in thread ]

  7. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 6 Aug 2020 @ 6:08pm

    Sooner or later we will all get cv19
    Children will get it
    Children dying from it is basically non existent
    So why don't you all get off of TDS
    and live in the real world

    Summary of Findings Reported on 7/30/20:
    (Note: Data represent cumulative counts since states began reporting)

    Cumulative Number of Child COVID-19 Cases*
    338,982 total child COVID-19 cases reported, and children represented 8.8% (338,982/3,835,573) of all cases

    Overall rate: 447 cases per 100,000 children in the population

    Change in Child COVID-19 Cases, 7/16/20 – 7/30/20
    97,078 new child cases reported from 7/16-7/30 (241,904 to 338,982), a 40% increase in child cases
    Testing (8 states reported)
    Children made up between 3%-11.3% of total state tests, and between 3.6%-18.4% of children tested were tested positive
    Hospitalizations (20 states and NYC reported)

    Children were 0.6%-3.7% of total reported hospitalizations, and between 0.6%-8.9% of all child COVID-19 cases resulted in hospitalization
    Mortality (43 states and NYC reported)*
    Children were 0%-0.8% of all COVID-19 deaths, and 20 states reported zero child deaths

    In states reporting, 0%-0.3% of all child COVID-19 cases resulted in death

    *Note: Data represent cumulative counts since states began reporting

    link to this | view in thread ]

  8. identicon
    Anonymous Coward, 6 Aug 2020 @ 6:42pm

    Content Moderation At Scale

    Content moderation at scale is a lot easier if you try to block obscene, lewd, or violent speech, instead of opinions.

    link to this | view in thread ]

  9. identicon
    Anonymous Coward, 6 Aug 2020 @ 7:15pm

    Re: Content Moderation At Scale

    I guess you don't come around here all that often (that or you don't bother trying to understand what you read here). Obscene or lewd postings? Breast feeding mothers' advocate posts are blocked. As are post of a lot of classic art. Violent speech? Post trying to expose tyranny, oppression and war crimes get suppressed. Content moderation is like everything difficult - it seems easy if you willfully ignore the difficult parts.

    link to this | view in thread ]

  10. identicon
    Anonymous Coward, 6 Aug 2020 @ 7:21pm

    Re:

    "Children made up between 3%-11.3% of total state tests, and between 3.6%-18.4% of children tested were tested positive

    Children were 0.6%-3.7% of total reported hospitalizations, and between 0.6%-8.9% of all child COVID-19 cases resulted in hospitalization"

    So...not immune then and your cult leader is a liar, as usual, got it.

    link to this | view in thread ]

  11. icon
    Stephen T. Stone (profile), 6 Aug 2020 @ 7:23pm

    Sooner or later we will all get cv19

    You can resign yourself to this fate all you want. I’ll be wearing a facemask and social distancing.

    Children will get it

    Yes, and their getting it might either kill them or fuck them up later on in life, which seems like a good enough reason to prevent children from getting it. (The fact that they could unknowingly spread it to people who have a greater chance of dying from COVID is another good reason.)

    Children dying from it is basically non existent

    That’s a funny way of saying “yes, children have died from COVID, but who gives a shit”.

    why don't you all get off of TDS

    Pointing out that Donald Trump and his administration have either unwittingly botched or knowingly fucked up their response to a pandemic is not “derangement”. Claiming that the virus will “go away like things go away” during a spike in both case numbers and deaths, opening schools and turning them into petri dishes for the virus, caring more about the economy than about the health and well-being of the general populace because of a goddamned political campaign — all those things are deranged.

    How many of the 160,000 COVID deaths could have been prevented if the pandemic response from the federal government hadn’t been a complete shitshow at the outset of the outbreak? More importantly: How many of those 160,000 deaths do you consider “acceptable losses”?

    link to this | view in thread ]

  12. icon
    Stephen T. Stone (profile), 6 Aug 2020 @ 7:29pm

    This sort of thinking reminds me of the Simpsons episode where Marge fought to take “Itchy & Scratchy” off the air, but relented when she couldn’t justify censoring that cartoon while supporting a museum displaying the famously naked statue of David.

    You can’t pick and choose what counts as “obscene, lewd, or violent” without catching a lot of content that might be newsworthy/of interest to the public. That explosion in Beirut injured thousands, killed more than a hundred people, and damaged property to the point where the city damn near looks like a war zone. Do videos of that explosion count as “violent” content?

    link to this | view in thread ]

  13. identicon
    Anonymous Coward, 6 Aug 2020 @ 7:34pm

    Re: Content Moderation At Scale

    What about the obscene, lewd, and violent opinions?

    link to this | view in thread ]

  14. icon
    That One Guy (profile), 6 Aug 2020 @ 8:05pm

    Re:

    Fuck off plague rat.

    link to this | view in thread ]

  15. identicon
    Anonymous Coward, 6 Aug 2020 @ 8:55pm

    Re: Content Moderation At Scale

    You'd probably end up in the trap of "what's an opinion" versus "what's a fact". It's not like we live in the days where knowledge comes from the # volume of World Encyclopaedia Britannica Books where you have a select few researchers hold the keys to compiling knowledge, where anyone who seeks "facts" can just reference that.

    link to this | view in thread ]

  16. icon
    techflaws (profile), 6 Aug 2020 @ 9:42pm

    Pot meet kettle

    "Live in the real world" coming from a plague rat that gets his info from Faux News is kinda rich, no?

    link to this | view in thread ]

  17. icon
    techflaws (profile), 6 Aug 2020 @ 9:43pm

    Re: Content Moderation At Scale

    Who defines what constitues obscene, lewd, or violent speech? You? Really?

    link to this | view in thread ]

  18. identicon
    Anonymous Coward, 6 Aug 2020 @ 10:06pm

    Re:

    Oh, this genius again. Inject yourself with bleach yet?

    link to this | view in thread ]

  19. icon
    PaulT (profile), 6 Aug 2020 @ 10:26pm

    Re:

    "Sooner or later we will all get cv19"

    I'll take later, after the vaccines are available, the spread has been controlled and we have medical facilities that aren't overrun by Karens who just realised they aren't immune because they shouted at the people telling them to wear a mask, thanks.

    "In states reporting, 0%-0.3% of all child COVID-19 cases resulted in death"

    Cool. But, why do morons like you think that it's only the health of the child themselves that's at risk? Why do you morons think that death is the only outcome worth worrying about, when there's mounting evidence of the disease causing other long term health problems?

    Oh, that's right, you're a Trump fan, so the facts have been carefully skewed to hide the truth.

    link to this | view in thread ]

  20. icon
    PaulT (profile), 6 Aug 2020 @ 10:33pm

    Re:

    "Yes, and their getting it might either kill them or fuck them up later on in life"

    ...and before they're diagnosed they will spread it to family, friends and anyone else in the community around them who might not be lucky enough to develop mild symptoms.

    That's one problem with these people - they're incapable of thinking beyond the individual.

    "How many of the 160,000 COVID deaths could have been prevented if the pandemic response from the federal government hadn’t been a complete shitshow at the outset of the outbreak?"

    There were studies that showed that most of the deaths in the first couple of months would have been prevented had action been taken just 2 weeks earlier. On the day that effective action should have been taken, despite numerous other countries having already locked down, Trump was still referring to it as a "democrat hoax". Oh, and don't forget that he'd already fired the pandemic response team and sold off a bunch of the necessary PPE before the pandemic started.

    link to this | view in thread ]

  21. icon
    PaulT (profile), 6 Aug 2020 @ 10:34pm

    Re: Content Moderation At Scale

    "obscene, lewd, or violent speech"

    Cool. Now define those things in a consistent objective way that will never block a simple opinion. I'll make it easy for you - you can't.

    link to this | view in thread ]

  22. icon
    Mike Masnick (profile), 6 Aug 2020 @ 11:34pm

    Re: If it's impossible...

    Because not doing anything is also a choice... and also fails.

    link to this | view in thread ]

  23. icon
    Scary Devil Monastery (profile), 7 Aug 2020 @ 1:02am

    Re:

    "So why don't you all get off of TDS
    and live in the real world"

    The real world, where covid is infectious? That real world where if enough people get sick in one go hospitals get overwhelmed and need to start bringing the dead out by freight hauler?

    That real world where Covid has, in the US, already killed more people than the US lost soliders in World War 1?

    That real world where you are currently facing the death toll of a 9/11 every two days?

    That real world where the US, alone, now stands for 25% of the global death toll from Covid while having only 4% of the global population?

    THAT real world?

    There are over 150,000 dead americans who didn't have to be dead, but WE are the ones suffering a derangement for your president being the man who was at the helm when every other nation managed to reduce their infection rate and death toll?

    I guess that's it. You people are officially a Death Cult now.

    link to this | view in thread ]

  24. icon
    Scary Devil Monastery (profile), 7 Aug 2020 @ 1:08am

    Re: Re:

    "Oh, that's right, you're a Trump fan, so the facts have been carefully skewed to hide the truth."

    Worse. by now the Trump death cult is a religious movement. They'll go to their deaths - or, more likely, the death of family members - to call it all a hoax and close their eyes and ears to the evidence that Fearless Leader might be wrong.

    If anyone had told me, ten years ago, that the world would close its borders against the US because a horde of plague rats were deliberately spreading a lethal pandemic I'd have called that person a liar.

    Today I'm inclined to believe we may all want to retain those closed borders once the pandemic's gone. A nation marching that predictably to the tune of the manifestly insane has to be held at arms length.

    link to this | view in thread ]

  25. icon
    Scary Devil Monastery (profile), 7 Aug 2020 @ 1:12am

    Re: Content Moderation At Scale

    "Content moderation at scale is a lot easier if you try to block obscene, lewd, or violent speech..."

    As some have pointed out it's not that easy. Consider that your criteria would have blocked about 95% of pulitzer-prize-winning news stories as well, including many of the ones which shook the world.

    A picture of a naked child running away from napalm bombardment will trigger every filter in the book, but if it can not be seen and acted on it guarantees there will be more children running - or failing to run from - bombardment.

    The most important news of all is the one which contains thoroughly unpleasant imagery and language which no one really wants to see.

    link to this | view in thread ]

  26. identicon
    Anonymous Coward, 7 Aug 2020 @ 3:56am

    It is genocide. They are trying to kill us.
    Will these people be held accountable for their intentional dereliction of duty? Will their crimes be war crimes?

    link to this | view in thread ]

  27. icon
    Stephen T. Stone (profile), 7 Aug 2020 @ 4:05am

    the Trump death cult is a religious movement

    Kind of a chicken/egg situation, that: The religious group behind a significant part of Trump’s victory and ongoing support — Christian evangelicals — is itself a death cult, since they believe in the Rapture and want to see it happen no matter what. That Trump supporters — who don’t care if they die so long as everyone they hate dies too — have all but turned their support of him into a religious movement is…well, “discomforting” is the most charitable descriptor for that turn of events.

    link to this | view in thread ]

  28. identicon
    Anonymous Coward, 7 Aug 2020 @ 4:45am

    Re: Content Moderation At Scale

    "Content moderation at scale is a lot easier if you try to block obscene, lewd, or violent speech, instead of opinions."

    translation: your life would be a lot easier if you just do what I tell you.

    link to this | view in thread ]

  29. icon
    PaulT (profile), 7 Aug 2020 @ 5:17am

    Re: Re: Content Moderation At Scale

    It's more like: your life would be a lot easier if you ignore the complexities of the real world and pretend there's an easy solution.

    link to this | view in thread ]

  30. icon
    Scary Devil Monastery (profile), 7 Aug 2020 @ 6:23am

    Re:

    "Kind of a chicken/egg situation..."

    Stephen, the sort of chicken imagery your argument invokes is a nightmare drawn by Giger. One laying facehugger pods to, at that.

    "That Trump supporters — who don’t care if they die so long as everyone they hate dies too — have all but turned their support of him into a religious movement is…well, “discomforting” is the most charitable descriptor for that turn of events."

    We've seen that type of personality cult before. And it's equally upsetting each and every time.

    Perhaps it's ironic that Trump himself appears to be less the type of cloth a Hitler or Kim Jong-un is cut from and more resembles one of the faceless minions who so dearly wished they could get away with being the Big Bad.

    link to this | view in thread ]

  31. identicon
    Anonymous Coward, 7 Aug 2020 @ 9:45am

    Re:

    You know that for almost all of the outbreak most schools have been closed, right?

    So they account for 8.8% of cases without being inside, close together, in large groups? With out engaging, en masse, in a highly risky activity?

    20 states reported 0 child deaths - Holy shit, it's like keeping them home from school has kept children safe!

    link to this | view in thread ]

  32. identicon
    William Null, 7 Aug 2020 @ 4:08pm

    Re:

    So what if it does? At least it makes pedos, potential terrorists and other criminals congregate in a single place, making them easier to round up and put behind bars.

    link to this | view in thread ]

  33. identicon
    William Null, 7 Aug 2020 @ 4:20pm

    Re: Re: If it's impossible...

    Fails how? You don't end up blocking anything, so you don't end up blocking fact checking or news reports/classical art. The best solution to bad speech is more speech. You used to believe that.

    Also as a wise person said, when you tear out a man's tongue, you are not proving him a liar, you're only telling the world that you fear what he might say.

    link to this | view in thread ]

  34. icon
    Stephen T. Stone (profile), 7 Aug 2020 @ 4:22pm

    So what if it does?

    Do you want Twitter turning into 8chan? What about Facebook? Maybe you're okay with YouTube, Instagram, or any other social media service you know turning into 8chan, hmm?

    You might not be alone in wanting that, but you’re certainly not in the majority.

    link to this | view in thread ]

  35. icon
    Stephen T. Stone (profile), 7 Aug 2020 @ 4:27pm

    In an ideal world, your bit about more speech would be fine. But we don’t live in that fantasy world. When the bad speech can overwhelm all other speech by sheer volume alone, the best solution is “kick the bad speakers all the way out of the house”. Thinking that’s “wrong” or “unfair” because of your free speech absolutism isn’t a Twitter problem or a Facebook problem. That’s a “you” problem.

    link to this | view in thread ]

  36. identicon
    Anonymous Coward, 7 Aug 2020 @ 5:20pm

    Re: Re: Re: If it's impossible...

    Your glorious free speech Internet would last about as long as it takes the politicians to write laws to clean it up, probalby by handing control to the copyright cartel.

    link to this | view in thread ]

  37. icon
    That One Guy (profile), 7 Aug 2020 @ 6:01pm

    Re: Re: Re: If it's impossible...

    I present to you the same challenge I have posed to others with similar ideas of just not moderating: Spend the next week on 4chan's /b board, going through the entire list of posts at least once a day. You don't have to open any of them but you're not allowed to hide or skip by any, you must look at any images that are posted and read any text.

    If, at the end of the week you still maintain that not blocking anything is the proper choice then people might be more willing to take the idea serious when you propose it.

    link to this | view in thread ]

  38. icon
    PaulT (profile), 8 Aug 2020 @ 2:29am

    Re: Re: Re: If it's impossible...

    "The best solution to bad speech is more speech."

    It's really not in online forums, but this simplistic fantasy does seem to make the round a lot among people who don't want people to be able to police their own property.

    link to this | view in thread ]

  39. identicon
    Anonymous Coward, 9 Aug 2020 @ 3:12am

    Re: Re: Re: If it's impossible...

    The best solution to bad speech is more speech.

    What about removal of speech when it is posted in inappropriate forums or contexts? Only allowing counter speech allows the trolls,bigots, racists and political activists to hijack any forum on any topic, a problem familiar to Usenet users.

    link to this | view in thread ]

  40. identicon
    William Null, 9 Aug 2020 @ 3:36am

    Re: Re: Re: Re: If it's impossible...

    Then move it to another forum section that's more appropriate for tge topic at hand, never REmove it.

    link to this | view in thread ]

  41. identicon
    William Null, 9 Aug 2020 @ 3:41am

    Re: Re: Re: Re: If it's impossible...

    That's the problem with people like you. I am strong believer of "if you don't like it, you don't have to watch/read/listen to it." Don't try to limit what people want to say just because YOU don't like it being discussed. If I don't like some, say, YouTube video, I just click away and don't worry about it. I suggest you trying out the same course of action.

    link to this | view in thread ]

  42. icon
    Stephen T. Stone (profile), 9 Aug 2020 @ 4:43am

    “Just don’t look” is a fine strategy for taking out giant advertising mascots come to life, but it’s a shit way of handling website moderation.

    link to this | view in thread ]

  43. icon
    Stephen T. Stone (profile), 9 Aug 2020 @ 4:44am

    What if the forum owners don’t want to host that speech on any part of the forum — would you support a law that forces them to host that speech and face some form of punishment if they refuse?

    link to this | view in thread ]

  44. identicon
    Anonymous Coward, 9 Aug 2020 @ 7:16am

    Re: Re: Re: Re: Re: If it's impossible...

    You do not want free speech, you want the ability to force your speech onto other people. Just because one platform will not allow your speech does not mean that your rights are being infringed, as other platforms and outlets exist. Other people rights are being violated when you insist on speaking where you are not welcome.

    link to this | view in thread ]

  45. icon
    That One Guy (profile), 9 Aug 2020 @ 7:26am

    'If you don't like the yelling racist losers just ignore them.'

    What a surprise, yet another person presented with the challenge of getting first-hand experience of what a site would look like if sites didn't moderate like they want refuses to subject themselves to what they would foist on others.

    If you don't like the idea that platforms aimed at the general public aren't going to be asshole friendly feel free to find or create your own platform for that kind of stuff, because hate to break it to you but most people aren't going to want to use a platform infested with bigoted losers and/or those that like posting/discussing stuff designed to disgust/horrify people just because they think it's funny, and telling them 'just don't look at/read it' isn't going to fly any more than telling someone eating at a fast-food place to 'just ignore the wildly racist and sexists group two tables over loudly discussing how inferiour those other groups are' would.

    link to this | view in thread ]

  46. icon
    PaulT (profile), 9 Aug 2020 @ 7:47am

    Re: Re: Re: Re: Re: If it's impossible...

    Or to put it another way - don't kick the drunken assholes who are smashing the furniture and driving other customers away, just move them into another room.

    Hopefully this will help you understand how stupid you are.

    link to this | view in thread ]

  47. identicon
    William Null, 9 Aug 2020 @ 10:34am

    Re: Re: Re: Re: Re: Re: If it's impossible...

    Oh, I see a fellow Portland resident.

    link to this | view in thread ]

  48. icon
    Scary Devil Monastery (profile), 10 Aug 2020 @ 3:38am

    Re: Re:

    "20 states reported 0 child deaths - Holy shit, it's like keeping them home from school has kept children safe!"

    I'm starting to think at least some parts of the evangelicals supporting Trump so well are thinking the fact children have been kept safe is the bad part. A partial armageddon is no good at summoning The Rapture, after all.

    link to this | view in thread ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.