Techdirt's think tank, the Copia Institute, is working with the Trust & Safety Professional Association and its sister organization, the Trust & Safety Foundation, to produce an ongoing series of case studies about content moderation decisions. These case studies are presented in a neutral fashion, not aiming to criticize or applaud any particular decision, but to highlight the many different challenges that content moderators face and the tradeoffs they result in. Find more case studies here on Techdirt and on the TSF website.

Content Moderation Case Studies: Facebook Removes Militia Event Following A Shooting (August 2020)

from the moderating-groups dept

Summary: Following the shooting of Black man Jacob Blake by Kenosha police officers, protests erupted in the Wisconsin town.

As law enforcement attempted to rein in the damage, citizens aligning themselves with private "militias" discussed taking action during the civil unrest.

Some of this organizing began on Facebook. A Facebook "event" created by the Kenosha Guard account (and promoted by conspiracy theorist/far right website Infowars) possibly caught the eye of 17-year-old Kyle Rittenhouse. Rittenhouse traveled from his home in Antioch, Illinois with his weapons to the protest/riot occurring less than 30 minutes away in Kenosha, Wisconsin. Before the night was through, Rittenhouse had killed two residents and injured one other.

Facebook finally removed the "event" posted by the Kenosha Guard account -- one the account referred to as a "call to arms." Posts by the group asked "patriots" to "take up arms" against "evil thugs." The event was deemed a violation of Facebook's policy regarding "Dangerous Individuals and Organizations." Facebook also claimed it could find no link between the account and this event and Kyle Rittenhouse.

Some viewed this response by Facebook as too little too late. Someone had already apparently heeded the call to "take up arms" and had taken people's lives. According to a report by BuzzFeed, the event had been reported 455 times before Facebook removed it. Four moderators had responded to multiple flaggings with a determination that the event (and the account behind it) did not violate Facebook's rules. During an internal meeting with moderators, CEO Mark Zuckerberg admitted the company should have reacted sooner to reports about the event.

Decisions to be made by Facebook:

  • Should moderators be given more leeway to remove events/accounts/pages (at least temporarily) that have generated hundreds of complaints, even if they don't immediately appear to violate policies?
  • Would a better/more transparent appeal process allow moderators to make more judgment calls that might address issues like this more expediently by allowing them to make mistakes that can be undone if no violation occurred?
  • How does the addition of more forms of content to the "unwanted" list complicate moderation efforts?
Questions and policy implications to consider:
  • Does the seemingly constant addition of new forms of content to "banned" lists invite closer government inspection or regulation?
  • Is a perceived failure to react quickly enough an impetus for change within the company?
  • Are policies in place to allow for judgment calls by moderators? If so, do they encourage erring on the side of caution or overblocking?
  • Does taking credit for actions not actually performed by Facebook make it appear more focused on serving its own interests, rather than its users or public safety in general?
Resolution: The event flagged by hundreds of users was ultimately removed… but not by Facebook, as was earlier reported. The group that posted the event took it down following the shooting in Kenosha. The company appears to realize this delay may have contributed to events that unfolded in Kenosha and has put policies in place to make things clearer for moderators.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: content moderation, kenosha, militia, shootings, wisconsin
Companies: facebook


Reader Comments

Subscribe: RSS

View by: Time | Thread


  1. identicon
    Bobvious, 23 Oct 2020 @ 4:35pm

    Perhaps widen the skillset of moderators?

    This seems like a classic case of "increased chatter", that someone with proper military and/or signals intelligence experience could assist with in terms of managing the moderation. I'm not suggesting this will only be one person or is entirely simplistic, but this can't just be left to people who are using "profanity and copyright" filtering algorithms.

    While the platforms are not responsible for morons and stupid behaviour, they have the opportunity to limit the negative outcomes of that, through thinking beyond the next financial incentive.

    link to this | view in thread ]

  2. This comment has been flagged by the community. Click here to show it
    identicon
    Libby Ruhl DeLuzionnes, 23 Oct 2020 @ 4:38pm

    The KEY criminal act is tried to stab a police officer.

    According to the release, Blake admitted to investigators that he did have a knife at the time and a knife was recovered from the driver's side floorboard of his vehicle. No other weapons were found, the report said.

    But that's simply elided here at dishonest TD and in the Rittenhouse piece:

    It was past 11 p.m. local time Tuesday, the third night of protests after a Kenosha police officer shot Jacob Blake seven times in the back.

    link to this | view in thread ]

  3. This comment has been flagged by the community. Click here to show it
    identicon
    Libby Ruhl DeLuzionnes, 23 Oct 2020 @ 4:39pm

    Another KEY part you omit: "third night of protests"

    Among protesters, the rumor spread: Hundreds of white men with guns had answered an online call from a self-described militia group known as the Kenosha Guard and would be waiting in the park to shoot them.

    Not nearly that many armed men showed up, but they were impossible to avoid. Some joined the marchers and pledged to protect them. Many protesters still felt more afraid than secure.

    The police weren't keeping order. Vigilantes decided to, as is the responsibility of sane people.

    That's why to stop riots immediately. Turned into armed confrontation.

    And of course YOU omit that "the marchers" also had guns on their side.

    link to this | view in thread ]

  4. This comment has been flagged by the community. Click here to show it
    identicon
    Libby Ruhl DeLuzionnes, 23 Oct 2020 @ 4:39pm

    "plans to argue self-defense" but TD clearly has convicted.

    It's absolutely clear that "the left" won't back off. But you're actually just a few weenies. I predict you'll be surprised, rather like Yamato said at start of WW2 "we have awakened a sleeping giant".

    link to this | view in thread ]

  5. This comment has been flagged by the community. Click here to show it
    identicon
    Libby Ruhl DeLuzionnes, 23 Oct 2020 @ 4:45pm

    And the sum is that slanted bigotry isn't "moderation".

    You're totally biased and try to stop all other viewpoints, especially for any organizing, formula for conflict.

    Now, flatly, not to stint you because you'll go berserk anyway: I don't find any wrongdoing by police or Rittenhouse, but DO by "the marchers". Leftists just pounce on any pretext without regard to facts.

    link to this | view in thread ]

  6. icon
    Stephen T. Stone (profile), 23 Oct 2020 @ 4:50pm

    Having a knife in your possession doesn’t automatically mean you tried to stab someone with it.

    link to this | view in thread ]

  7. icon
    Stephen T. Stone (profile), 23 Oct 2020 @ 4:51pm

    The police weren't keeping order. Vigilantes decided to, as is the responsibility of sane people.

    I’m glad to see you show your true colors by supporting vigilante violence.

    link to this | view in thread ]

  8. icon
    Stephen T. Stone (profile), 23 Oct 2020 @ 4:54pm

    It's absolutely clear that "the left" won't back off.

    Who was it that was charged this morning with shooting up and helping burn down a police precinct in Minneapolis? Was it…

    1. Antifa

    2. Black Lives Matter

    3. A right-wing nutjob associated with the Boogaloo movement and had contact with a cop-killer before the precinct burned down but after said cop-killer had already killed a cop?

    …it—it’s #3. The answer is #3.

    link to this | view in thread ]

  9. icon
    Stephen T. Stone (profile), 23 Oct 2020 @ 4:56pm

    You're totally biased and try to stop all other viewpoints

    So what? It’s his legal right. Whether you think it’s morally or ethically wrong is irrelevant — doubly so, considering the kinds of voices you do support.

    You, uh…you friendly with the Proud Boys, Brainy?

    link to this | view in thread ]

  10. identicon
    Anonymous Coward, 23 Oct 2020 @ 5:08pm

    Re: And the sum is that slanted bigotry isn't "moderation".

    You're totally biased and try to stop all other viewpoints

    Says the man who is so biased that he will not even consider a different viewpoint.

    link to this | view in thread ]

  11. icon
    That One Guy (profile), 23 Oct 2020 @ 5:55pm

    The answer is always #3

    Adding to that which side is it that's driven vehicles into crowds multiple times now, killing at least one person in the process?

    I do so laugh when someone tries to portray 'the left' as this pile of violent nutjobs that the police need to protect the helpless public from, as while some pissed off rioters might have torched some property when it comes to violence against people their side seems to have no problem not just crossing that line but vaulting over it with no shortage of outright sadistic glee.

    link to this | view in thread ]

  12. icon
    That One Guy (profile), 23 Oct 2020 @ 6:20pm

    Not the best look...

    Reported literally hundreds of times, reviewed for moderation four times, and a page that called for armed responses to 'thugs' ended up being taken down by the people that put it up in the first place, who apparently realized quicker than Facebook what a bad look keeping it up made for.

    Even if it didn't result in death what did they think that sort of 'event' was likely to result in, and how in the hell did not one but four moderators conclude that a page like that didn't violate rules against 'Dangerous individuals and organizations', because honestly if a page calling for armed responses to a group of people doesn't meet the bar required I struggle to think of anything that could.

    link to this | view in thread ]

  13. icon
    That One Guy (profile), 23 Oct 2020 @ 6:25pm

    Re:

    You missed the best part in that their argument could easily be turned around and used in favor of those they would vilify, as people are pissed off and protesting and rioting because the police have shown that they aren't interested in order or laws but merely protecting their own no matter what, such that people have to step in and try to fix a broken system, 'as is the responsibility of sane people'.

    link to this | view in thread ]

  14. icon
    PaulT (profile), 24 Oct 2020 @ 2:36am

    Re: Re: And the sum is that slanted bigotry isn't "moderation".

    The subject at hand is a kid who, inspired and emboldened by the same right-wing echo chamber he lives in, went and killed two people - one of whom was in the process of trying to legally apprehend him after he had been witnessed murdering the first. The kid was seen palling around with cops just before going on his murder spree, who let him get away comfortably with his gun to his home in another state, in stark contrast to the violent deaths handed out to black men who they suspect might have a weapon.

    Dickhead has to take a viewpoint like the one he's taking, because he has no leg to stand on morally or factually if he takes another one.

    link to this | view in thread ]

  15. identicon
    Anonymous Coward, 24 Oct 2020 @ 7:34am

    Re: "plans to argue self-defense" but TD clearly has convicted.

    "It's absolutely clear that "the left" won't back off."

    Why should someone backoff when being murdered? They should just backoff and accept their fate, amirite?

    link to this | view in thread ]

  16. This comment has been flagged by the community. Click here to show it
    identicon
    Anna, 24 Oct 2020 @ 8:00am

    hi

    link to this | view in thread ]

  17. icon
    Toom1275 (profile), 24 Oct 2020 @ 11:37am

    "Rittenhouse acted in self-defense!" Said nobody who has any understanding of self-defense.

    link to this | view in thread ]

  18. icon
    Toom1275 (profile), 24 Oct 2020 @ 11:38am

    Re:

    These same semi-ambulatory colostomy bags tried the exact same "self-defense" lie after Charlottesville.

    link to this | view in thread ]

  19. icon
    Stephen T. Stone (profile), 24 Oct 2020 @ 12:35pm

    And that worked about as well as one might think in the trial of Some Asshole for the murder of Heather Heyer.

    link to this | view in thread ]

  20. icon
    That One Guy (profile), 24 Oct 2020 @ 12:45pm

    Re:

    That or they're using the same twisted definition of 'self-defense' that police like to use, where anything might be a threat to their lives and therefore they are always justified in applying lethal force 'just in case'.

    link to this | view in thread ]

  21. icon
    That One Guy (profile), 24 Oct 2020 @ 12:49pm

    Re:

    Oh how I wish the Some Asshole initiative was the rule, I can only imagine how many scumbags wouldn't bother knowing that what they did would be all that was mentioned with the main focus being on how the people/community was healing as all the while they would just be known as 'Some Asshole' and treated as just a minor footnote.

    link to this | view in thread ]

  22. icon
    PaulT (profile), 24 Oct 2020 @ 12:53pm

    Re:

    It could be argued that he acted in self defence in the second murder because he killed someone who was trying to physically attack and apprehend him at the time.

    Of course, the fact that the reason why he was being apprehended was because he'd just been witnessed murdering another person might undermine that defence somewhat.

    link to this | view in thread ]

  23. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Rex, 26 Oct 2020 @ 1:35pm

    Re:

    Only on TechDirt would someone shooting a convicted child molester and a convicted wife beater be seen as a bad thing.

    link to this | view in thread ]

  24. identicon
    Anonymous Rex, 26 Oct 2020 @ 1:38pm

    Re: Re: Re: And the sum is that slanted bigotry isn't "moderatio

    He shot and killed a convicted child molester and a convicted wife beater.

    Nope. Doesn’t bother me.

    link to this | view in thread ]

  25. identicon
    Anonymous Coward, 26 Oct 2020 @ 3:08pm

    Re: Re:

    Would you be ok with shooting a convicted bank robber? Or a convicted burglar? Or a convicted drug dealer? Or a convicted drunk driver? Would they deserve to be murdered in cold blood for their crimes that they have already served their sentence for?

    They have been convicted of their crime and have paid the price for it. It is not the job of everyday citizens to enforce the law. That's what the police are (supposed to be) for.

    Cops get away with murdering people all the time. But ordinary citizens? Murdering someone (no, it was not "self-defense") is a crime. Point blank.

    You can't get away with murdering someone just because you don't like them for whatever reason. Even if you think your reason is valid.

    link to this | view in thread ]

  26. identicon
    Anonymous Coward, 26 Oct 2020 @ 5:13pm

    Re: Re:

    Saying to hell with due process is a good thing? You Judge Dredd? Extrajudicial killings are not supposed be allowed, unless maybe it is one of those secret laws we do not know about but are expected to not violate.

    link to this | view in thread ]

  27. icon
    That One Guy (profile), 26 Oct 2020 @ 5:40pm

    Most people wouldn't so blatantly show contempt for the rule of law and support for vigilante murder, but you do you I suppose.

    link to this | view in thread ]

  28. icon
    Toom1275 (profile), 26 Oct 2020 @ 9:35pm

    Re: Re:

    Yeah no, victim-blaming to excuse cold-blooded murder isn't a good look there, slick.

    link to this | view in thread ]

  29. icon
    Toom1275 (profile), 26 Oct 2020 @ 9:36pm

    Re: Re: Re:

    It makes you sound like a police union leader.

    link to this | view in thread ]

  30. icon
    PaulT (profile), 27 Oct 2020 @ 12:05am

    Re: Re: Re: Re: And the sum is that slanted bigotry isn't "moder

    Death Wish is not a documentary about a desirable society.

    link to this | view in thread ]

  31. icon
    PaulT (profile), 27 Oct 2020 @ 12:07am

    Re: Re:

    Only on Techdirt would an "edgy" contrarian claim that people supporting due process and the rule of law is a bad thing, and support extrajudicial murder instead.

    link to this | view in thread ]

  32. icon
    Scary Devil Monastery (profile), 27 Oct 2020 @ 4:07am

    Re: The KEY criminal act is tried to stab a police officer.

    "According to the release, Blake admitted to investigators that he did have a knife at the time and a knife was recovered from the driver's side floorboard of his vehicle."

    This is still the United States we're talking about, where a white man is not to be questioned about carrying a loaded AR-15, a shotgun, and have a crate of handguns or rifles in the trunk, but apparently, a black man possessing a knife in his vehicle is justification for said black man being killed?

    Thank you very much, Baghdad Bob, for giving us yet more evidence you're a hypocritical racist shitbag. As if we needed that given your previous assertions about how law shouldn't require evidence or a courtroom.

    "But that's simply elided here at dishonest TD and in the Rittenhouse piece..."

    "dishonest" for bringing to attention that if Blake had been white he could have had his cars stuffed with shotguns or worn them openly and the officers wouldn't have so much as unholstered their weapons. In fact a white person already known to have committed murder could walk through a police line still carrying his AR without even being questioned.

    link to this | view in thread ]

  33. icon
    Scary Devil Monastery (profile), 27 Oct 2020 @ 4:16am

    Re:

    "You, uh…you friendly with the Proud Boys, Brainy?"

    Pretty obvious, innit? Old Baghdad Bob's pulled out all the stops in this one; Blake, being unarmed, gets shot 7 times in the back and old Bobmail justifies it by saying he had a knife in his car.

    Then in the very next sentence Bobmail turns right around and glorifies the right-wing extremist violence as vigilantism.

    With the only notable differences between the violence Baghdad Bob condones and condemns being the skin color of the victim it's pretty damn clear where he's standing.

    Not, bluntly put, that this should be any surprise. We've known since his Torrentfreak days he's a paranoid schizophrenic with an unhealthy attraction to fascism, racism, bigotry and outright sadism.

    Or to summarize, a loser in life who compensates for nothing going his way by inventing a world inside his head where he's the eternal victim of other people and only showing up from that world to write the most recent case of thuggery in on "his" side of that narrative.

    He'd be tragic and pitiable if he wasn't such an obvious and persistent asshole.

    link to this | view in thread ]


Follow Techdirt
Essential Reading
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

Close

Email This

This feature is only available to registered users. Register or sign in to use it.