Moderation Of Racist Content Leads To Removal Of Non-Racist Pages & Posts (2020)
from the moderation-mistakes dept
Summary: Social media platforms are constantly seeking to remove racist, bigoted, or hateful content. Unfortunately, these efforts can cause unintended collateral damage to users who share surface similarities to hate groups, even though many of these users take a firmly anti-racist stance.
A recent attempt by Facebook to remove hundreds of pages associated with bigoted groups resulted in the unintended deactivation of accounts belonging to historically anti-racist groups and public figures.
The unintentional removal of non-racist pages occurred shortly after Facebook engaged in a large-scale deletion of accounts linked to white supremacists, as reported by OneZero:
Hundreds of anti-racist skinheads are reporting that Facebook has purged their accounts for allegedly violating its community standards. This week, members of ska, reggae, and SHARP (Skinheads Against Racial Prejudice) communities that oppose white supremacy are accusing the platform of wrongfully targeting them. Many believe that Facebook has mistakenly conflated their subculture with neo-Nazi groups because of the term “skinhead.”
The suspensions occurred days after Facebook removed 200 accounts connected to white supremacist groups and as Mark Zuckerberg continues to be scrutinized for his selective moderation of hate speech.
Dozens of Facebook users from around the world reported having their accounts locked or their pages disabled due to their association with the "skinhead" subculture. This subculture dates back to the 1960s and predates the racist/fascist tendencies now commonly associated with that term.
Facebook’s policies have long forbidden the posting of racist or hateful content. Its ban on "hate speech" encompasses the white supremacist groups it targeted during its purge of these accounts. The removals of accounts not linked to racism -- but linked to the term "skinhead' -- were accidental, presumably triggered by a term now commonly associated with hate groups.
Questions to consider:
- How should a site handle the removal of racist groups and content?
- Should a site use terms commonly associated with hate groups to search for content/accounts to remove?
- If certain terms are used to target accounts, should moderators be made aware of alternate uses that may not relate to hateful activity?
- Should moderators be asked to consider the context surrounding targeted terms when seeking to remove pages or content?
- Should Facebook provide users whose accounts are disabled with more information as to why this has happened? (Multiple users reported receiving nothing more than a blanket statement about pages/accounts "not following Community Standards.")
- If context or more information is provided, should Facebook allow users to remove the content (or challenge the moderation decision) prior to disabling their accounts or pages?
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: bias, case studies, content moderation, mistakes, racist speech, skinheads
Companies: facebook
Reader Comments
Subscribe: RSS
View by: Time | Thread
Yes, they should look at context. Just like DMCA moderation should look at context and... whether or not a notice is even accurate or valid.
They should also supply context to users when accounts or posts are moderated.
They should also have a reasonable appeals system, not one that depends upon such egregious mis-moderation that it makes the news somewhere that they notice it.
[ link to this | view in chronology ]
It should, but it legally can’t. Anyone who receives a DMCA takedown notice must take down or disable access to the content in question or risk losing their “safe harbor” protections. Companies often automate DMCA takedown systems for that exact reason. And yes, that means those systems will honor bogus takedowns. It is what it is.
[ link to this | view in chronology ]
Re:
And just how many people, speaking how many languages will that require?
An appeals system would have the problem that every moderation decision would be appealed, and it would be abused to become a complaints system.
[ link to this | view in chronology ]
Re: Re:
Then they shouldn't provide a fake appeal system, should they?
And just how many people, speaking how many languages will that require?
I call that a bloody business opportunity.
[ link to this | view in chronology ]
Re: Re: Re:
"I call that a bloody business opportunity."
The real world calls it "impossible at this kind of scale".
[ link to this | view in chronology ]
Re:
"Yes, they should look at context."
They should. Now, do you have an automated method and/or other scalable solution that can do that accurately for the amount of posts they get every minute? That's the problem.
"They should also have a reasonable appeals system"
Define "reasonable" in a way that's actionable.
[ link to this | view in chronology ]
When you start with the stupid idea that some people shouldn't be allowed to say anything that you don't agree with because... well, you just don't agree with them, then the stupid just goes on and on...
[ link to this | view in chronology ]
Re:
People are allowed to say whatever they want, but don't expect them to be able to do it at someone else's expense. It's as simple as that.
[ link to this | view in chronology ]
Re:
The end result is fascists shouting anti-fascist slogans as they prevent you from presenting your side.
[ link to this | view in chronology ]
You have the right to speak your mind; no one is obliged to host your speech or listen to it.
[ link to this | view in chronology ]
Re:
Who is it you believe has that idea?
[ link to this | view in chronology ]
Re: Re:
Trump, Hawley, and Koby certainly do.
[ link to this | view in chronology ]
Re: Re: Re:
Con men and objectively demonstrated idiots are not who we should be basing these thing on, however...
[ link to this | view in chronology ]
Re: Re: Re: Re:
Unfortunately - and I still can't believe I'm saying this in earnest -Trump is actually the president of the united states and is arguably one of those who these things will be based on.
[ link to this | view in chronology ]
It would be so much simpler for individual contributors to just ... be polite!
Learn what kind of manners a particular online community expects. Learn by being moderated, if you're incapable of taking gentler hits. Be patient: nobody wants to know how much you want to blame someone else for your own stupidity or obstinacy. If you're having trouble communicating online, try communicating face to face. Then come back online when you've gotten that figured out.
There! problem solved.
[ link to this | view in chronology ]
You can’t fix human nature by asking people to “be nice”.
[ link to this | view in chronology ]
Re:
FTFY...
"You can’t fix human nature."
[ link to this | view in chronology ]
Re:
If these idiots were capable of such things, the problem would not exist in the first place...
[ link to this | view in chronology ]
Re:
Among the people who got taken down in that purge of skinheads was a black man (so they didn't even look at his profile) who was one of the founders of the Two Tone movement.
[ link to this | view in chronology ]
That's the problem, as it's impossible to moderate on a granular level at a massive scale, purges like this will create collateral damage. At least they seem to have reacted correctly and quickly when notified of the error.
"so they didn't even look at his profile"
Because no racist has ever used a fake profile to avoid consequences. /s
The moderation could have been better, but you're hopelessly naive if you think that all social media accounts contain accurate personal information and real photos of the person they claim to be.
[ link to this | view in chronology ]
What makes a bad term bad
This touches on a related phenomenon I've been wondering about recently. Words or expressions being labelled as bad, unwelcome and usually racist terms, even in contexts where I find them perfectly appropriate. In this case it's the word "skinhead" being used in a very narrow fashion, but others I've wondered about include but are not limited to the word color, the ok sign, and frogs. Skinhead picking up a negative meaning from guilt by association I can sort of understand, but the others are mysteries to me.
How do words and concepts go from everyday use to being practically banned? I've seen the term dogwhistling used to explain some cases, but now the word dogwhistling's on the bad word list too so this only confuses me more. I have no idea how this stuff works, but watching it happen in real time is very interesting.
[ link to this | view in chronology ]
Re: What makes a bad term bad
"How do words and concepts go from everyday use to being practically banned?"
Language is fluid. Talk to someone 100 years ago about them being gay compared to someone now. You'll be talking about 2 totally different things even though you're using the same word.
"I've seen the term dogwhistling used to explain some cases, but now the word dogwhistling's on the bad word list too so this only confuses me more"
If that term is on a list, whoever made the list is confused.
To explain, we've made enough progress in society now that being outwardly bigoted or racist is not acceptable. People who would use the n word constantly 50 years ago now has to keep that in check, as even the most ape-like knuckledragger understands that they can't use it around non-racists. So, they have to hide their bigotry by using words that don't necessarily mean anything racist on their own but give a knowing wink to other bigots when used in a certain context.
"others I've wondered about include but are not limited to the word color, the ok sign, and frogs"
Colour isn't a negative word, really, but I suppose it depends on context. The other 2 are simple examples of dogwhistling and co-opting. The OK sign thing started out as a prank, but white supremacists have decided to start using it to provide plausible deniability. If they're signalling to other Nazis and get caught flashing the OK sign, they can pretend they were using it in ways other than the one they're really using.
As for "frogs", it's not all frogs but a specific meme called Pepe The Frog, which was co-opted by white supremacists in their online communications. It sucks for the original creator who despises that stuff, but the association is now as clear as when the swastika was taken from its spiritualist roots and used as a symbol of the Third Reich.
[ link to this | view in chronology ]
Re: Re: What makes a bad term bad
What do the not-actually-an-OK-sign OK-sign and the cartoon frog actually mean? Are they slurs that rightfully deserve to be shunned? Are they just things a certain subset of people use? Should things be banned because someone you don't like used them? Taking things to their ridiculous extreme, should oxygen be banned because bad people sorry no i can't write this with a straight face never mind.
There's clearly some lines to this which I'm still in the process of discovering. It's all very fascinating.
Now, for the amphibians. I'm aware of Pepe's more recent symbology and the ties to the Kekistan thing, so I can understand that being on the bad things list. But for some reason I thought it had spread to actual frogs too, but I can't figure out why I thought so and there's nothing on Google either. I must've dreamed that part up, sorry about that.
[ link to this | view in chronology ]
Re: Re: Re: What makes a bad term bad
They're signals for white supremacy.
https://www.npr.org/2019/09/26/764728163/the-ok-hand-gesture-is-now-listed-as-a-symbol-of -hate
[ link to this | view in chronology ]
Re: Re: Re: What makes a bad term bad
"Should things be banned because someone you don't like used them?"
Did I say they should be banned? No, I didn't.
I'm simply pointing out that otherwise innocent things are being co-opted by white supremacists, and this historically has had the effect of making even innocent uses of those things questionable. Meaning of things can change over time, and words and symbols that were once wholesome are now offensive.
Put it this way - assuming you're not in Germany, it's likely that flying a swastika flag is not illegal. It will, however get you labelled a Nazi, even if you're flying a flag with the original sanskrit version that existed before the Nazis co-opted it. It's not illegal for you to fly such a thing, but you will get negative reactions no matter what your original intention was.
" But for some reason I thought it had spread to actual frogs too"
Alex Jones once went on a hilarious rant about chemicals turning frogs gay that went viral, and the sort of dull-witted person who actually takes Infowars seriously might have used that seriously in some way, though I'm not aware of any specific examples.
[ link to this | view in chronology ]
"...others I've wondered about include but are not limited to the word color, the ok sign, and frogs."
Apparently frogs are a sensitive issue, being religiously considered by a certain type of alt-right nut to be the hapless subjects of what appears to be chemically induced reversed gay "therapy". Just ask Alex Jones whose genius brainchild might be the origin of the reason frogs have been included among controversial keywords.
"How do words and concepts go from everyday use to being practically banned?"
Two reasons, mainly. One is where a conspiracy theory involving <innocent concept X> actually being the key revelation for <global conspiracy Y> goes viral and since every nine out of ten new online mentions of, for example, "frogs" may be the start of a flame war about "gay frogs" and Alex Jones, canny moderators eager to nip the trolling in the bud include that specific word among the verba non grata for a while.
The second reason is where the innocent word in question has, a few times too often in recent times, been adopted by a certain type of bigot in lieu of using, for instance, the N-word or other ethnic, gender-based or LGBT-phobic slur.
And it's all because the angry and upset bigot really wants to be able to publicly espouse his opinion about the lesser beings with which he has to share the planet, and thus he leapfrogs from one word to the next, never quite realizing that it is the way he uses those words which eventually give those words a bad online reputation.
[ link to this | view in chronology ]
Re:
"And it's all because the angry and upset bigot really wants to be able to publicly espouse his opinion about the lesser beings with which he has to share the planet, and thus he leapfrogs from one word to the next, never quite realizing that it is the way he uses those words which eventually give those words a bad online reputation."
And then we go and overreact banning those words, no matter the context or even if they were used as meme before.
This isn't too different as to how terrorists harm our society the moment it overreacts to them taking away our liberties and fundamental rights.
And thus, the racists win. Sure, it's some stupid words and memes, but they got their win the moment they took something away from us.
[ link to this | view in chronology ]
A society can’t “ban” a word — American society in particular (thanks, First Amendment). It can “regulate” the use of that word through social shaming and altering social norms around that word, though. And anyone can refuse to associate with people who use that word, including the owners and operators of online services such as Twitter.
I don’t know if you’ve noticed, but nobody has made illegal the carrying of a Confederate flag. Sure, lots of people refuse to associate with someone who flies/wears/defends the Stars ’n Bars. But that doesn’t make those people “terrorists”. It makes them people who don’t like someone who flies/wears/defends a flag associated with a failed state that betrayed and fought a war with the United States over the institution of slavery.
Which is why we see the “reclaiming” of words and images. “Queer”, for example, was (and kinda still is) a slur against LGBT people. But plenty of LGBT people reclaimed the word as a self-descriptor and a broad descriptor of people who aren’t straight, cisgender, heteroromantic, or any combination thereof (e.g., asexual people, non-binary people). Yes, not every LGBT person agrees with that reclaiming. But enough of them have reclaimed the word that it isn’t odd to see people use “queer” as a shorthand for their queerness when a situation dictates such usage.
Racists can “take” words and images from us. Some we can take back, some we can’t. (You can’t reclaim the swastika, for example.) But it’s not “terrorism” — it’s tribalistic idiocy.
[ link to this | view in chronology ]
Re:
I think what he's talking about is how the 9/11 attacks, while awful, really did not do all that much damage directly when measured against the scale of the entire US. However our overreaction to them has been immensely harmful to ourselves.
[ link to this | view in chronology ]
Re: Re:
Thanks nasch, it still baffles me how he managed to twist my whole post into saying that I'm calling those who fight against racism as "terrorists", when I was associating the supremacists with its effects, not those who fight them.
Stephen, not everyone who disagrees with the finer points is one of Trump's lackeys, you know.
For me, the end doesn't justify the means even if the cause is noble as fuck, and I'm not going to shoot on my own foot to make "nigger" disappear from the dictionary.
Racism, among other things, is one of the stupidest things ever, but I'm not going to fight stupid with stupid.
I'm not going to support shit that I rejected in the "war against terrorism".
[ link to this | view in chronology ]
Re: Re: Re:
"Stephen, not everyone who disagrees with the finer points is one of Trump's lackeys, you know."
But, people who parrot the most inane spins on how people should be allowed to be openly racist without consequences usually are.
[ link to this | view in chronology ]
Re: Re:
"And thus, the racists win. Sure, it's some stupid words and memes, but they got their win the moment they took something away from us."
True enough. Here, you remember the swastika? That old germanic solar wheel connotating fortune, light and prosperity? (as it is still used today in asia where the symbol hasn't become synonymous with genocide).
The word negro is basically portuguese/spanish for dark. It is an offensive word only when it is used in English - because that's when it refers to skin color. It gets truly offensive when it is pronounced in southern fashion, with an i and two g's.
George W Bush Jr. did his level best to remove the word french from any itemry considered positive -anyone recall "freedom fries"?
And apparently the declaration of independence is now seen as Anti-Trump by Trump adherents.
There is, unfortunately, no part of language - from single words to whole declarations - whose meanings does not change depending on individual perception and pre-understanding.
[ link to this | view in chronology ]
Re: Re: Re:
"The word negro is basically portuguese/spanish for dark."
Actually is means the colour black. Oscuro would be the better translation for dark in Spanish. But, yes, the imported word has certainly been co-opted in English by racists.
"George W Bush Jr. did his level best to remove the word french from any itemry considered positive -anyone recall "freedom fries"?"
To be fair... that was more likely redneck morons reacting to the French daring to (correctly it turns out) object to unnecessary war against a country that had not attacked the US. I doubt this was W.'s personal doing, especially since the idiots were calling for things like a ban on French's mustard (which is, of course, a purely American company with no relation to France). Plus, of course "French fries" are likely Belgian in their original version...
Your central point stands, but it's worth making sure the criticisms are factually as well as logically accurate :)
[ link to this | view in chronology ]
Re: Re: Re: Re:
It was actually Bob Ney.
https://en.wikipedia.org/wiki/Freedom_fries
[ link to this | view in chronology ]
Re: Re: Re: Re:
"Your central point stands, but it's worth making sure the criticisms are factually as well as logically accurate :)"
Mea Culpa.
In my defense so much stupid and moron was spilled by GWB, Rumsfeld, Ashcroft and Cheney it's, by now, hard to remember the specific stupid they weren't directly responsible for.
You are right, however, I shouldn't blame poor dubya for the idiocy perpetrated by the people now wearing MAGA hats...
[ link to this | view in chronology ]
Mods
Moderation is un-American. "Racist" content is as ridiculous and false as "hate" speech.
Neither exists in real life. Moderation fails every time. Who moderates the moderators?
There is no answer for that. There never will be.
[ link to this | view in chronology ]
Re: Mods
"Moderation fails every time"
Except in the thousands of examples I can think of where it actually does. For example - do you honestly think that your workplace isn't moderated, assuming you have one, and that this usually works out better for everyone than if it weren't moderated?
[ link to this | view in chronology ]
Re: Re: Mods
You noticed you're replying to someone who denies the existence of racism right?
[ link to this | view in chronology ]
Re: Re: Re: Mods
That is dumb, I just wanted to poke at the part of the comment that a sane person might be fooled into agreeing with. He's "moderated" every day of his life even if he decides he wants to take racism out of the equation.
[ link to this | view in chronology ]
Re: Re: Re: Re: Mods
It's not surprising. restless is one of the dumbasses who thinks vaccination is poisoning your children and would rather everyone got shingles and smallpox all over again. The fact that Facebook et al are moderating away his anti-vaccine celebrity posts is something that makes his anti-vaccine erection sad.
I wonder if he realizes that the anti-vaccine/Republican/conservative response to Twitter, Parler, is in itself a very heavily moderated platform for a solution that claims to be "American".
[ link to this | view in chronology ]
Re: Mods
Moderation is free speech
Free speech is American
restless94220 is a compulsively lying shitwit
[ link to this | view in chronology ]
That’s a funny way of saying “I don’t like that Twitter bans racists”.
[ link to this | view in chronology ]
Re: Mods
"Moderation is un-American."
Someone ought to tell Americans that. Neither party has ever believed in free speech. It has to be said that only one of the parties consider free speech to be an abomination however.
""Racist" content is as ridiculous and false as "hate" speech."
Duly noted that you are upset you can't advocate lynching "niggers" anymore on FaceBook.
"Who moderates the moderators? "
On private property? Only the property owner who is of course free to set the rules. I realize this must suck for you in particular given your historical adherence to alt-right stormfront/Alex jones echo chamber narrative.
In the public space? No one. Feel free to enjoy as much of the public space as the rest of society. I realize it must suck if your views are so hideously repugnant to so many people you will end up getting heckled by all the people making use of their free speech to inform you what an asshat you are.
Once again for the benighted lackwits among the bigots and racists who still don't understand what even small children easily realize - if you are free to speak in public then so is everyone else. In private the host and owner of the premises sets the rules.
[ link to this | view in chronology ]