Twitter Admits It Messed Up In Suspending Accounts Under Its New Policy, But Policies Like This Will ALWAYS Lead To Overblocking
from the who-didn't-see-that-coming dept
Last week, we called out a questionable move by Twitter to put in place a new policy that banned images or videos that included someone without their permission. The company claimed that this was to prevent harassment and also (for reasons I still don't understand) buried the fact that this was not supposed to apply to public figures or newsworthy media. However, as we pointed out at the time, the standard here was incredibly subjective, and wide open to abuse. Indeed, we pointed out examples in that article of the policy clearly being abused. And even as Fox News talking heads insisted the policy would only be used against conservatives, in actuality, a bunch of alt right/white nationalists/white supremacists immediately saw this as the perfect way to get back at activists who had been calling out their behavior, leading to a mass brigading effort to "report" those activists for taking pictures and videos at white nationalist rallies and events.
In other words, exactly what we and tons of others expected.
And, a few days later, Twitter admitted it messed up the implementation of the policy -- though it doesn't appear to be rolling back the policy itself.
Twitter said Friday that it had mistakenly suspended accounts under a new policy following a flood of “coordinated and malicious reports” targeting anti-extremism researchers and journalists first reported Thursday by The Washington Post.
The company said it had corrected the errors and launched an internal review to ensure that its new rule — which allows someone whose photo or video was tweeted without their consent to request its removal — was “used as intended.”
[....]
In a statement Friday, Twitter spokesman Trenton Kennedy said that the company had been overwhelmed with a “significant amount” of malicious reports and that its “enforcement teams made several errors” in the aftermath.
What I am perplexed about, however, is that I know for a fact that Twitter has a bunch of smart and thoughtful people who work on trust and safety issues and who clearly would know how this policy would play out in practice, and yet for whatever reason the policy was still rolled out the way it was. I don't know if this is because people underestimated how quickly and wildly it would be abused, or if others at the company simply overruled concerns raised by experts, or if it was something else entirely. Either way... it's a weird and surprising misstep for Twitter.
That said, it's also illustrative of a really important point that we've been trying to raise for ages, going back to the DMCA takedown process, and covering all sorts of other policy debates regarding content moderation: if you give people tools to take down some content, they will be abused. Always. That's not to say you should never moderate or never take down content, because that's impossible. There is always going to be some content that sites need to take down, whether for legal purposes or because it's harming the integrity of the site (things like spam or harassment).
But any such policy always opens itself up to abuse and dishonest reporting. And any company that has such policies (i.e., every company with third party content) needs to have a plan in place not just to deal with the abusive/problematic content on the site, but also the abuse of the moderation process to silence voices that shouldn't actually be silenced.
Again, from my interactions with Twitter trust & safety people, I know they know this. And this is part of why I find the rollout of this policy so perplexing.
However, it's also an important lesson for policymakers in various state legislatures, in DC, and around the globe. There is so much effort these days to pressure (or require!) internet companies to remove "bad" content, but almost none of those policy plans take into account the ways in which those policies will be abused to silence reporting, silence marginalized voices, and silence those calling out abuses of power. Twitter's rollout of this policy has been a disaster (and one that could have been prevented) but at the very least it should be a warning to policymakers who seem to think that they can design policy requirements to moderate certain content, without bothering to explore the likelihood that those mandates will be abused to silence important speech.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: content moderation, overblocking, photos, private information, private media, videos
Companies: twitter
Reader Comments
Subscribe: RSS
View by: Time | Thread
Gee I guess that policy about not taking complaints from random people not involved in the tweet in anyway/shape/form went out the window again, which is sort of sad because ending the whole get all of your followers to mass report someone was a shitty tactic & when it ended it improved things.
And some sort of logical application that things that are like over a yr old shouldn't be up for reporting under a rule under a week old.
“enforcement teams made several errors”
So SSDD?
The enforcement teams make LOTS of mistakes & despite PR we've reviewed it and addressed it somehow they keep making the same fscking mistakes over and over depending on the players involved.
Does someone want to sent a card to Trust & Safety to remind them that their heads are up their arses?
No one would EVER abuse one of our systems... system gets abused... lather rinse repeat with every new system.
How many times can Lucy pull the football away before you decide maybe you shouldn't play Lucy's game anymore?
"at the very least it should be a warning to policymakers"
You'd think that but they live in a world of absolutes where is they say it can happen of course it can happen even if everyone else knows if the sun suddenly rose in the west & set in the east we're fscked.
Sometimes it pays to have someone paid specifically to not be a yes man & raise concerns about the plans. Making them actually address those issues rather than hand waving that the system will take care of it. They assume their system allowed for judgement calls but when all of the calls are strikes against people no matter what perhaps the system is flawed.
But then I still hate Twitter and hope it burns when they pee.
[ link to this | view in chronology ]
hard to do automatic flagging when behavior changes
This is a broad general problem for automatic/algorithmic flagging, which the FB guys are farther down the road of understanding than the twitter folks because they got burned on it earlier. An automatic flagging system predicts whether there is a policy violation based on observables (including, e.g., photo content, automatic systems can be super powerful.) As long as that relationship is stable, the system will work. When a hostile actor can change that relationship, either by trying to change it or just by changing behavior, it blows up.
Why don't engineers quickly tumble to this? It is a human behavior problem, not a technical one.
[ link to this | view in chronology ]
Ultimate Test
Looks Fox News was wrong, and instead of being reactionary, these groups performed the first strike. But the point still stands: Twitter may undo the effects of this operation, and then only allow left wing groups to use the rule in the future. If so, it answers your question from before:
"The company claimed that this was to prevent harassment and also (for reasons I still don't understand)"
The reason would be to manipulate the outcome of events in a biased fashion. This was the first step by the conservative side to test for that bias. Harassment isn't an objective policy; anything that you don't like is harassment.
[ link to this | view in chronology ]
Re: Ultimate "Conservative"
[ link to this | view in chronology ]
Re: Re: Ultimate "Conservative"
(The dreaded enter entered)
I see the victim-complex is in full force. Stop sniffing the alt-right glue, it'll do wonders for your mental faculties.
Anyway, I find it funny how "conservatives" complain about how they are silenced and the moment a way to bad faith silence people comes along their first thought is to use it en masse.
No, this was not a test for bias, it was pure and unadultered assholery and it aptly displayed the sheer dishonesty of "conservatives" that has been complaining loudly about "censorship". It showed that they have zero compunctions about sinking to new lows they accuse everyone else of doing without evidence.
You yourself have been loudly complaining about "censorship" of "conservative" views (for which you have never given any specific examples of, strange that but you have your chance now in a rebuttal to this post), and here you are actually defending real "censorship" per your own definition. Congratulations, you have just proven for all that you are a dishonest asshole and your words are that of a liar, totally worthless.
If I someone says they don't like you and what you are saying and that you aren't welcome on their property, if you think that's harassment then you are just another entitled snowflake having a meltdown. Harassment is using a tool to silence people you don't like, just like your friends did and who you are now defending. You can tell a lot about people by what friends they keep.
Kindly, go put you head where the sun doesn't shine.. Oh, my mistake, you had already done that...
[ link to this | view in chronology ]
Re: Re: Re: Ultimate Test
Conservatives are convinced that bad faith censorship will be used against them, in the future as it is today. After this test, we expect reversals and account restorations to work in our favor in the future, or else it is more proof of bias.
They simply played by the rules, and didn't want their images used, and filed the proper notice. Now the pictures are probably going to go back up. Twitter effectively can't abide by it's own rules that it created.
I think it speaks volumes about you that you believe social media is property of leftist speech. This is why most Americans are convinced of anti-conservative bias with the big tech corporations.
[ link to this | view in chronology ]
Re: Re: Re: Re: Ultimate Test
I'm gonna pay attention to your post.
But first answer the question.
[ link to this | view in chronology ]
Re: Re: Re: Re: Ultimate Test
You are free to point out where I claimed that - or is this also one of things you say happened but you refuse to provide specific examples? I see a trend here, claim shit without one shred of factual evidence seems to be how you operate.
You argue that "most" (read stupid) Americans are convinced of this, there should exist ample examples of it. So once again, Koby the Coward, are you going to give specific examples of this anti-conservative bias: What conservative have been "censored" and what speech prompted this action?
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Ultimate Test
"So once again, Koby the Coward, are you going to give specific examples of this anti-conservative bias: What conservative have been "censored" and what speech prompted this action?"
You know he isn't going to answer that. If he did he'd first have to own up to racism and bigotry being a "conservative" viewpoint today. All he has is conflation and false premises. Like every other alt-right shill who tries to imply civil rights and democracy are "leftist".
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Ultimate Test
I've been pointing out specific conservative censorship on social media for months. Everything from Trump, to Project Veritas, but much much more gets listed weekly on the uncensored portion of the internet, outside of your echo chamber. Censorship such as the facebook rittenhouse support ban, Joe Rogan episode blacklisting, congressmen getting censored, as well as numerous smaller content creators.
The speech that triggered the censorship was perfectly legal, and even if someone felt insulted, it never warranted a takedown. It's up to you to defend the censorship, because I don't.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Ultimate Test
"Censorship such as the facebook rittenhouse support ban, Joe Rogan episode blacklisting, congressmen getting censored, as well as numerous smaller content creators."
So, in order you list, as examples:
1) Tasteless hero worship by self-confessed "proud chauvinists", bigots and racists, of a 17 year old would-be vigilante who drove into state lines with an unlawful AR-15, was accosted by and shot to death a mentally ill person, then ran at another person and his girlfriend with weapon high prompting another fracas and shooting, then finally shot the paramedic who tried to interfere?
2) Joe Rogan using his platform of influence to provide suicidal advise against the concensus of the expert medical community?
3) Congresspeople telling people the california wildfires were caused by Jewish space laser, which is admittedly among the weirdest anti-semitic assertion I've heard.
4) Smaller content creators, right. Tell us of these "conservative views" they were censored over.
"The speech that triggered the censorship was perfectly legal..."
Yeah, I can believe that. Calling some black man a <N-word> or ranting about the protocols of the elders of zion is perfectly legal under US law.
But that wasn't what we were asking, Koby. Being an asshole is also legal. And no one argues when you toss an asshole out of your house. So. Please. Give us just a few examples of what caused them to get blocked.
"...even if someone felt insulted, it never warranted a takedown."
So when someone walks into your house and insults you that doesn't mean you tossing them out is warranted? 😂
Damn, Koby, you've turned into some real life Monty Python skit there.
"It's up to you to defend the censorship, because I don't."
No one needs a defense for tossing an asshole out of their own property, Koby. Only in alt-right la-la land is that even a question.
And we all take note of your assertion that you don't believe in the concept of property ownership.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Re: Ultimate Test
Okay, so you've mentioned your reasoning as to why you feel that those takedowns were legitimate. Just looking at point #1:
Still isn't grounds for censoring any and all support of others.
Never entered WI with it, the weapon was already in WI. Charges dropped, you're lying here.
Not grounds for censorship.
Was down on the ground and assaulted prior to the shooting, and only shot assailants. Cleared of charges. Not grounds for censorship.
The paramedic lowered his arms while pointing a firearm, and charged forward. Cleared of charges. Not grounds for censorship.
All your points have been so wrong that facebook was forced to reverse its decision to censor, and discussion and support is now allowed.
Basically, you've got an opinion, which is fine. But then you classify any difference of opinion as being such an egregious violation of some nonexistent rule that it requires content takedown. The public sees this immoral behavior by leftists, and is disgusted. Your other points are the same: this is clear political difference of opinion, and cannot warrant censorship. If you want to takedown speech and ban users, without appearing biased, then you've got to provide better reasoning than "maybe" with a complete reversal after the trial. And you can say that corporations have the right to moderate however they want, but it won't make the bias disaapear.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Re: Re: Ultimate Test
It's not a hard question to answer bro.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Re: Re: Re: Ultimate Test
It's not, no. Koby knows better than to answer it and instead chooses to gaslight.
I guess his new tactic is to double down and derail rather than just vanish. Won't change much as we've all seen this playbook before around these parts.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Re: Re: Ultimate Test
"Still isn't grounds for censoring any and all support of others."
The only one who gets to choose what defines grounds for censoring is the owner of the property where that party took place.
If a bunch of Proud Boys want to show up and celebrate Rittenhouse "owning the libs" - shooting a mentally ill person, one person defending his girl, and the paramedic who tried to stop it - then I think I'm entitled to toss those people out of my house. And so is the owner of a social media platform.
What planet do you live on, Koby, where the owner of property can't evict someone from that property, based entirely on the property owner's opinion.
"But then you classify any difference of opinion as being such an egregious violation of some nonexistent rule that it requires content takedown. "
So all you've got is, once again, the statement that a property owner needs to justify tossing an unpleasant asshole out? Noted. Again.
"If you want to takedown speech and ban users, without appearing biased, then you've got to provide better reasoning than "maybe" with a complete reversal after the trial."
Why? Trial or not Rittenhouse remains an unpleasant asshole who shot three people needlessly.
Feel free to open your doors to the Proud Boys caroling over his release. I maintain that there's nothing strange at all for anyone with half a mind to keep denying them entry.
No, Koby, the fact that most people still think Rittenhouse is an ass and his new friends are all assholes is enough for any private enterprise engaged in hosting debates online to bar them entry. Few people want them around and thus they have no place where the majority gather. Because the majority does not want them anywhere near them.
That, incidentally, is how 1A's "right of association" works. You know, the one you keep trying to overturn.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Ultimate Test
No, you haven't and I'll explain why in a moment.
Trump - Broke the TOS several times and they let him stay because he was the president and the public interest.
Project Veritas - Known to edit videos to fit their narrative. Known to leave critical facts out of their stories.
Rittenhouse ban - People praising Rittenhouse for killing people (is that a conservative value now too?)
Joe Rogan - A dude making shit up as he go to befuddle his followers. A dude giving advice that goes against everything we know about medical science.
Congressmen - What did they do or say then?
Small content creators - What did they do or say then?
As usual, you didn't give specific examples. Anyway, if we sum this up and apply your reasoning that the above are all conservative views it doesn't paint a pretty picture. It's no surprise then that people are showing you the door, because nobody wants to deal with assholes.
What censorship? I constantly hear you and your ilk bleating "I'm being censored" while being disingenuous assholes everywhere you go. The entitlement is mindboggling from you snowflakes because you don't see anything wrong with forcing yourself upon others. Learn some fucking discretion.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Ultimate Test
Why wont you answer a simple question?
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Ultimate Test
FFS, please tell us Koby, seriously, why do you think Trump was given the boot from social media?
A) He is a conservative.
B) He used his large # of followers to instigate an attempted violent insurrection and overthrow congress to keep him as president once he lost the election.
If you seriously think that you can use Trump fucker as an example of anti-conservative bias, then you are living in an alternate reality where it seems that Trump can do nothing wrong and the entire world is out to get him.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Ultimate Test
I've been pointing out specific conservative censorship on social media for months.'
Show me the money bro
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Ultimate Test
[Hallucinates facts not in evidence]
[ link to this | view in chronology ]
Re: Re: Re: Re: Ultimate Test
Conservatives are convinced that bad faith censorship will be used against them
When you predict or guess at other people's behaviors and motivations by thinking, "What would I do?"
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Ultimate Test
It's called projection. They think they are being victimized and they complain how unfair it is blah blah, but the moment you give them the tools to victimize others they don't even hesitate to use it and brag about it.
For these people, everyone else not adhering to their beliefs has become like the beast from Lord of the Flies, and this is unscrupulously used by the GOP to lead them around.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Ultimate Test
Every accusation a confession, every self-given label a rejection of.
[ link to this | view in chronology ]
Re: Re: Re: Re: Ultimate Test
And it's 100% projection, same as the reason they cling to their white-supremacy privilege - the baseless phobia that those they mistreat will act exactly as they do.
Because notice how all the bad-faith censorship, contrary to Koby's hallucinations, has been pretty much exclusively from Republicans.
[ link to this | view in chronology ]
Re: Re: Re: Re: Ultimate Test
That you c;aim that's anything even remotely like what he said says volumes about your (lack of) literacy.
[ link to this | view in chronology ]
Re: Ultimate Test
Hey Koby we lost you on the other thread. Answer the question bro.
[ link to this | view in chronology ]
Re: Ultimate Test
The fact you side with white nationalists trying to take away the free speech of others and manage to blame those others for it to make yourself a victim is telling.
Conservatives don't want free speech, they want the rest of society to sit down and shut up while they talk and to be unable to offer pushback of any kind. Remember how Conservatives silenced people for years after 9/11? Of course you do, that's the free speech you want.
[ link to this | view in chronology ]
Masnick's Law of Process Abuse
"If you give people tools to take down some content, they will be abused."
[ link to this | view in chronology ]
In other words, they say the problem was/is not the policy itself but that the enforcement teams applied it equally for everyone, instead of showing favoritism to people the management likes.
[ link to this | view in chronology ]
First rule of any policy which permits input on decisions from outside sources: the overwhelming majority of input will be from malicious parties attempting to misuse the policy for their own purposes. If your system cannot correctly handle at least 50% of the input being the absolute worst sort of abuse possible, it should be considered unfit for purpose.
Corollary: if you think you've thought of the worst sort of abuse possible, your naivety and innocence are about to be destroyed in the worst possible way.
[ link to this | view in chronology ]
Re:
In short:
All policies are abuse-proof, until the emergence of a better troll.
Corollary:
There is always a better troll.
[ link to this | view in chronology ]
Any system that makes it easy to remove content will be use to silence or censor minoritys or someone who makes political statements ,the dmca law is being used every day by big companys to remove public domain videos parodys or legal content, eg videos that are fair use that contain snippets of film or tv shows for the purposes of review .or maybe contain a few secons of music
Many youtubers just accept illegal takedown because they do not have the time to fight false claims or they are afraid they might get a copyright strike against their account.
[ link to this | view in chronology ]
Re:
More importantly, they do not have the money, and the cost of pursuing claims far exceeds any potential income from the videos.
[ link to this | view in chronology ]
new CEO
Every new CEO (or manager at any level) has to pee on the turf they took over to show their dominance. And the result was just like a stream of urine.
[ link to this | view in chronology ]
“Some” “malicious“ reports are referred to as inaccurately reported or otherwise falling outside the reporting guidelines regarding public-personage or newsworthy images. Their teams simply got some wrong, they say.
Others were also outside the guidelines, apparently. But how? That “anti-extremism researchers” were affected seems an important reason, judging by the Post article.
It appears that what Twitter at least in part means by “used as intended” is that their guidelines (which were drawn to evince neutrality) are appropriately applied when some of the wrong people are neutrally punished and/or some of the right people are neutrally protected.
[ link to this | view in chronology ]
'So... when are you going to start talking about the downsides?'
There is so much effort these days to pressure (or require!) internet companies to remove "bad" content, but almost none of those policy plans take into account the ways in which those policies will be abused to silence reporting, silence marginalized voices, and silence those calling out abuses of power.
DC politicians/policymakers: That's a feature, not a bug.
[ link to this | view in chronology ]
Masnick's Law of Process Abuse?
Masnick's third law: "If you give people tools to take down some content, they will be abused"
[ link to this | view in chronology ]
Demonstration case
Maybe that was the reason they did it? Next time they get called before congress they can say "a bunch of right wing trolls actively coordinated to exploit our system". It pushes back against the "conservatives are being silenced" narrative as well.
[ link to this | view in chronology ]