Twitter Testing Secret Filter To Stop Abuse: Is That A Good Thing Or An Attack On Free Speech?
from the this-isn't-easy dept
For quite some time now, it's become popular for people to point to some (often horrible) abuse on Twitter and demand that Twitter "do something" about it. It's rather easy to find tons of articles calling for Twitter to do more.And apparently Twitter is listening. As Sarah Jeong notes in a great article over at The Verge, it appears that Twitter recently deployed a feature to block some very abusive tweets, going further than its past tactics of pulling abusive tweets and killing accounts. In this case, it blocked certain tweets from being sent altogether. It appeared to be a filter that combined certain keywords with an @ symbol towards a person receiving a lot of abuse. Of course, as with many such things, the abusers just sought ways around the filter.
For a while, at least, Berger didn’t receive any tweets containing anti-Semitic slurs, including relatively innocuous words like "rat." If an account attempted to @-mention her in a tweet containing certain slurs, it would receive an error message, and the tweet would not be allowed to send. Frustrated by their inability to tweet at Berger, the harassers began to find novel ways to defeat the filter, like using dashes between the letters of slurs, or pictures to evade the text filters. One white supremacist site documented various ways to evade Twitter’s censorship, urging others to "keep this rolling, no matter what."The assumption, from those with at least some understanding of what happened, is that it was done via Twitter's spam filter:
Trollish behavior and abusive behavior is definitely a concern. And making people feel welcome online on services like Twitter, without fear of harassment seems like a welcome goal. But there are also some pretty serious concerns about how this is all happening in a non-transparent manner, leaving it open to abuse in its own way:A source familiar with the incident told us, "Things were used that were definitely abnormal."
A former engineer at Twitter, speaking on the condition of anonymity, agreed, saying, "There’s no system expressly designed to censor communication between individuals. … It’s not normal, what they’re doing."
He and another former Twitter employee speculated that the censorship might have been repurposed from anti-spam tools—in particular, BotMaker, which is described here in an engineering blog post by Twitter. BotMaker can, according to Twitter "deny any Tweets" that match certain conditions. A tweet that runs afoul of BotMaker will simply be prevented from being sent out—an error message will pop up instead. The system is, according to a source, "really open-ended" and is frequently edited by contractors under wide-ranging conditions in order to effectively fight spam.
What’s worrisome to free speech advocacy groups like the EFF about this incident is how quietly it happened. Others may see the bigger problem being the fact that it appears to have been done for the benefit of a single, high-profile user, rather than to fix Twitter’s larger harassment issues. The selective censorship doesn’t seem to reflect a change in Twitter abuse policies or how they handle abuse directed at the average user; aside from a vague public statement by Twitter that elides the specific details of the unprecedented move, and a few, mostly-unread complaints by white supremacists, the entire thing could have gone unnoticed.It opens up some pretty serious questions, and you enter into the same slippery slope that questions around legislating against harassing speech, or even things like revenge porn, start to raise some concerns. It's not that anyone wants to support those activities. The harassment here is absolutely deplorable. But there is a serious concern about where it leads when an intermediary suddenly takes it upon itself to determine what is and what is not acceptable speech. Was it used for good reasons in this case? Probably. But will it always be done in that manner? That's where it gets a lot trickier.
Eva Galperin thinks incidents like these could be put in check by transparency reports documenting the application of the terms of services, similar to how Twitter already puts out transparency reports for government requests and DMCA notices. But while a transparency report might offer users better information as to how and why their tweets are removed, some still worry about the free-speech ramifications of what transpired. One source familiar with the matter said that the tools Twitter is testing "are extremely aggressive and could be preventing political speech down the road." He added, "are these systems going to be used whenever politicians are upset about something?"
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: abuse, filter, free speech
Companies: twitter
Reader Comments
The First Word
“Why is this complicated? Just give spam control to users
Instead of foisting spam filters unilaterally, just allow users to filter and reject tweets based on their own individual filter settings.It's what I do with Open Tweet Filter and my choices apply to me and no one else.
Subscribe: RSS
View by: Time | Thread
Stupid computer.
[ link to this | view in thread ]
Good first step.
Just sayin...
[ link to this | view in thread ]
[ link to this | view in thread ]
Wrong approach to the problem
Among other problems, what if someone wanted to tweet a supportive (if vacuous) statement like "@Berger is a good person. Don't call her a rat." or an informative (in certain contexts) statement like "@Berger, when you travel to #NewYorkCity, beware the rats. They're mean and usually diseased." Both of these are harmless to good statements, yet would likely be wrongly flagged by the filter because it happens to contain a word that has been twisted to be a slur.
[ link to this | view in thread ]
[ link to this | view in thread ]
Re:
That makes 'em a public company.
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
Re:
Their motivation is important. If they are doing this kind of filtering under pressure that if they do not they may be held liable, this kind of thing can have a chilling effect not only on the use of Twitter, but on other companies that may want to compete in the same space.
Proponents of free speech should always cheer on companies and services that allow free expression of unpopular ideas and frown upon companies that quell speech in any way.
[ link to this | view in thread ]
Re: Re:
Remember these services are shopping malls, not public squares.
[ link to this | view in thread ]
Re: Re: Re:
[ link to this | view in thread ]
Content-based measures don't work and won't work
Content-based measures do have their uses: properly deployed, they can yield interesting and useful information about possible sources of spam, phishing, typosquatting and other forms of abuse. But the best use for that information is to collect it, collate it, summarize and then bring it to the attention of (well-trained) humans, who will hopefully exercise due diligence and will use out-of-band data to augment and check their conclusions.
That's not what most people want to hear: they want to hear that the problem can be completely solved via automation, because that's cheap and easy. But it can't, and attempts to do so inevitably devolve into a cyclical pattern of (new) attack and (new) defense, followed by (newer) attack and (newer) defense, followed by more of the same. We've seen this with anti-virus software. We've seen this with anti-spam software. And now we're seeing with anti-whatever-it-is that Twitter wants to call this.
Marcus Ranum, in his brilliant rant "The Six Dumbest Ideas in Computer Security", see http://www.ranum.com/security/computer_security/editorials/dumb/ for the whole thing, nailed this nearly a decade ago, as dumb idea #2: Enumerating Badness. He says in part: "One clear symptom that you have a case of "Enumerating Badness" is that you've got a system or software that needs signature updates on a regular basis [...]". Bingo.
[ link to this | view in thread ]
Re: Re: Re:
Kind of like how private phone companies should be able censor what people say on the phone.
[ link to this | view in thread ]
Pointless and dangerous
[ link to this | view in thread ]
Why is this complicated? Just give spam control to users
It's what I do with Open Tweet Filter and my choices apply to me and no one else.
[ link to this | view in thread ]
Re: Re: Re: Re:
That's the beauty of a free and open internet; all sites are equally accessible. Centralization and censorship only exist so long as people support it.
I'd look into supporting and/or hosting your own StatusNet or pump.io instance. They're by no means perfect, but supporting and using software that guarantees rights is a much better long term option than asking entities to grant you rights that they have no obligation to. Especially as this censorship becomes more concealed and insidious.
[ link to this | view in thread ]
sticks and stones may break my bones ...
But as is usually the case, people who are high up tend to get special privileges not afforded to regular people, and these special privileges have a tendency to increase over time.
"The harassment here is absolutely deplorable."
And that's the whole point. There have been numerous cases in the past in which bigoted speech (or misconduct in general) by anonymous people turned out to be a "false-flag operation" done to create sympathy for the victim and demonize the side thought to be responsible. Even in "legitimate" cases, an agent provocateur may be planted behind the scenes. We need to keep in mind that these childish antics are not always what they seem.
[ link to this | view in thread ]
Re: Re: Re: Re:
[ link to this | view in thread ]
Re: Why is this complicated? Just give spam control to users
Friends, Fun, Family.
Many applications do anyway, but I see no reason why I as a user can't have the tools to restrict my own coversations to suit what I want from it.
[ link to this | view in thread ]
Re: Re: Re: Re:
i have NEVER (except excerpted in articles) seen a tweet (*sheesh* i feel like i'm losing IQ points just using this stupid lingo) or care to; my life is NOT poorer for that...
that there are a few REAL dissidents, occupy types, etc who have used it to good effect is besides the point, it is mostly useless...
[ link to this | view in thread ]
Re: Re: Re: Re: Re:
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Re: Re: Re: Re:
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
And on the other hand...
Including emails I specifically requested.
[ link to this | view in thread ]
Fascist nonsense.
[ link to this | view in thread ]
not feasible
There are some two million words in English, and I would defy any filter to be able to deal with them, never mind the implications of an enormous number of ways to interpret combinations.
Quite a few companies on the net have gone out of business attempting censorship
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
The same Twitter?
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Re: Re:
Interesting comment, since courts have ruled that places like shopping malls are public spaces for certain things, including free speech issues.
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
If you do not like it then use another service. And that is another reason why I do not consider it violation of anyons freedom of speech. It is not the only way to say something on the internet, just one of the most widely used. That does not justify concerns about free speech.
[ link to this | view in thread ]
Re:
Who said otherwise?
"So it is completely within their rights to block content they find unacceptable and not be transparent about it."
Again, who said otherwise. Straw-man much?
"And that is another reason why I do not consider it violation of anyons freedom of speech."
But if you're trying to twist that into saying that no one's speech is being restricted, then you're just wrong. No one is saying that Twitter is breaking any laws. Just pointing out what how they are restricting speech. And restricted speech is not free speech.
[ link to this | view in thread ]
[ link to this | view in thread ]
They own the damn thing, you agree to their terms, end of story. Not happy? Use one of the 50 competitors.
[ link to this | view in thread ]
Their Business, Their Rules
They're publishers. Users are just the saps who write their copy for free.
If you don't want to be edited, get your own site.
[ link to this | view in thread ]
Freedom of speech upheld
Actually, Twitter didn't decide what people are allowed to say, they simply upheld the harassment recipients' right to freedom of speech by blocking stuff aimed directly at them. While people have a right to say what they like, that right ends when it interferes with the rights of others, and tweeting @[name] to put something directly into someone's Twitter stream forces them to see it, thus interfering with their right to freedom of speech. Simples!
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Why is this complicated? Just give spam control to users
[ link to this | view in thread ]
Re: Re: Re: Re:
If you can get enough people to start using your platform, it then becomes viable. The problem is, everybody is on Twitter, so nobody wants to go anywhere else because nobody is there.
When they don't like what Twitter does, people just quit using it instead of pursuing alternatives. It's the easy (lazy) approach - getting your friends to sign up for another platform is just too much work.
[ link to this | view in thread ]
Re: Freedom of speech upheld
The Open Tweet Filter recommended by NoahVail seems like the most reasonable response. They can't bug you if they can't get to you.
[ link to this | view in thread ]
Re: Re: Freedom of speech upheld
And isn't that what I said?
[ link to this | view in thread ]