Twitter's New 'Private Information' Policy Takes Impossible Content Moderation Challenges To New, Ridiculous Levels

from the the-end-of-everything dept

I've been working on a post about a whole bunch of interesting (and good!) things that Twitter has been doing over the last few months. I'm not sure when that post will be ready, but I need to interrupt that process to note a truly head-scratching change in Twitter's moderation policy announced this week. Specifically, Twitter announced that its "private information policy" has now been expanded to include "media." Specifically, Twitter says that it will remove photos and videos that are posted "without the permission of the person(s) depicted." In other words, Twitter has taken the old, meme-ified, "I'm in this photo and I don't like it" into official policy for taking down content.

Buried deeper in the rules is a very subjective conditional:

This policy is not applicable to media featuring public figures or individuals when media and accompanying text are shared in the public interest or add value to public discourse.

But that's going to lead to some very, very big judgment calls about (a) who is a "public figure" and (b) what is "in the public interest." And early examples suggest that Twitter's Trust & Safety team are failing this test.

I can understand the high level, first pass thinking that leads to this policy: there are situations in which photos or videos that are taken surreptitiously and then are used to mock or harass someone can certainly raise questions and there are perfectly reasonable policy choices to be made on how to handle those scenarios. But how do you distinguish those rare circumstances with a much wider series of cases where people may not have given permission to be in photos or videos, but keeping that content online is clearly beneficial. This can range from the obvious incidental background images of people walking by in a crowded place to -- much more concerning -- situations of journalism being done by individuals, recording important events. Twitter's insistence that it won't apply to "public interest" issues is hardly reassuring. We've seen those claims before and they rarely work in practice.

The most obvious example of this is one of the biggest stories of 2020: the video taping of police officer Derek Chauvin kneeling on George Floyd's neck until he died. In theory, under the broadness of this policy, that video would be taken down off of Twitter. There are lots of other situations as well, including things like Amy Cooper, who was filmed in Central Park calling the police on Christian Cooper (no relation to Amy) who was in the park bird-watching. There are plenty of other examples where people are filmed in public, without their permission, but it's done to reveal important things that have happened in the world. For example, law enforcement relied on help from social media to help identify people who stormed the Capital on January 6th. It seems that under this new policy, all those photos of January 6th insurrectionists could be removed from Twitter. Is sharing them in the public interest? Depends on who you ask, I imagine...

For years we've seen tons of people abusing other systems to take down content they didn't like. For example, there was the part owner of the Miami Heat who literally sued over an unflattering photo by first obtaining the copyright for it. Or the revenge porn extortionist who tried to force stories about him offline with copyright notices. In Europe, we've seen something similar with abuses of the "right to be forgotten" to memory hole news stories.

And here, Twitter is setting up to just take down any such photo or video upon request? This seems wide open for massive levels of abuse. Indeed, there are already a number of reports about the policy being used to silence activists and researchers:

Yes, even some of the examples above may be considered edge cases with more nuance than is presented by the people posting them, but as we've seen with copyright and the right to be forgotten, give people a tool to get any information removed from social media, and it will be massively and widely abused to try to hide bad behavior.

I'm honestly perplexed at why Twitter implemented such a broad policy, so difficult to enforce, and so open to abuse. It seems extremely unlike the more thoughtful trust & safety moves the company has made over the past few years.

Filed Under: content moderation, media, newsworthy, photos, private information, public figure, trust & safety, videos
Companies: twitter

Trust & Safety Professional Association Launches: This Is Important

from the exciting-news dept

One of the most frustrating things out there is the idea that content moderation choices made on various platforms are coming directly from the top. Too often, I've seen people blame Jack Dorsey or Mark Zuckerberg for content moderation decisions, as if they're sitting there at their laptops and twiddling their fingers over who gets blocked and who doesn't. Over the last decade or so, an entire industry has been built up to figure out how to make internet services as usable as possible, to deal with spam, and abuse, and more. That industry is generally called "trust and safety," and as a new industry it has grown up and professionalized quite a bit in the last decade -- though it rarely (if ever) gets the respect it deserves. As I mentioned on a recent episode of The Pivot podcast, many of the assumptions that people make about content moderation unfairly malign the large crew of people working in trust and safety who aren't interested in political bias, or silencing voices, but who legitimately are working very, very hard to figure out how to balance the many, many tradeoffs in trying to make internet services useful and welcoming to users.

That's why I'm really happy to see a new organization launch today, the Trust & Safety Professional Association, along with a sister organization, the Trust & Safety Foundation.

Today, we’re pleased to announce the Trust & Safety Professional Association (TSPA) and the Trust & Safety Foundation Project (TSF).* TSPA is a new, nonprofit, membership-based organization that will support the global community of professionals who develop and enforce principles and policies that define acceptable behavior online. TSF will focus on improving society’s understanding of trust and safety, including the operational practices used in content moderation, through educational programs and multidisciplinary research. Neither TSPA nor TSF are lobbying organizations, and will not advocate for public policy positions on behalf of corporate supporters or anyone else. Instead, we will support the community of people doing the work, and society’s understanding of it.

And I should note that the people behind this organization are incredible. If you told me about such an organization and asked me to suggest who should be involved, I would have included exactly the people who put this together, starting Adelin Cai and Clara Tsao, who both have tremendous experience in the trust and safety space, and the knowledge and thoughtful, balanced approach necessary to build organizations like the two launched today. If you ever need someone to talk through all the challenges to think through in building a successful trust and safety team, I'd highly recommend both Adelin and Clara. The board also includes some names you may recognize, including Professor Eric Goldman, former Twitter/Google lawyer and White House deputy CTO Alex Macgillivray, and former Mozilla Chief Legal Officer/COO and current Stellar Development Foundation CEO Denelle Dixon.

And... one of the initial projects that the Trust & Safety Foundation has launched is an ongoing series of trust and safety case studies written by... us. Techdirt's think tank arm, the Copia Institute, will be providing a series of trust and safety case studies to the Trust & Safety Foundation, which they'll be posting each week. We'll eventually be posting many of them to Techdirt as well, so you can expect those coming later this summer. The point of this library of case studies is to give people a better understanding of the impossible choices and tradeoffs that internet services need to make on a daily basis, and to highlight why what often seems like an "obvious" way to deal with some piece of content may not be so obvious once you explore it from all sides. Personally, I'm excited to get to help build out this library and to work with such a great team of people who are devoted to improving and professionalizing the space, while further educating everyone (both inside and outside the trust and safety space) how trust and safety efforts actually work.

Filed Under: case studies, content moderation, education, professional association, trust & safety, trust and safety
Companies: copia institute, tsf, tspa

Twitter Suspends Reporter's Account... After He Gets Targeted By Russian Twitter Bots

from the nice-work,-geniuses dept

Over the last few weeks, we've written a number of times about how systematically bad internet platforms are at determining how to deal with abuse online. This is not to mock the platforms, even though many of the decisions are comically bad, but to note that this is inevitable in dealing with the scale of these platforms -- and to remind people that this is why it's dangerous to demand that these companies be legally liable for policing speech on their platforms. It won't end well. Just a few weeks ago, we wrote about how Twitter suspended Ken "Popehat" White for posting an email threat he'd received (Twitter argued he was violating the privacy of the guy threatening him). From there, we wrote about a bunch of stories of Facebook and Twitter punishing people for documenting abuse that they had received.

But this latest story is even slightly crazier, as it appears that abusers were taking advantage of this on purpose. In this case, the story involves Russian Twitter bots. First, the Atlantic Council wrote about Russian Twitter trolls trying to shape a narrative after the Nazi event in Charlottesville. In response, those very same Twitter bots and trolls started bombarding the Twitter feeds of the researchers. And here's where the story gets even weirder. When Joseph Cox, writing for The Daily Beast, wrote about this (at the link above), those same Twitter bots started targeting him too.

And... that caused Twitter to suspend his account. No, really.

“Caution: This account is temporarily restricted,” a message on my account read Tuesday. “You’re seeing this warning because there has been some unusual activity from this account,” it continued.

Again, it's not hard to see how this happened. Cox's Twitter account suddenly took on a bunch of bot followers, many of whom started retweeting him. From Twitter's perspective, it's easy to see how that looks like someone gaming the system -- possibly buying up fake followers and fake retweets. But, here, it appears to have been done to target the user, rather than to fake boost him. After all, it's completely understandable why Twitter would have a system that would seek out situations where a ton of fake followers were suddenly following someone and retweeting them. That would be a clear pattern indicating spam or something nefarious. And, in designing the system, you might think that such a thing would never be used to harm someone -- but by building in the mechanism to recognize this is happening and to suspend the account, you're now creating a weapon that will be gamed.

Cox eventually got his account back and got an apology ("for the inconvenience") from Twitter. But, once again, for everyone out there demanding that these platforms be more forceful in removing users, or (worse) arguing that there should be legal liability on them if they fail to kick off people expeditiously, be careful what you wish for. You may get it... and not like the results.

Filed Under: fraud, joseph cox, moderation, russian bots, trust & safety, twitter bots
Companies: twitter