Content Moderation Case Study: Documenting Police Brutality (2007)

from the documenting-abuse-vs-promoting-abuse dept

Summary: Wael Abbas is an Egyption journalist/activist who began documenting protests in Egypt in 2006, including multiple examples of Egyptian police brutality, which he would then upload to YouTube.

In 2007, after posting a few explicit examples of Egyptian police brutality, he discovered that his entire YouTube account was shut down, taking down 181 videos covering not just police brutality, but also voting irregularities, and street protests. At first YouTube refused to comment on this, and only told Abbas that the account was shut down due to multiple complaints about the content.

Later, after the US press got ahold of the story, YouTube put out a statement saying:

Our general policy against graphic violence led to the removal of videos documenting alleged human rights abuses because the context was not apparent

Having reviewed the case, we have restored the account of Egyptian blogger Wael Abbas. And if he chooses to upload the video again with sufficient context so that users can understand his important message, we will of course leave it on the site

Wael believes that if large media organizations like Reuters and CNN hadn’t covered his case, that it was unlikely his account would have been restored, or that he would have been allowed to re-upload the videos.

Decisions to be made by YouTube:

• How do you determine the difference between a journalist/activist documenting violence and an account that is “glorifying” violence?
• Is there a way to determine the context of a video showing police brutality?
• Should content moderation decisions change based on whether or not a specific situation is getting mainstream press attention?
• Are there alternatives beyond shutting down an entire account based on complaints about some videos?

Questions and policy implications to consider:

• Should context play a bigger role in content moderation and if so, how can you take that into account? Or is it the responsibility of the account holder to supply the context?
• How do you manage moderation of content from a country with different rules than in the US?
• Will suspensions by US social media companies be used as evidence against the content creators in certain countries?

Resolution: As noted above, YouTube did reinstate his account, but as issues like this continued to arise, the company has adjusted its policies for handling violent but newsworthy content multiple times in the intervening years. At the time of writing this case study, Abbas’ videos showing Egyptian police brutality from many years ago now contain content warnings saying that the content “may be inappropriate for some viewers” and asking users to acknowledge that before being able to view the videos.

Abbas has faced many more content moderation challenges since then with his work in Egypt. Yahoo shut down his email account after getting complaints. Both Twitter and Facebook have suspended his accounts at times as well.

In both 2010 and 2018 Abbas was arrested in Egypt for his work, with Egyptian authorities using the social media account suspensions as evidence of his alleged crimes in “spreading fake news.”

Filed Under: content moderation, documenting, egypt, journalism, police brutality, reporting, wael abbas

The Impossibility Of Content Moderation: YouTube's New Ban On Nazis Hits Reporter Who Documents Extremism, Professor Teaching About Hitler

from the so-that's-all-working-well dept

So just as the recent big content moderation mess was happening on YouTube, the company announced that it had changed its policies to better deal with violent extremism and supremacism on the platform:

Today, we're taking another step in our hate speech policy by specifically prohibiting videos alleging that a group is superior in order to justify discrimination, segregation or exclusion based on qualities like age, gender, race, caste, religion, sexual orientation or veteran status. This would include, for example, videos that promote or glorify Nazi ideology, which is inherently discriminatory. Finally, we will remove content denying that well-documented violent events, like the Holocaust or the shooting at Sandy Hook Elementary, took place.

The timing of this announcement was seen as curious (or, at the very least, poorly timed) as it came basically hours after they had refused to take down Steven Crowder's account (see the earlier post linked above), even though that wasn't an identical situation -- though analogous enough that tons of people commented on it.

In making the announcement, YouTube correctly noted that this new bit of line drawing could represent some problems, including among those tracking hate and extremism:

We recognize some of this content has value to researchers and NGOs looking to understand hate in order to combat it, and we are exploring options to make it available to them in the future. And as always, context matters, so some videos could remain up because they discuss topics like pending legislation, aim to condemn or expose hate, or provide analysis of current events. We will begin enforcing this updated policy today; however, it will take time for our systems to fully ramp up and we’ll be gradually expanding coverage over the next several months.

But within hours of the new policy rolling out, we were already seeing how difficult it is to implement without taking down content that probably deserves to remain up. Ford Fischer, a reporter who tracks extremist and hate groups, and whose work is regularly cited, noted that his own channel had been demonetized.

Fischer than discusses the specific videos that YouTube says is the reason for this -- and it does include holocaust denialism, but for the sake of documenting it, not promoting it:

And this gets, once again, to the very problem of expecting platforms to police this kind of speech. The exact same content can mean very different things in different contexts. In some cases, it may be used to promote odious ideology. In other cases, it's used to document and expose that ideology and the ignorance and problems associated with it.

But how do you craft a policy that can determine one from the other? As YouTube is discovering (truth is, they probably already knew this), the answer is that you don't. Any policy ends up creating some sort of collateral damage, and the demands from well meaning people mean that the direction this tends to go in leads to greater and greater takedowns. But, if in the process of doing this we end up sweeping the documentation under the rug, that's a problem as well.

Here's another example: right after YouTube's new policy was put in place, a history teacher found that his own YouTube channel was banned. Why? Because he hosted archival footage of Hitler:

“My stomach fell,” Allsop told BuzzFeed News via email. “I’m a history teacher, not someone who promotes hatred. I share archive footage and study materials to help students learn about the past.”

Once again, it often sounds easy to say something like "well, let's ban the Nazis." I'd even argue it's a reasonable goal for a platform to have a blanket "no Nazis" policy. But the reality is that the implementation is not nearly as easy as many people believe. And the end result can be that archival and documentary footage gets blocked. And that could have serious long term consequences if part of our goal is to educate people about why Nazis are bad.

Of course, none of this should come as a surprise to anyone who's been dealing with these issues over the past couple of decades. Early attempts to ban "porn" also took down information on breast cancer. Attempts to block "terrorist content" have repeatedly taken down people documenting war crimes. This kind of thing happens over and over and over again and believing that this time will magically be different is a fool's errand.

Filed Under: content moderation, content moderation at scale, documenting, learning, nazis, policing speech, reporting, research
Companies: youtube