Oversight Board Tells Facebook It Needs To Shape Up And Be More Careful About Silencing Minorities Seeking To Criticize The Powerful

from the pay-attention-to-this dept

Tomorrow, the Oversight Board is set to reveal its opinion on whether Facebook made the right decision in banning former President Trump. And that will get tons of attention. But the Board came out with an interesting decision last week regarding a content takedown in India, that got almost no attention at all.

Just last week, we wrote about an ongoing issue in India, where the government of Prime Minister Narendra Modi has failed in almost every way possible in dealing with the COVID pandemic, but has decided the best thing to focus on right now is silencing critics on Twitter. That backdrop is pretty important considering that the very next day, the Oversight Board scolded Facebook for taking down content criticizing Modi's government.

That takedown was somewhat different and the context was very different. Also, it should be noted that as soon as the Oversight Board agreed to take the case, Facebook admitted it had made a mistake and reinstated the content. However, this case demonstrates something important that often gets lost in all of the evidence free hand-wringing about "anti-conservative bias" from people who wrongly insist that Facebook and Twitter only moderate the accounts of their friends. The truth is that content all across the board gets moderated -- and often the impact is strongest on the least powerful groups. But, of course, part of their lack of power is that they're unable to rush onto Fox News and whine about how they're being "censored."

The details here are worth understanding, not because there was some difficult decision to make. Indeed, as noted already, Facebook realized it made a mistake almost immediately after the Oversight Board decided to look into this, and when asked why the content was taken down, basically admitted that it had no idea and that it was a complete and total mistake. Here was the content, as described by the Oversight Board ruling:

The content touched on allegations of discrimination against minorities and silencing of the opposition in India by “Rashtriya Swayamsevak Sangh” (RSS) and the Bharatiya Janata Party (BJP). RSS is a Hindu nationalist organization that has allegedly been involved in violence against religious minorities in India. “BJP” is India’s ruling party to which the current Indian Prime Minister Narendra Modi belongs, and has close ties with RSS.

In November 2020, a user shared a video post from Punjabi-language online media Global Punjab TV and an accompanying text. The post featured a 17-minute interview with Professor Manjit Singh, described as “a social activist and supporter of the Punjabi culture.” In its post, Global Punjab TV included the caption “RSS is the new threat. Ram Naam Satya Hai. The BJP moved towards extremism.” The media company also included an additional description “New Threat. Ram Naam Satya Hai! The BJP has moved towards extremism. Scholars directly challenge Modi!” The content was posted during India’s mass farmer protests and briefly touched on the reasons behind the protests and praised them.

The user added accompanying text when sharing Global Punjab TV’s post in which they stated that the CIA designated the RSS a “fanatic Hindu terrorist organization” and that Indian Prime Minister Narendra Modi was once its president. The user wrote that the RSS was threatening to kill Sikhs, a minority religious group in India, and to repeat the “deadly saga” of 1984 when Hindu mobs attacked Sikhs. They stated that “The RSS used the Death Phrase ‘Ram naam sat hai’.” The Board understands the phrase "Ram Naam Satya Hai" to be a funeral chant that has allegedly been used as a threat by some Hindu nationalists. The user alleged that Prime Minister Modi himself is formulating the threat of “Genocide of the Sikhs” on advice of the RSS President, Mohan Bhagwat. The accompanying text ends with a claim that Sikhs in India should be on high alert and that Sikh regiments in the army have warned Prime Minister Modi of their willingness to die to protect the Sikh farmers and their land in Punjab.

The post was up for 14 days and viewed fewer than 500 times before it was reported by another user for “terrorism.” A human reviewer determined that the post violated the Community Standard on Dangerous Individuals and Organizations and took down the content, which also triggered an automatic restriction on the use of the account for a fixed period of time. In its notification to the user, Facebook noted that its decision was final and could not be reviewed due to a temporary reduction in its review capacity due to COVID-19. For this reason, the user appealed to the Oversight Board.

So, you had an ethnic minority -- one who had been attacked in the past -- warning about those currently in power. And Facebook took it down, refused to review the appeal... until the Oversight Board turned its eye on it, and then admitted it was a mistake, and basically threw its hands in the air and said it had no idea why it had been taken down in the first place.

According to Facebook, following a single report against the post, the person who reviewed the content wrongly found a violation of the of the Dangerous Individuals and Organizations Community Standard. Facebook informed the Board that the user’s post included no reference to individuals or organizations designated as dangerous. It followed that the post contained no violating praise.

Facebook explained that the error was due to the length of the video (17 minutes), the number of speakers (two), the complexity of the content, and its claims about various political groups. The company added that content reviewers look at thousands of pieces of content every day and mistakes happen during that process. Due to the volume of content, Facebook stated that content reviewers are not always able to watch videos in full. Facebook was unable to specify the part of the content the reviewer found to violate the company’s rules.

Got that? Facebook is basically saying "yeah, it was a mistake, but that was because it was a long video, and we just had one person reviewing who probably didn't watch the whole video."

Here's the thing that the "oh no, Facebook is censoring people" don't get. This happens all the time. And none of us hear about it because the people it happens to often are unable to make themselves heard. They don't get to run to Fox News or Parler or some other place and yell and scream. And, this kind of "accidental" moderation especially happens to the marginalized. Reviewers may not fully understand what's going on, or not really understand the overall context, and may take the "report" claim at face value, rather than having the ability or time to fully investigate.

In the end, the Oversight Board told Facebook to put back the content, which was a no-brainer since Facebook had already done so. However, more interesting were its policy recommendations (which, again, are not binding on Facebook, but which the company promises to respond to). Here, the Oversight Board said that Facebook should make its community standards much more accessible and understandable, including translating the rules into more languages.

However, the more interesting bit was that it said that Facebook "should restore human review and access to a human appeals process to pre-pandemic levels as soon as possible while fully protecting the health of Facebook’s staff and contractors." There were some concerns, early in the pandemic, about how well content moderation teams could work from home, since a lot of that job involves looking at fairly sensitive material. So, there may be reasons this is not really doable just yet.

Still, this case demonstrates a key point that we've tried to raise about the impossibility of doing content moderation at scale. So much of it is not about biases, or incompetence, or bad policies, or not wanting to do what's right. A hell of a lot of it is just... when you're trying to keep a website used by half the world operating, mistakes are going to be made.

Filed Under: appeals, content moderation, free speech, india, minorities, mistakes, review, takedowns
Companies: facebook, oversight board

NIST Study Of 189 Facial Recognition Algorithms Finds Minorities Are Misidentified Almost 100 Times More Often Than White Men

from the white-dudes-score-another-unearned-perk dept

The development and deployment of facial recognition tech continues steadily, but the algorithms involved don't seem to be getting much better at recognizing faces. Recognizing faces is pretty much the only thing it's expected to do and it can't seem to get the job done well enough to entrust with it things like determining whether or not a person is going to be detained or arrested.

That critical failure hasn't slowed down deployment by government agencies. There are a handful of facial recognition tech bans in place around the nation, but for the most part, questions about the tech are being ignored in favor of the potential benefits touted by government contractors.

Last year, members of Congress started demanding answers from Amazon after its "Rekogition" tech said 28 lawmakers were criminals. Amazon's response was: you're using the software wrong. That didn't really answer the questions raised by this experiment -- especially questions about the tech's disproportionate selection of minorities as potential perps.

This has been a problem with facial recognition tech for years now. Biases introduced into the system by developers become amplified when the systems attempt to match faces to stored photos. A recent study by the National Institute of Standards and Technology (NIST) found that multiple facial recognition programs all suffer from the same issue: an inordinate number of false positives targeting people of color.

Asian and African American people were up to 100 times more likely to be misidentified than white men, depending on the particular algorithm and type of search. Native Americans had the highest false-positive rate of all ethnicities, according to the study, which found that systems varied widely in their accuracy.

The faces of African American women were falsely identified more often in the kinds of searches used by police investigators where an image is compared to thousands or millions of others in hopes of identifying a suspect.

As usual, the group most likely to be accurately assessed by facial recognition tech is also the group that most often promotes and deploys facial recognition systems.

Middle-aged white men generally benefited from the highest accuracy rates.

That this study was performed by NIST makes it a lot tougher to ignore. While other studies could be brushed off as anomalies or biased themselves (when performed by civil rights activist groups), a federal study of 189 different facial recognition algorithms submitted by 99 companies isn't as easy to wave away as unsound.

One of the more adamant critics of facial recognition tech critics is Amazon. It's the company that told the ACLU it was using the system wrong after the rights group took the system for a spin last year and netted 28 false positives using Congressional reps' photos. Amazon had a chance to prove its system was far more accurate than the ACLU's tests showed it was but it chose to sit out the NIST trials.

The problems found by NIST exist in both "one-to-one" and "one-to-many" matching. False positives for "one-to-one" matching allows unauthorized access of devices, systems, or areas secured with biometric scanners. "One-to-many" mismatches are even more problematic, as they can result in detentions, arrests, and other infringements on people's freedoms.

The number of participants show this problem can't be solved simply by switching service providers. It's everywhere and it doesn't appear to be improving. The DHS wants to subject all travelers in international airports to this tech as soon as possible, which means we'll be seeing the collateral damage soon enough. A few lawmakers want to slow down deployment, but they remain a minority, surrounded by far too many legislators who feel US citizens should beta test facial recognition tech in a live environment with real-world consequences.

Filed Under: bias, facial recognition, minorities, nist

NYPD Chief Says Ending Stop And Frisk Will 'Hurt Minorities,' Result In A Surge Of Criminal Activity

from the wrong-on-both-counts dept

We've all heard Bloomberg's defense of the NYPD's officially unconstitutional stop and frisk program. Basically, the mayor claims that without being able to harass thousands of minorities a year (88% of whom are never charged, detained or given a summons), the city will immediately fall prey to a crimewave of epic proportions.

In fact, the hint of any sort of regulation or oversight is also greeted by the mayor's proclamations that, if implemented, the streets will run red with the blood of the innocent (also: cops). According to Bloomberg, any additional layer of oversight will so severely impair the decision-making skills of his police force, they'll be unable to respond to a pulled gun with anything other than dying.

So, now we'll have another layer of monitors... You're a police officer. You have to know what orders to follow. If somebody pulls a gun and you WANT TO GET HOME, you don't have time to say, "Well, now wait a second, the commissioner said one thing, the monitor said another and the IG said another." By that time, you're dead.

Fortunately, Bloomberg will be leaving office soon and won't have to bear the burden of a "bunch of people dying" (his words) as a result of this recent court decision. On the other hand, Police Chief Ray Kelly may be around long enough to see his stop and frisk program hampered by oversight and the Constitution, provided he doesn't bolt for the Department of Homeland Security beforehand.

Faced with this still-under-appeal reality, Kelly has been making the rounds to various talk shows, offering his defense of the stop and frisk program. His wording suggests he and Bloomberg have been reading over each other's shoulders.

"The losers in this, if this case is allowed to stand, are people who live in minority communities," he said on CBS' Face the Nation. He noted that 97% of shooting victims are black or Hispanic, reasoned that similar demographics apply if a stop deters a killing and added that there have been more than 7,300 fewer killings in the 11 full years of Mayor Michael Bloomberg's tenure so far than in the 11 years before.

I'd like to hear from some minorities if that's OK with you, Kelly. I'd like to see if they're willing to "trade off" a little bit of theoretical safety for the freedom to walk around without being judged as "furtive" or "suspicious" or "not white." You'll have to forgive me for not taking your word for it.

Kelly states crime stats as if that excuses the violations committed by his officers. It doesn't. Judge Scheindlin refused to consider the lowered crime rates as justification for constitutional violations. We've all heard the phrase, "the end justifies the means," and it's never used in reference to appropriate "means." But Kelly and Bloomberg do this all the time in reference to stop and frisk. "Our crime rate is historically low, therefore we'll do whatever we want to maintain it." Not acceptable.

Once the stats have been presented, it's time to switch to Argument B -- the violent crime spectre.

If stop and frisk were abandoned, "no question about it —violent crime will go up," he said on NBC's Meet the Press.

This is one of stop and frisk defenders' favorite claims, one that plays on irrational fear. Irrational fear is one of the government's biggest allies, whether it's the DHS or a local police force. Claiming bad things will happen keeps bad laws and policies in place. As long as the laws and policies stay on the books and "bad things" fail to happen, officials are constantly "proven" right.

Citing these irrational fears also prevents these laws and policies from being struck down. There are very few government officials willing to roll back anything out of fear that if something bad does happen, the blame will rest squarely on those who removed the so-called safeguard.

Kelly's repetition of this argument is the most self-serving of rhetoric, seeing as it's almost completely failsafe. Should there be an uptick in criminal activity post-stop and frisk, the supporters will point out that they warned this would happen. It doesn't even have to be a large or sustained uptick. All it takes is a couple of random acts of violence to trigger the "I told you so" contingent.

It's a win-win argument, even at this point. As the appeals process rolls on, crime level fluctuations will be watched and deployed by stop and frisk supporters. If crime stays at its present rate or slides lower, it will be held up as evidence that unconstitutional searches work. If it heads up, Kelly, Bloomberg et al will claim the "uncertainty" surrounding the court's decision is impairing police work.

Filed Under: minorities, nypd, ray kelly, stop and frisk, up is down