India's Supreme Court Opens Investigation Into Targeting Of Indian Citizens' Phones By NSO Malware

from the is-it-too-late-to-rebrand-as-'International-Pariah' dept

NSO Group's terrible 2021 is flowing seamlessly into an equally terrible 2022. The leak of a list of alleged targets for its malware -- a list that included journalists, activists, government critics, political officials, and religious leaders -- led to an outpouring of discoveries linking the company to abusive deployments of malware by a number of questionable governments.

NSO is currently being sued by two US companies over its malware. Facebook and WhatsApp claim NSO committed terms of service violations by sending malware via the messaging service. Apple claimed the same thing, pointing to the targeting of iPhones owned by users infected with NSO spyware.

Both companies are notifying users who appear to have been targeted by this malware. All over the world, people are reporting they've been targeted, often due to investigations performed by Canada's Citizen Lab and Amnesty International.

Governments are getting into the act as well. The Israeli government -- which once helped NSO broker deals with nearby authoritarians -- is investigating the company. It has also drastically slashed the number of foreign governments it can sell to. Other governments around the world are engaging in their own investigations following reports of residents (or their elected representatives) having been hit with malware payloads created by NSO.

NSO-related phone infections are now part of a federal case in India. The nation's top court has created a committee to look into allegations Indian citizens have been targeted by NSO's Pegasus spyware.

The Supreme Court-appointed Technical Committee looking into the usage of Pegasus against Indian citizens has issued a public notice asking those who believe they have been targeted using the spyware to come forward and say whether they would be willing to let their device be examined by the committee.

The public notice, published in newspapers across the country on 2 January, requests "any citizen of India who has reasonable cause to suspect that her/his mobile has been compromised due to specific usage of NSO grow Israel's Pegasus software (sic)" to contact the committee.

Those who suspect they've been targeted will turn their phones over to the technical committee for examination. They'll receive an image file of the contents of their phone after relinquishing their phones and receive their device back after it has been forensically examined.

This response was prompted by a lawsuit brought against the Indian government for spying on its own citizens using NSO malware. The court also wants the government to answer a few questions as it moves this litigation forward. It wants to know how the malware was used (interception, eavesdropping, etc.), which government entities have access to Pegasus, and whether or not it has been used to target Indian citizens.

Some of those answers will likely be answered by the examination of submitted phones. The others may never receive direct answers -- not if the government chooses to invoke national security mantras rather than discuss its purchase and use of NSO spyware in open court.

So far, the government has chosen to say nothing about alleged targeting of its own constituents, which hasn't made the Supreme Court very happy.

The bench headed by Chief Justice of India NV Ramana criticised the Union government for its refusal to clarify whether it had purchased and used the spyware, and said it had to accept the prima facie case of the petitioners, including victims of Pegasus hacking, and examine their allegations.

The government will be forced to respond. Forensic examinations will uncover malware infections and perhaps even the source of those infections. Refusing to respond to questions now just means answering harder questions later. And it's just more of the same for NSO Group, which is now primarily known for being the enabler of government corruption and oppression.

Filed Under: india, malware, pegasus, spyware, surveillance
Companies: nso group

Sci-Hub's Creator Thinks Academic Publishers, Not Her Site, Are The Real Threat To Science, And Says: 'Any Law Against Knowledge Is Fundamentally Unjust'

from the put-up-or-shut-up dept

A year ago, Techdirt wrote about an important lawsuit in India, brought by the academic publishers Elsevier, Wiley, and the American Chemical Society against Sci-Hub and the similar Libgen. A couple of factors make this particular legal action different from previous attempts to shut down these sites. First, an Indian court ruled in 2016 that photocopying textbooks for educational purposes is fair use; the parallels with SciHub, which provides free access to copies of academic papers for students and researchers who might not otherwise be able to afford the high subscription fees, are clear. Secondly, the person behind Sci-Hub, Alexandra Elbakyan, is fighting, rather than ignoring, the case, as she has done on previous occasions.

One manifestation of her new pro-active approach is a tweet she posted recently. It included a screenshot of an email she wrote to Nature magazine, which had contacted her about a forthcoming article on the Indian court case. Following standard practice, the journalist writing the article, Holly Else, asked Elbakyan to comment on some of the accusations the academic publishers had made against Sci-Hub. Her responses are fascinating, not least because they provide Elbakyan's perspective on several important issues.

For example, according to the publishers' comments as transmitted by Else, "Pirate sites like Sci-Hub threaten the integrity of the scientific record, and the safety of university and personal data". In reply, Elbakyan points out Sci-Hub is unique, and the use of the phrase "Pirate sites like Sci-Hub" is a clever attempt to lump Sci-Hub in with quite different sites, thus prejudging the legality of its activities. Elbakyan says that it's academic publishers -- not Sci-Hub -- which threaten the progress of science:

open communication is [a] fundamental property of science and it makes scientific progress possible. Paywalled access prevents this and is a great threat to science. Also the great threat is also when the whole scientific knowledge became the private property of some corporation such as Elsevier, that has full control of it. That is the threat, not Sci-Hub.

Elbakyan points out that Sci-Hub doesn't threaten the "integrity of the scientific record", since she simply disseminates copies of the academic papers without changing them in any way. But perhaps the most interesting part of her reply concerns the accusation that Sci-Hub threatens the safety of university and personal data. Techdirt has written previously about claims that Elbakyan allegedly has links to Russian intelligence, and that Sci-Hub is some kind of security risk. According to Else, the publishers assert:

Pirate sites like Sci-Hub compromise the security of libraries and higher education institutions to gain unauthorized access to scientific databases and other proprietary intellectual property, and illegally harvest journal articles and e-books.

Sci-Hub uses stolen user credentials and phishing attack to extract copyrighted articles illegally

These are serious allegations, and ones that have been made several times in the past. Elbakyan's response is probably the first time that she has addressed them directly:

Do they have any actual case when Sci-Hub somehow compromised the security of any library or a person? Any person that complained about credentials that were 'stolen' from them? Or is it again, nothing more than empty accusations. Nobody is complaining about 'compromised security' except academic publishers.

In other words, it is time for Elbakyan's accusers to put up or shut up. She concludes by stating that "Any law against knowledge is fundamentally unjust", and hopes that "Nature will have enough honesty to publish my comments in full.

It didn't, of course.

Follow me @glynmoody on Twitter, Diaspora, or Mastodon.

Filed Under: academic journals, academic research, alexandra elbakyan, copyright, education, holly else, india, publishers, security
Companies: nature, sci-hub

Indian Gov't Orders YouTube To Block 20 Channels For 'Blasphemy' And 'Impinging On National Security'

from the wherein-Piss-Christ-is-the-same-thing-as-a-pipe-bomb dept

India's Information Technology Act has been problematic since its inception. Almost a decade ago, it was deployed to justify the arrest of an Indian citizen who'd done nothing more than criticize a politician, insinuating the politician had used his position to amass personal wealth.

Once the politician complained, the government ignored available libel law, which would have resulted in a civil case and opted to use a clause of the IT Act which allowed prosecution for "sending an email or other electronic message that causes annoyance or inconvenience." That's how the law was used in 2012.

The 2021 version of the law is much more dangerous. Folding into the act passed in 2000 are new rules about national security. This escalates the Act's abusability, since nearly anything can be called a threat to national security and drastically limits the defenses those targeted by this part of the act can raise.

Added into this revision are new mandates for social media companies, which allows the government to hold companies responsible for content created by their users. There's a long list of forbidden content that starts with child sexual abuse material and terroristic content and ends with things like "promoting tobacco use," "blasphemy," and "harms minors in any way."

It is those changes that have resulted in the banning of 20 supposedly "anti-India" YouTube channels.

India banned 20 YouTube channels and two websites on Monday for allegedly running anti-India propaganda from Pakistan, invoking for the first time the emergency powers under the newly notified intermediary guidelines and digital media ethics code in the IT Act.

I&B secretary Apurva Chandra wrote to YouTube and the Department of Telecom, directing them to immediately block the content as it affects the sovereignty and integrity of India, people aware of the development told ET on condition of anonymity.

According to officials, these channels were being run by a Pakistan government agency. The officials also made reference to fake news and reporting on farmers' protests. The first is another term as abusable as "national security." The latter is something the Indian government has expended a lot of effort to control the narrative, often through censorship.

This is censorship as well. Even if we accept the Indian government's assertions about the origin and intent of the banned channels, the government can't even decide why it's doing this. It claims two violations -- neither of which should have anything to do with each other.

The inquiry revealed that these websites were being run from Pakistan. The content run on these channels is blasphemous and hugely impinges on national security," said the official, who was part of the review.

If your national security and national religion are intertwined, a law addressing both is a censorial cudgel. And it won't make anyone safer or make your religion any more unassailable. What it will do is allow the government to disappear information and content it doesn't like with impunity.

This has been the point of the law all along. Every iteration gives the government more power and takes it away from the country's citizens and the services they use. It elevates the government, placing it on the same level as its preferred deities. When a government does that, it no longer serves the public. The public is now expected to serve (and revere) the government.

The Indian government has all the tools it needs to ensure its narrative is the most prominent. Even before these amendments affecting intermediary liability went into effect, the government had pulled the plug on the internet in areas where farmers' protests were taking place and threatened Twitter employees with arrest for restoring accounts and content the government had ordered removed. More assertions about god(s) and country are undoubtedly on their way, forcing those carrying third party content to give the Indian government control of the narrative or face fines, fees, and possible imprisonment.

Filed Under: blasphemy, censorship, content moderation, free speech, india, information technology act, national security, pakistan
Companies: youtube

It's Ridiculous The 'Developing World' Wasn't Given Access To The Facebook Files

from the do-this-the-right-way dept

By now it's fairly clear the Facebook leaks showcase a company that prioritized near-mindless international growth over the warnings of their own experts. They also show a company that continues to painfully struggle to be even marginally competent at scale, whether we're talking about content moderation or rudimentary customer service. While this has become an all-encompassing media spectacle, the real underlying story isn't particularly unique. It's just a "growth for growth's sake" mindset, where profit and expansion trumped all reason. It just happens to be online, and at unprecedented international scale.

One thing I haven't seen talked a lot about is the fact that if you look back a few years, an awful lot of of folks in developing nations saw these problems coming a mile away, long before their Western counterparts. For a decade, international activists warned repeatedly about the perils of Facebook's total failure to understand the culture/regulations/language/norms of the countries they rapidly flooded into. Yet bizarrely, Frances Haugen's PR team somehow excluded most of these countries when it came time to recently release access to the Facebook files:

The "consortium" handling release of the files is basically a loose collaboration between 17 U.S. news orgs and a handful of European outlets. They're all being given access to redacted versions of documents Haugen provided the Securities and Exchange Commission, showing Facebook repeatedly prioritized growth and profit over, well, everything else. The whole thing was handled by Haugen's PR team via an ordinary embargo, which some oddly saw as itself somehow nefarious (it's not, embargoes, though often kind of stupid, are commonly used to maximize impact).

The real problem was who was included in that consortium. And a bigger problem, oddly not talked about until a month into the Facebook leak news cycle, was that much of the developed world was just... excluded... from the coalition by her PR reps. The exclusion of academics and researchers that could make the most sense of the data was a problem. But restricting analysis to most white, western newsrooms, (despite Haugen's very clear understanding that most of Facebook's impact problems disproportionately harmed developing nations) is particularly odd and tasteless.

For all the problems Facebook (sorry, Meta) has had in the United States in regards to managing the company's platform at scale, those problems have been dramatically worse internationally. Facebook was so excited to flood into dozens of international locations to wow investors, they didn't dedicate the time, resources, or attention needed to actually understand what they were doing. Even if they had (and the whistleblowers keep showing they absolutely didn't), the sheer scale of the expansion made it impossible to do well. That Facebook did so anyway despite being warned about it is an act of greed and hubris.

It was actually a net neutrality debate that keyed many overseas activists into Facebook's problems more than a decade ago. Activists in India were particularly sensitive to Facebook's attempts to conflate "the internet" with Facebook in developing nations. If you recall, activists in India successfully derailed Facebook's Free Basics program, which was Facebook's attempt to corner developing nation ad markets under the banner of altruism.

Basically, it involved Facebook striking deals with local wireless companies to offer discounted access to Facebook, under claims that aggressively curated version of "online" was better than no online access at all. It was a highly curated walled garden bastardized variation of AOL or CompuServe, in which Facebook decided what information, services, and resources mattered (they initially even banned encrypted services). But activists and international experts were quick to see the problem with giving Facebook this kind of power, especially in countries they didn't take the time to understand.

We've seen repeatedly how conflating "Facebook" with "the internet" (or Whatsapp with "the internet") has created a laundry list of problems that are especially pronounced in developing nations. The centralized approach of programs like Free Basics defeated the purpose of the open internet, reduced transparency, was a big boon for authoritarian governments, and helped create an even more potent funnel for propaganda. One recurring theme in whistleblower accounts is that Facebook's own researchers generally warned about all of this, repeatedly, but were ignored for profit and growth's sake.

As early as 2015 organizations like Mozilla were busy arguing that if Facebook genuinely cared about information access in developing nations, they should simply fund access to the internet itself. Facebook was ultimately forced to back off its plan in some countries like India and Egypt, but if you were a reporter or activist in these countries who pointed out the problems with Facebook's ambition, you were accused of being an enemy of the poor. When the Free Basics brand became toxic, Facebook just named Free Basics something else (sound familiar?).

There has been some valid and not so valid criticism of the way Haugen handled these latest revelations. Some have tried to argue that because she was smart enough to hire lawyers and a PR team to maximize impact she can't possibly be technically seen as a "whistleblower." There was also some brief hyperventilation over a Politico report, with some trying to claim that because she had received some money from investor Pierre Omidyar, she shouldn't be taken seriously. But a shrewd, organized whistleblower is still a whistleblower, and Omidyar proxy groups actually just funded whistleblower orgs Haugen was part of after she went public. It's actually a good thing to see a whistleblower do the right thing and not be economically and reputationally devastated for once.

But it's both weird and telling that people freaked out about these perceived injustices, but didn't notice that the whistleblower's PR coalition apparently just forgot the developing world existed:

Nobody I've talked to so far at news organizations seems clear why this seems to have happened (a strange decision for an effort geared toward greater transparency). It's not like it would be particularly difficult to coordinate the release via the same organizations in places like India that warned about Facebook's consolidated power almost a decade ago (see: IFEX). Some outlets, like Gizmodo, have been trying to expand access to the source documents to everyone. That's apparently to the chagrin of Haugen's PR team, who seems to think they can put the genie back in the bottle.

There was a certain hubris in Facebook stumbling its way across the developing world in a quest for growth without bothering to understand the impact their platform would have on foreign cultures. But there's a fairly substantial amount of hubris in excluding these developing nations from accessing raw data on a problem they've disproportionally been harmed by.

Filed Under: bias, developing nations, facebook files, facebook papers, frances haugen, india, journalists
Companies: facebook, meta

American Malware Purveyor That Did Nothing To Limit Misuse Now Horrified To Find Gov't Of India Misused Its Products

from the who-could-possibly-have-seen-this-inevitable-outcome dept

Another malware purveyor is shocked, SHOCKED to discover its products have been used to do Very Bad Things. Thomas Brewster has more details for Forbes. Here's the setup:

Earlier this year, researchers at Russian cybersecurity firm Kaspersky witnessed a cyberespionage campaign targeting Microsoft Windows PCs at government and telecom entities in China and Pakistan. They began in June 2020 and continued through to April 2021. What piqued the researchers’ interest was the hacking software used by the digital spies, whom Kaspersky had dubbed Bitter APT, a pseudonym for an unspecified government agency. Aspects of the code looked like some the Moscow antivirus provider had previously seen and attributed to a company it gave the cryptonym “Moses.”

More digging by Kaspersky and others discovered who was actually behind these deployments. And the source wasn't some state-supported hackers or a malware purveyor with a malleable set of morals. No, the exploits -- which were deployed to indiscriminately target people in Pakistan and China -- were sold (in a way) to the government of India by an American firm, Exodus Intelligence.

Operating out of Austin, Texas, Exodus doesn't craft many exploits of its own, but rather provides access to information about known exploits, including where to obtain them, and how they can be utilized and leveraged.

Exodus, when asked by Five Eyes countries (an alliance of intelligence-sharing countries that includes the U.S., U.K., Canada, Australia, and New Zealand) or their allies, will provide both information on a zero-day vulnerability and the software required to exploit it. But its main product is akin to a Facebook news feed of software vulnerabilities, sans exploits, for up to $250,000 a year. It’s marketed primarily as a tool for defenders, but customers can do what they want with the information on those Exodus zero days—ones that typically cover the most popular operating systems, from Windows to Google’s Android and Apple’s iOS.

The government of India chose to leverage this knowledge to indiscriminately assault China and Pakistan entities in hopes of hitting targets of interest. That wasn't what Exodus Intelligence's info feed was designed to do. It's only what it ended up being used for. And now the CEO of Exodus is acting like a parent disappointed a child has exceeded the boundaries he never bothered to set.

That feed is what India bought and likely weaponized, says 37-year-old Exodus CEO and cofounder Logan Brown. He tells Forbes that, after an investigation, he believes India handpicked one of the Windows vulnerabilities from the feed—allowing deep access to Microsoft’s operating system—and Indian government personnel or a contractor adapted it for malicious means. India was subsequently cut off from buying new zero-day research from his company in April, says Brown, and it has worked with Microsoft to patch the vulnerabilities. The Indian use of his company’s research was beyond the pale, though Exodus doesn’t limit what customers do with its findings, Brown says, adding, “You can use it offensively if you want, but not if you’re going to be . . . shotgun blasting Pakistan and China. I don't want any part of that.”

While it's great the CEO doesn't want any part of that, not placing limits on end users is always going to result in things like this. And while it's unlikely writing up a new ToS is going to deter customers from "shotgun blasting" people with the weaponry you've provided, it at least allows you to terminate contracts and access without having to engage in a bunch of costly litigation or fruitless negotiations.

And, if you're going to be in the business of selling exploits (or indirect access to exploits), you need to be way more proactive on the security front.

Brown is also now exploring whether or not its code has been leaked or abused by others. Beyond the two zero days already abused, according to Kaspersky, “at least six vulnerabilities” made by Moses have made it out “into the wild” in the last two years.

Whoops. That doesn't look good. But, in all fairness, even the NSA and CIA have seen their tech tools and exploits leaked, resulting in the infliction of misery worldwide by people a shade more malicious than the entities belatedly bemoaning the unplanned distribution of their digital secrets.

Speaking of belated, here's some regret from the cofounder of Exodus Intelligence, Aaron Portnoy.

[T]oday, the 36-year-old self-taught hacker, who dropped out of Northwestern to carve his own career in cybersecurity, worries that he never knew who had access to his code or how they used it. He now regrets relinquishing control over his zero days to salespeople. “It's almost like I was being taken advantage of . . . It felt very much like I was a tool that was being used for a bigger purpose that I really had no insight into,” says Portnoy, now plying his trade at Randori, a Massachusetts-based cybersecurity firm.

Sure, but not so concerning Portnoy didn't leap from Exodus to defense contract Raytheon, and from there to startup Boldend, which partnered with Raytheon to (and I'm directly quoting here) "accelerate cyber operations with greater force."

While it's great that Exodus has revoked the Indian government's access to its exploit feed, the larger problem remains. American companies are aiding and abetting mass surveillance, targeting of dissidents and activists, and other human rights abuses by not being more selective of who they sell to or placing limits on how their products are used. This puts them in the same shady neighborhood as overseas malware merchants like NSO Group and Hacking Team. Sooner or later, it's going to put them on the wrong end of UN sanctions or DOJ investigations. Until then, it appears it will be risky business as usual, making the United States home to plenty of proxy human rights violators.

Filed Under: china, cybersecurity, india, malware, pakistan, security, zero days
Companies: exodus intelligence

WhatsApp Sues The Indian Government Over New Laws That Would Force It To Break Encryption

from the backdoors dept

For many years now, we've reported on efforts by the Indian government to demand that WhatsApp break its encryption to give the government access. Much of this comes from the fact that the Indian government wants to pin the blame for certain violence and disinformation on WhatsApp, rather than on those actually responsible. WhatsApp has, in the past, pushed back on individual demands to break its encryption.

However, things have stepped up a notch. The Indian government recently put in place new regulations that are, to put it mildly, quite troubling. India has framed the laws -- like so many other laws like this around the globe -- as being about stopping "abuse." But, of course, the government gets to decide what is abuse.

The Indian government says the rules are designed to prevent “abuse and misuse” of social media. “The new rules instead seek to empower the users of social media by requiring these platforms to put in place a robust public grievance redressal mechanism,” a representative of India’s Ministry of Electronics and Information Technology said in a March 17 letter to TIME. “It brings digital media outlets at par with print and electronic media with the underlying principle of self-regulation to ensure compliance to existing laws of India.”

And one thing that the Indian government believes is abuse... is encryption (remember, again, those earlier stories of the government blaming WhatsApp for violence). Exactly what the rules mean for encryption has been a bit... unclear. The rules do require that encrypted services keep certain information about who the "first originator" of a message is and to provide that to the government on demand. The Indian government insists this is not about breaking encryption. It claims they only need certain metadata and only in extreme circumstances. The government also claims that WhatsApp did not object to this proposal when it was first put forth and there was an open consultation period.

WhatsApp (owned by Facebook) claims that's completely bogus, because in order to find the "first originator" of a message you have to track everything:

“Traceability” is intended to do the opposite by requiring private messaging services like WhatsApp to keep track of who-said-what and who-shared-what for billions of messages sent every day. Traceability requires messaging services to store information that can be used to ascertain the content of people’s messages, thereby breaking the very guarantees that end-to-end encryption provides. In order to trace even one message, services would have to trace every message.

That’s because there is no way to predict which message a government would want to investigate in the future. In doing so, a government that chooses to mandate traceability is effectively mandating a new form of mass surveillance. To comply, messaging services would have to keep giant databases of every message you send, or add a permanent identity stamp -- like a fingerprint -- to private messages with friends, family, colleagues, doctors, and businesses. Companies would be collecting more information about their users at a time when people want companies to have less information about them.

And now, WhatsApp is suing the Indian government, claiming that the law violates India's privacy rights.

Of course, this is also against the backdrop of even bigger conflicts between social media platforms and the government of Narendra Modi, including threatening to jail Twitter employees for not taking down criticism of Modi's policies, and then raiding Twitter's local offices for daring to fact check a tweet by a government spokesperson.

Tragically, these attacks on social media moderation and encryption are becoming a toolkit of angry regulators around the globe, as they seem perpetually put out by the fact that the public actually has a voice. Whatever you might think of Facebook, this is a scenario where you should be rooting for WhatsApp to win this case to protect basic privacy rights.

Filed Under: encryption, going dark, india, private messaging, traceability
Companies: facebook, whatsapp

Oversight Board Tells Facebook It Needs To Shape Up And Be More Careful About Silencing Minorities Seeking To Criticize The Powerful

from the pay-attention-to-this dept

Tomorrow, the Oversight Board is set to reveal its opinion on whether Facebook made the right decision in banning former President Trump. And that will get tons of attention. But the Board came out with an interesting decision last week regarding a content takedown in India, that got almost no attention at all.

Just last week, we wrote about an ongoing issue in India, where the government of Prime Minister Narendra Modi has failed in almost every way possible in dealing with the COVID pandemic, but has decided the best thing to focus on right now is silencing critics on Twitter. That backdrop is pretty important considering that the very next day, the Oversight Board scolded Facebook for taking down content criticizing Modi's government.

That takedown was somewhat different and the context was very different. Also, it should be noted that as soon as the Oversight Board agreed to take the case, Facebook admitted it had made a mistake and reinstated the content. However, this case demonstrates something important that often gets lost in all of the evidence free hand-wringing about "anti-conservative bias" from people who wrongly insist that Facebook and Twitter only moderate the accounts of their friends. The truth is that content all across the board gets moderated -- and often the impact is strongest on the least powerful groups. But, of course, part of their lack of power is that they're unable to rush onto Fox News and whine about how they're being "censored."

The details here are worth understanding, not because there was some difficult decision to make. Indeed, as noted already, Facebook realized it made a mistake almost immediately after the Oversight Board decided to look into this, and when asked why the content was taken down, basically admitted that it had no idea and that it was a complete and total mistake. Here was the content, as described by the Oversight Board ruling:

The content touched on allegations of discrimination against minorities and silencing of the opposition in India by “Rashtriya Swayamsevak Sangh” (RSS) and the Bharatiya Janata Party (BJP). RSS is a Hindu nationalist organization that has allegedly been involved in violence against religious minorities in India. “BJP” is India’s ruling party to which the current Indian Prime Minister Narendra Modi belongs, and has close ties with RSS.

In November 2020, a user shared a video post from Punjabi-language online media Global Punjab TV and an accompanying text. The post featured a 17-minute interview with Professor Manjit Singh, described as “a social activist and supporter of the Punjabi culture.” In its post, Global Punjab TV included the caption “RSS is the new threat. Ram Naam Satya Hai. The BJP moved towards extremism.” The media company also included an additional description “New Threat. Ram Naam Satya Hai! The BJP has moved towards extremism. Scholars directly challenge Modi!” The content was posted during India’s mass farmer protests and briefly touched on the reasons behind the protests and praised them.

The user added accompanying text when sharing Global Punjab TV’s post in which they stated that the CIA designated the RSS a “fanatic Hindu terrorist organization” and that Indian Prime Minister Narendra Modi was once its president. The user wrote that the RSS was threatening to kill Sikhs, a minority religious group in India, and to repeat the “deadly saga” of 1984 when Hindu mobs attacked Sikhs. They stated that “The RSS used the Death Phrase ‘Ram naam sat hai’.” The Board understands the phrase "Ram Naam Satya Hai" to be a funeral chant that has allegedly been used as a threat by some Hindu nationalists. The user alleged that Prime Minister Modi himself is formulating the threat of “Genocide of the Sikhs” on advice of the RSS President, Mohan Bhagwat. The accompanying text ends with a claim that Sikhs in India should be on high alert and that Sikh regiments in the army have warned Prime Minister Modi of their willingness to die to protect the Sikh farmers and their land in Punjab.

The post was up for 14 days and viewed fewer than 500 times before it was reported by another user for “terrorism.” A human reviewer determined that the post violated the Community Standard on Dangerous Individuals and Organizations and took down the content, which also triggered an automatic restriction on the use of the account for a fixed period of time. In its notification to the user, Facebook noted that its decision was final and could not be reviewed due to a temporary reduction in its review capacity due to COVID-19. For this reason, the user appealed to the Oversight Board.

So, you had an ethnic minority -- one who had been attacked in the past -- warning about those currently in power. And Facebook took it down, refused to review the appeal... until the Oversight Board turned its eye on it, and then admitted it was a mistake, and basically threw its hands in the air and said it had no idea why it had been taken down in the first place.

According to Facebook, following a single report against the post, the person who reviewed the content wrongly found a violation of the of the Dangerous Individuals and Organizations Community Standard. Facebook informed the Board that the user’s post included no reference to individuals or organizations designated as dangerous. It followed that the post contained no violating praise.

Facebook explained that the error was due to the length of the video (17 minutes), the number of speakers (two), the complexity of the content, and its claims about various political groups. The company added that content reviewers look at thousands of pieces of content every day and mistakes happen during that process. Due to the volume of content, Facebook stated that content reviewers are not always able to watch videos in full. Facebook was unable to specify the part of the content the reviewer found to violate the company’s rules.

Got that? Facebook is basically saying "yeah, it was a mistake, but that was because it was a long video, and we just had one person reviewing who probably didn't watch the whole video."

Here's the thing that the "oh no, Facebook is censoring people" don't get. This happens all the time. And none of us hear about it because the people it happens to often are unable to make themselves heard. They don't get to run to Fox News or Parler or some other place and yell and scream. And, this kind of "accidental" moderation especially happens to the marginalized. Reviewers may not fully understand what's going on, or not really understand the overall context, and may take the "report" claim at face value, rather than having the ability or time to fully investigate.

In the end, the Oversight Board told Facebook to put back the content, which was a no-brainer since Facebook had already done so. However, more interesting were its policy recommendations (which, again, are not binding on Facebook, but which the company promises to respond to). Here, the Oversight Board said that Facebook should make its community standards much more accessible and understandable, including translating the rules into more languages.

However, the more interesting bit was that it said that Facebook "should restore human review and access to a human appeals process to pre-pandemic levels as soon as possible while fully protecting the health of Facebook’s staff and contractors." There were some concerns, early in the pandemic, about how well content moderation teams could work from home, since a lot of that job involves looking at fairly sensitive material. So, there may be reasons this is not really doable just yet.

Still, this case demonstrates a key point that we've tried to raise about the impossibility of doing content moderation at scale. So much of it is not about biases, or incompetence, or bad policies, or not wanting to do what's right. A hell of a lot of it is just... when you're trying to keep a website used by half the world operating, mistakes are going to be made.

Filed Under: appeals, content moderation, free speech, india, minorities, mistakes, review, takedowns
Companies: facebook, oversight board