A Fight Between Facebook And The British Medical Journal Highlights The Difficulty Of Moderating 'Medical Misinformation'

from the what-a-disaster dept

There are multiple efforts under way in the US to pass laws that require social media sites to take down "medical misinformation." As we've described repeatedly, these are really dangerous ideas. Bills like those from Senators Amy Klobuchar and Ben Ray Lujan seek to force social media to remove medical misinformation as declared by the Ministry of Truth... er... Secretary of Health & Human Services. Of course, it was not all that long ago that we had an administration that was actively anti-science, and wanted to declare anything that made the president look bad as "fake news."

Also, in the midst of a pandemic, when the data and the science are rapidly evolving, what might seem reasonable at one point, may later turn out to be misinformation -- and vice versa. Forcing down misinformation leads to all sorts of dangerous consequences. Hell, we saw this in China, where such a law was used to silence a doctor who tried to raise the alarm about COVID-19, and was forced to apologize for spreading "untruthful information online."

But there's another aspect of this which people rarely try to deal with: content moderation involves a lot of very gray areas and an awful lot of context, much of which may not be immediately obvious. An ongoing war of words between the former British Medical Journal (now just "The BMJ") and Meta/Facebook demonstrates nicely just how impossible it is to claim that "medical misinformation" must be taken offline. There's a bit of background here, and it's a, well, touchy subject, so try to go through the whole thing before you react.

First off, the BMJ is not, in any way, anti-vaccine. Somewhat famously, the BMJ was a key player in exposing the fraudulent behavior of Dr. Andrew Wakefield, whose fraudulent study created the modern anti-vax movement. That said, in November, The BMJ published an investigative journalism piece, based on a supposed "whistleblower" suggesting that there was some data integrity issues with the way Pfizer's vaccine was tested, specifically involving a research partner of Pfizer, Ventavia Research Group.

Ventavia responded to the allegations by noting that the supposed whistleblower in question had raised the issues a year earlier, and they were investigated and found to be unsubstantiated. That said, many reasonable people noted that this should be further investigated and worried that it might lead to further damaging the public's trust in science.

But, of course, you can fully predict what happened next. It didn't just "damage the public's trust in science," the BMJ article instead was instantly championed by all of the big anti-vax voices all over social media as "proof" that the COVID vaccine was dangerous and rushed into approval -- key talking points among that crowd, repeated despite tons of evidence that the vaccine is both incredibly effective and incredibly safe.

This resulted in Lead Stories, a fact checking organization, to fact check the article, and slap it with a "missing context" label, and calling into question the way that people were interpreting the article:

Did the British Medical Association's news blog reveal flaws that disqualify the results of a contractor's field testing of Pfizer's COVID-19 vaccine, and were the problems ignored by the Food & Drug Administration and by Pfizer? No, that's not true: Pfizer and the FDA were made aware of the allegations about the contractor in 2020. Medical experts say the claims aren't serious enough to discredit data from the clinical trials, which is also what Pfizer and the FDA say they concluded. The FDA says its position is unchanged: The benefits of the Pfizer vaccine far outweigh rare side effects and the clinical trial data are solid.

Because of this fact check and because of the way the article was being used in a misleading way by thousands of anti-vaxxers, users who tried to share The BMJ article were flagged with fact check warnings saying: "Missing context ... Independent fact-checkers say this information could mislead people," which is accurate, but incomplete, and very dependent on the context of who was sharing it and for what purpose.

The BMJ kinda flipped out about this and published an angry open letter to Mark Zuckerberg (who, I assure you, had nothing to do with the decision on the fact check and flagging). To be honest, I find the BMJ's anger here completely disingenuous. They act like they don't understand at all why Lead Stories highlighted the "missing context" point on their story, when -- of anyone -- the BMJ should be willing to acknowledge how their own article was being weaponized by ignorant anti-vaxxers.

But from November 10, readers began reporting a variety of problems when trying to share our article. Some reported being unable to share it. Many others reported having their posts flagged with a warning about “Missing context ... Independent fact-checkers say this information could mislead people.” Those trying to post the article were informed by Facebook that people who repeatedly share “false information” might have their posts moved lower in Facebook’s News Feed. Group administrators where the article was shared received messages from Facebook informing them that such posts were “partly false.”

Readers were directed to a “fact check” performed by a Facebook contractor named Lead Stories.[2]

We find the “fact check” performed by Lead Stories to be inaccurate, incompetent and irresponsible.

-- It fails to provide any assertions of fact that The BMJ article got wrong

-- It has a nonsensical title: “Fact Check: The British Medical Journal Did NOT Reveal Disqualifying And Ignored Reports Of Flaws In Pfizer COVID-19 Vaccine Trials”

-- The first paragraph inaccurately labels The BMJ a “news blog”

-- It contains a screenshot of our article with a stamp over it stating “Flaws Reviewed,” despite the Lead Stories article not identifying anything false or untrue in The BMJ article

-- It published the story on its website under a URL that contains the phrase “hoax-alert”

We have contacted Lead Stories, but they refuse to change anything about their article or actions that have led to Facebook flagging our article.

The BMJ open letter also gets unnecessarily snarky (which also seems out of character for a prestigious medical journal):

Rather than investing a proportion of Meta’s substantial profits to help ensure the accuracy of medical information shared through social media, you have apparently delegated responsibility to people incompetent in carrying out this crucial task.

That's ridiculous. Clearly this is a difficult situation. Even if the reporting was accurate -- there is crucial context here. Did the revelations support the claims of anti-vaxxers who were using it as evidence that the Pfizer vaccine was not safe? The answer is no, it did not. And there's a strong argument that The BMJ could have and should have made that point a lot clearer in their own reporting, recognizing how the article would be weaponized by grifters and fed to the ignorant.

Lead Stories then responded to the BMJ, in fairly great detail, more or less saying "you can't honestly be that naïve."

It is ironic to read that BMJ.com objects to the headline on Lead Stories' fact check of a BMJ.com article when the original BMJ piece carries a scare headline that oversells the whistleblower and overstates the jeopardy. Their November 2, 2021, headline "Covid-19: Researcher blows the whistle on data integrity issues in Pfizer's vaccine trial" is the reason BMJ.com's article has appeared in hundreds of Facebook posts and tweets, many by anti-vaccine activists using it as "proof" the entire clinical trial was fraudulent and the vaccine unsafe.

Lead Stories also points out that The BMJ's headline to its article is extremely misleading, as it can be read to say that there were data integrity issues with the entirety of the Pfizer vaccine test, rather than 3 sites out of 153, and then also highlights that the whistleblower in question is not a scientist who is an expert on this. It also notes that the whistleblower appears to have some... questionable beliefs and associations regarding vaccines:

The BMJ.com article eventually gets around to saying she worked at the lab for just two weeks. But BMJ's open letter fails to mention important context: The Brook Jackson Twitter account agreed with leading COVID misinformation-spreader Robert F. Kennedy Jr.'s criticism of the "Sesame Street" episode in which Big Bird encourages kids to get a COVID-19 vaccine. "Shocking, actually." she wrote in a November 9, 2021, response to a Kennedy tweet blasting Sesame Street (archived here). Elsewhere on Twitter, the Brook Jackson account wrote to a vaccine-hesitant person that vaccination makes sense if a person is in a high-risk category. When the U.S. 5th Circuit Court of Appeals ruled against a federal employee vaccine mandate, she tweeted "HUGE!" and not with a frowny emoji.

Lead Stories talked to Jackson, looked at available documents (after BMJ refused to permit us to see their basis for the story and did not make the documents available on a transparency site). Unlike BMJ.com, Lead Stories then tested Jackson's assertions with Pfizer, with the lab contractor in question and with the FDA and then published their responses. It's not at all clear yet whether there are data integrity issues if you ask the other stakeholders, and that's the crucial missing context. We also talked to experienced medical researchers for perspective, one of whose credentials BMJ editorial staff demeaned for reasons we can only imagine.

By talking to Ventavia, we contributed context BMJ.com missed: Ventavia said the whistleblower had not worked on the Pfizer trial, but Lead Stories set that straight by embedding in its story a copy of a letter, provided by Jackson in which she was expressly welcomed to the Pfizer trial team. That's what we mean by context.

The BMJ has thus far failed to document what is "inaccurate" in the Lead Stories fact check, but again oversells by using that and other name-calling to vent frustration at our documentation of obvious missing context

All of this involves an awful lot of judgment calls, understanding of context, and a lot more. But under a law that requires the pulling down of medical misinformation, how the hell would anyone handle this kind of scenario? The BMJ story isn't wrong per se, but there is a lot of important context that seems like it's missing (which Lead Stories highlighted above). On top of that, there's all the important context around how people are using the article and stretching an already weaker-than-it-seems story to pretend to be a lot more damning on the overall vaccine.

In other words, how the article is being represented and used is an important piece of context as well. And this is frequently the case with medical misinformation. People will take something that is factual or accurate, and present it out of context or in a misleading light, in order to make an argument that doesn't support it. So which part is the "misinformation" and how do you police that?

In an ideal world, we'd be able to see all the details and the back and forth, and figure it all out. Frankly, when I first heard about this -- via The BMJ's open letter -- I initially thought that the details would support The BMJ, and that Facebook mislabeled something (which, of course, happens all the time because of the old Masnick Impossibility Theorem). It was only after reading multiple articles on both sides of this, and going through the details of Lead Stories' process, that I realized that they had (to me) a much stronger argument, that there's an awful lot of important context that is missing from The BMJ piece that you would hope a journal like that would have considered before publishing the article the way in which it did.

But to expect every social media platform to be able to determine this on every piece of medical sharing out there is next to impossible -- and putting legal liability on top of it, as Senators Klobuchar and Lujan want to do -- would be dangerously impossible.

Filed Under: content moderation, context, covid, fact checking, health, health misinformation, journalism, misinformation, reporting, vaccines
Companies: facebook, lead stories, meta, pfizer, the bmj

Oversight Board Overturning Instagram Takedown Of Ayahuasca Post Demonstrates The Impossibility Of Content Moderation

from the this-is-not-easy dept

Congress has been holding lots of "but think of the children online!" hearings over the past couple of months, and one prominent topic that comes up over and over again is the fact that people can find "drug" information online. Fears about kids and drugs goes back decades, but politicians love it, because it always works. And, of course, the media loves to run these overhyped stories. A quick search finds dozens of stories like the following in just the last month or so:

• Teens have easier access to drugs as illegal trade booms on social media
• Instagram pushes drug content to teens
• Social media platforms becoming sites of illegal fentanyl drug sales targeting teens, L.A. officials warn
• WARNING: Dealers tempt kids with edible drugs via social media
• Record number of drug overdose deaths brings new scrutiny to social media apps
• Instagram offers 'drug pipeline' to kids, tech advocacy group claims

There are many more such stories, but you get the idea. The Instagram stories, in particular, were targeted by the recent Senate hearing which was focused on allowing Senators to grandstand on how bad Instagram supposedly is for kids. Instagram responded to the report by pointing out that it's doing a ton of stuff to try to block such content:

"We removed 1.8 million pieces of content related to drug sales in the last quarter alone, and due to our improving detection technology, the prevalence of such content is about 0.05 percent of content viewed, or about 5 views per every 10,000. We'll continue to improve in this area in our ongoing efforts to keep Instagram safe, particularly for our youngest community members."

But, within a day of that report getting so much attention, you had the Oversight Board step in to demonstrate how much more difficult an issue this is than it sounds to grandstanding politicians or narrative-driving journalists. The Oversight Board overturned Instagram's decision to take down a post that was promoting ayahuasca as medicine. Instagram took down the post (which had just started trending) because it was promoting it as medical treatment:

In July 2021, an Instagram account for a spiritual school based in Brazil posted a picture of a dark brown liquid in a jar and two bottles, described as ayahuasca in the accompanying text in Portuguese. Ayahuasca is a plant-based brew with psychoactive properties that has deeply-rooted religious and ceremonial uses among Indigenous and other groups in some South American countries, and related communities elsewhere. Ayahuasca contains plants which are sources of dimethyltryptamine (DMT), a substance that is prohibited under Schedule I of the 1971 UN Convention on Psychotropic Substances and the law of many countries, though there are relevant exceptions under international law for substances containing DMT such as ayahuasca, and exceptions under some national laws, including in Brazil, for other uses such as religious and Indigenous use.

The text states that “AYAHUASCA IS FOR THOSE WHO HAVE THE COURAGE TO FACE THEMSELVES” and includes statements that ayahuasca is for those who want to “correct themselves,” “enlighten,” “overcome fear” and “break free.” It further states ayahuasca is a “remedy” and “can help you” if one has humility and respect. It ends with “Ayahuasca, Ayahuasca!/Gratitude, Queen of the Forest!”

The content was viewed over 15,500 times and no users reported it. The post was flagged for review by Meta’s automated systems because it had received around 4,000 views and was “trending.” Meta specified neither the image nor the text triggered automatic review. The post was subsequently reviewed by a human moderator and removed. Meta told the Board that it was removed for violating Facebook’s Community Standard on Regulated Goods, but later stated it removed the content for “violating the Instagram Community Guidelines, which include a link to the Facebook Community Standard on Regulated Goods.” Meta notified the user that the post went against Instagram’s Community Guidelines, stating “post removed for sale of illegal or regulated goods.” The messaging also noted that Meta removes “posts promoting the use of hard drugs.”

So, there are a few interesting tidbits there worth pointing out. Contrary to the fear-mongering and grandstanding, this notes that Instagram has a system to proactively flag this kind of content, and that some of the priority of such reviews is based on whether or not the content is getting a lot of attention and going viral (these are both things that many critics of Facebook/Instagram/Meta insist the company does not do). And, if you just listened to the Senators at hearings, or just read all those news articles above, you might not realize th.

But, the more important thing that this case showcases, is just how difficult it is to make these kinds of calls at scale. What seems obvious to some is not so obvious, and there's a lot of subjectivity involved. This can be seen in the way the two sides (the poster of the original content and the company) explained their thinking. The organization that made the original post about ayahuasca says that it was just an image of a regulated and legal ceremony:

The user stated in their appeal that they are certain the post does not violate Instagram’s Community Guidelines, as their page is informative and never encouraged or recommended the purchase or sale of any product prohibited by the Community Guidelines. They said that they took the photo at one of their ceremonies, which are regulated and legal. According to the user, the account aims to demystify the sacred ayahuasca drink. They said that there is a great lack of knowledge about ayahuasca. The user stated that it brings spiritual comfort to people and their ceremonies can improve societal wellbeing. They further state that they have posted the same content previously on their account and that post remains online.

Instagram, for its part, takes the more hardline "drugs bad, mmkay" line that Congress and the press seems to want them to take:

Meta stated that the content violated Facebook’s Community Standards because “the user described ayahuasca with a heart emoji, referred to it as ‘medicine,’ and stated that it ‘can help you.’” Following a question from the Board about whether the content was removed for violating Instagram’s Community Guidelines or Facebook’s Community Standards, Meta responded the content was removed “for violating the Instagram Community Guidelines, which include a link to the Facebook Community Standards on Regulated Goods.” Specifically, the user violated Instagram’s prohibition on content “buying or selling illegal or prescription drugs (even if legal in your region).” Meta also cited another line of the Community Guidelines which links to the Community Standard on Regulated Goods, which “clarifies that Facebook prohibits content that ‘[c]oordinates or promotes (by which we mean speaks positively about [...] non-medical drugs.’”

Referring to Meta’s values, the company stated that “Safety” displaced “Voice.” Meta noted that users are permitted to advocate for the legalization of non-medical drugs and to discuss the medical and scientific benefits of non-medical drugs, but that there is no religious or traditional use allowance. Meta argued that this rule strikes the correct balance between “Voice” and “Safety.”

Meta also stated that prohibiting this content follows human rights principles. It stated that it considered the right to freedom of expression under Article 19 ICCPR and the right to freedom of religion or belief under Article 18 ICCPR, and argued that its decision satisfied the conditions required to restrict these rights.

According to Meta, Facebook’s Community Standard is easily accessible and its non-public definition of non-medical drug as a “substance that causes ‘a marked change in consciousness’” is apparent. It further argued that the decision sought to protect public health. It stated that dimethyltryptamine (DMT), one of the active hallucinogenic substances in ayahuasca, poses significant safety risks, citing the 1971 UN Convention on Psychotropic Substances.

The final ruling by the Oversight Board overturn's Instagram's decision to take it down, citing that Instagram's own community guidelines were somewhat confusing and focused on the buying and selling of illegal drugs (though it notes that the rules are confusing, because they said "illegal or prescription drugs (even if legal in your region)" which is, as the Board notes "confusing and contradictory." There was also some level of conflict between Instagram's rules and Facebook's rules, and an unclear delineation of which one should apply where.

But where the impossibility of content moderation on topics like this really shines through is in the discussion about "voice" vs. "safety," which was one of the key points that Instagram made in defending its decision. The Board says it values things differently than the company:

The Board concludes that Meta’s decision to remove the content was not consistent with the company’s values. In this case, as in many, Meta's values of “Voice,” “Safety,” and “Dignity” point in different directions. Meta's decision to take down the post weighted “Safety” over “Voice.” The Board would balance the values differently, believing that the genuine but not particularly strong interests in “Safety” are outweighed in this context by the value of “Voice” and the importance of recognizing the “Dignity” of those engaging in traditional or religious uses where there is historic evidence of such use, including by Indigenous and religious communities. Scientific research indicates that the use of ayahuasca in a controlled context in traditional and religious ceremonies is not linked to a serious risk of harm. Meta cited the European Court of Human Rights case of Franklin-Beentjes and Ceflu-Luz da Floresta v. The Netherlands (Case No. 28167/07, European Court of Human Rights, May 6, 2014) to demonstrate the risks of ayahuasca use. Other courts, however, have reached different conclusions – for example, the United States Supreme Court, in considering the risk of harm from “the circumscribed, sacramental use of hoasca” by members of an ayahuasca based religion, found that the government had not put forward sufficient evidence of harm from religious use, which was its burden, to justify the prohibition in these circumstances. Meta’s rationale does not appear to have taken into account controlled uses of ayahuasca which aim to mitigate health risks. In light of scientific research, Meta’s rationale did not demonstrate the danger of this post to the value of “Safety” in a manner sufficient to displace “Voice” and “Dignity” to the extent to justify removal of the post.

There should be no way to look at that paragraph and not realize just how subjective all of this is. Every one of these involves judgment calls and people weigh different values in very different ways. Even people who work at the same company. Figuring out how to (1) write rules that take all this into account, (2) express those rules in AI moderation systems, (3) express those rules clearly to tens of thousands of human moderators across hundreds of languages and cultures, (4) and expect it to work at a scale of billions of users and millions of posts per day... is literally impossible. It cannot be done well.

That doesn't mean that everyone should give up, but there should at least be some level of humility in recognizing the level of this task, and highlighting individual decisions that we disagree with is not evidence that people or the company "don't care" about this issue, but rather evidence of just how impossible a task this truly is.

Filed Under: ayahuasca, brazil, content moderation, drugs, health misinformation
Companies: facebook, instagram, meta, oversight board

A Grope In Meta's Space

from the the-value-of-user-tools-in-moderation dept

Horizon Worlds is a VR (virtual reality) social space and world building game created by Facebook. In early December, a beta tester wrote about being virtually groped by another Horizon Worlds user. A few weeks later, The Verge and other outlets published stories about the incident. However, their coverage omits key details from the victim’s account. As a result, it presents the assault as a failure of user operated moderation tools rather than the limits of top-down moderation. Nevertheless, this VR groping illustrates the difficulty of moderating VR, and the enduring value of tools that let users solve problems for themselves.

The user explains that they reported and blocked the groper, and a Facebook “guide”, an experienced user trained and certified by Facebook, failed to intervene. They write, “I think what made it worse, was even after I reported, and eventually blocked the assaulter, the guide in the plaza did and said nothing.” In the interest of transparency, I have republished the beta user’s post in full, sans identifying information, here;

**Trigger Warning** Sexual Harassment. My apologies for the long post: Feel free to move on.

Good morning,

I rarely wake up with a heavy heart and a feeling of anger to start a fresh new day, but that is how I feel this morning. I want to be seen and heard. I reach out to my fellow community members in hopes of understanding and reassurance that they will be proactive in supporting victims and eliminating certain types of behavior in horizon worlds. My expectations as a creator in horizon worlds aren't unreasonable and I'm sure many will agree.

You see this isn't the first time, I'm sure it won't be the last time that someone has sexually harassed me in virtual reality. Sexual harassment is no joke on the regular Internet but being in VR adds another layer that makes the event more intense. Not only was I groped last night, but there were other people there who supported this behavior which made me feel isolated in the Plaza. I think what made it worse, was even after I reported, and eventually blocked the assaulter, the guide in the plaza did and said nothing. He moved himself far across the map as if to say, you're on your now.

Even though my physical body was far removed from the event, my brain is tricked into thinking it's real, because.....you know......Virtual REALITY. We can't tout VR's realness and then lay claim that it is not a real assault. Mind you, this all happened within one minute of arriving in the plaza, I hadn't spoken a word yet and could have possibly been a 12-year-old girl.

MY ASK:

I would like a personal bubble that will force people away from my avatar and I would like to be able to upload my own recording with my harassment ticket. I would also like that all guides are given sensitivity training on this specific subject, so they will understand what is expected. If META won't give guides tools that will allow them to remove a player immediately from a situation, at least train them to deal with it and not run away.

Rant over, I'm still mad, but I will sort through and process. I love this community and the thought of leaving it makes me deeply sad. So I am hopeful we can evolve as a community and foster behaviors that support collaboration, understand, and a willingness to speak out against gross behaviors.

Initial coverage in The Verge did not mention the victim’s use of the block feature, even as the user describes using it in the post above. Instead, reporter Alex Heath relayed Facebook’s account of the incident, saying “the company determined that the beta tester didn’t utilize the safety features built into Horizon Worlds, including the ability to block someone from interacting with you.”

These details are important because subsequent writing about the incident builds on the false, but purported non-use of the blocking feature to make the case that offering users tools to control their virtual experience is “unfair and doesn’t work.” In Technology Review, Tanya Basu makes hay of the user’s failure to use the “safe zone” feature, which temporarily removes users from their surroundings. Yet this is a red herring. The user might not have immediately disappeared into her safe zone, but she used the block feature to do away with her assailant.

In reality, contra Basu or Facebook’s description of events, it seems that user-directed blocking put a stop to the harm while the platform provided community guide failed to intervene. VR groping is a serious issue, but it is not one that will be solved from the top-down. Inaccurate reporting that casts user-operated moderation tools as ineffective may spur platforms to pursue less effective solutions to sexual harassment in VR.

Implications of the incident’s misreporting aside, it provides a useful case study in the difficulties of moderating VR. One suggestion put forward by the user and backed by University of Washington Professor Katherine Cross warrants discussion. Closer inspection of their proposals illustrates the careful tradeoffs that inform the current safe zone and blocking tools offered to Horizon users.

They request a “personal bubble that will force people away from my avatar” or “automatic personal distance unless two people mutually agreed to be closer.” This might make some groping harder, but it creates other opportunities for abuse.

If players’ avatars can take up physical space and block movement, keeping others at bay, then they can block doorways and trap other players in corners or against other parts of world. Player collision could render abuse inescapable or allow players to hold others’ avatars prisoner.

MMOs (Massively Multiplayer Online games) have long struggled with this problem – “holding the door” is only a contextually heroic action. Player collision makes gameplay more realistic, but allows some players to limit everyone else’s access to important buildings by loitering in the doorway.

Currently, players’ avatars in Horizon may pass through one another. They can retreat into a safe zone, disappearing from the world. They can also “block” other users – preventing both the blocked and blocking users from seeing one another. Even through a block, they can still see one each other’s nametags – total invisibility created problems I covered here. As such, the current suite of user moderation tools strikes a good balance between empowering users and minimizing new opportunities for misuse.

Finally, given the similarity of the transgression, it is worth recalling Julian Dibbell’s “A Rape in Cyberspace”, one of the first serious accounts of community governance online. In this Village Voice article, Dibbell relates how users of role-playing chatroom LambdaMOO (the best virtual reality to be had in 1993) responded to a string of virtual sexual assaults. After fruitless deliberation, a LambdaMOO coder banned the offending user. After the incident, LambdaMOO established democratic procedures for banning abusive users, and created a “boot” command allowing users to temporarily remove troublemakers.

As the internet has developed content moderation has centralized. Today, users are usually expected to let platforms moderate for them. However, just as in the web’s early days, empowering users remains the best solution to interpersonal abuse. The tools they need to keep themselves safe may be different, but in virtual reality as in role-playing chat, those closest to abuse are best positioned to address it. Users being harassed should not be expected to wait for the mods.

Will Duffield is a Policy Analyst at the Cato Institute

Filed Under: blocking, content moderation, horizon worlds, sexual harassment, user tools, vr
Companies: facebook, meta

Facebook Blocks Seven Malware Purveyors, Deletes Hundreds Of Accounts, Notifies 50,000 Potential Hacking Targets

from the move-fast-and-break-oppressors dept

Thanks to the ongoing onslaught of negative press involving malware merchants like Israel's NSO Group, tech companies whose devices and platforms have been used to deploy exploits targeting journalists, activists, and religious leaders are punching back. You're a human rights abuser with high-dollar spyware at your disposal? Too bad. Ask for a refund, I guess.

Apple sued NSO Group for targeting iPhone users a few weeks ago. It also began notifying users who were targeted by NSO spyware, potentially nullifying further surveillance efforts by unfriendly nation-states.

But before Apple got in on the anti-NSO action, Facebook sued the company for using WhatsApp to deploy malware. Both lawsuits contain some troubling implications for the CFAA -- something that could pose future problems for researchers who scrape data and security researchers who search for security flaws. The unintended consequences of this litigation have yet to be seen, but it's enough to justify holding your applause until the lawsuits have run their course.

Denying state actors the fruits of their purchased spyware labor is now the name of the game, something that benefits everyone. Facebook (now Meta) has just thrown a decently-sized tech wrench into the malware works of an unknown number of entities.

Facebook has disrupted the operations of seven different spyware-making companies, blocking their Internet infrastructure, sending cease and desist letters, and banning them from its platform.

"As a result of our months-long investigation, we took action against seven different surveillance-for-hire entities to disrupt their ability to use their digital infrastructure to abuse social media platforms and enable surveillance of people across the internet," said Director of Threat Disruption David Agranovich and Head of Cyber Espionage Investigations Mike Dvilyanski.

"These surveillance providers are based in China, Israel, India, and North Macedonia. They targeted people in over 100 countries around the world on behalf of their clients."

Seven companies. 100 countries. 1,500 Facebook and Instagram accounts. 50,000 potential targets notified. That's going to hurt paying customers of companies like NSO Group and its competition, most of which have yet to obtain the international infamy NSO has.

The full report [PDF] from Meta lists the companies ejected in this surveillance-for-hire purge. And there's a common strain running through the list, one that's going to cause even more problems for a government already dealing with blowback for running interference for a company selling spy tools to a long list of human rights violators.

We removed about 200 accounts which were operated by Cobwebs [Technologies] and its customers worldwide. This firm was founded in Israel with offices in the United States and sells access to its platform that enables reconnaissance across the internet, including Facebook, Instagram, WhatsApp, Twitter, Flickr, public websites and “dark web” sites.

[...]

We removed about 100 accounts on Facebook and Instagram which were linked to Cognyte (formerly known as WebintPro) and its customers. This firm is based in Israel and sells access to its platform which enables managing fake accounts across social media platforms including Facebook, Instagram, Twitter, YouTube, and VKontakte (VK), and other websites to social-engineer people and collect data.

[...]

We removed about 300 Facebook and Instagram accounts linked to Black Cube, an Israeli-based firm with offices in the UK, Israel and Spain. It provides surveillance services that include social engineering and intelligence gathering.

[...]

We removed about 100 Facebook accounts linked to Bluehawk, a firm based in Israel with offices in the UK and the US. We collaborated on this investigation with The Daily Beast who had identified a subset of this activity leading us to uncover the full cluster and who’s behind it earlier this year.

[...]

We removed about 400 Facebook accounts, the vast majority of which were inactive for years, linked to BellTroX and used for reconnaissance, social engineering and to send malicious links. BellTroX is based in India and sells what’s known as “hacking for hire” services…

[...]

We removed about 300 accounts on Facebook and Instagram linked to Cytrox. This North Macedonian company develops exploits and sells surveillance tools and malware that enable its clients to compromise iOS and Android devices…

[...]

We removed about 100 Facebook and Instagram accounts linked to an unidentified entity in China responsible for developing surveillanceware for Android, iOS, Windows, and also Linux, Mac OS X, and Solaris operating systems. It also engaged in reconnaissance and social engineering activity before delivering malicious payload to its targets.

Four of the seven entities identified and blocked call Israel home. Cytrox also has links to Israel as both Citizen Lab and the Times of Israel have reported. Cytrox is now part of a spyware conglomerate that has been criminally charged for human rights violations.

Israel has a malware problem. And the government can't claim it was unaware of these companies and their selling of tools to authoritarians and human rights violators. The government was actively involved in brokering some of these deals.

The customers for these products, however, are far more varied. Cobwebs products were observed "frequently targeting" activists and opposition parties in Hong Kong and Mexico. But its customer base includes entities in Bangladesh, New Zealand, Poland, Saudi Arabia, and… the United States.

The list compiled by Meta shows malware firms are more than happy to sell to governments that like targeting critics and opponents, rather than criminals and terrorists. Serbia, Morocco, Mexico, China, Qatar, Saudi Arabia, Egypt, Oman, the Philippines, and Myanmar make this list.

But it's not just the blocking. It's the notification. And there's plenty of that.

We also alerted around 50,000 people who we believe were targeted by these malicious activities worldwide, using the alert system we launched in 2015. We recently updated it to provide people with more granular details about the types of targeting and the actor behind it so they can take steps to protect their accounts, depending on the phase of the surveillance attack chain we detect in each case.

That's a lot of disrupted surveillance efforts. State actors are paying good money for these exploits and now they're facing more resistance than ever from the private sector being used to transport malware to targets. That's a lot of money and a lot of surveillance being undone. Governments buying exploits won't be happy but so what. There's no reason to assume that just because it's a government agency doing the targeting there's any legitimacy to the hacking efforts. This is the way it should be -- platforms and device makers protecting customers and users against hacking attempts, no matter the origin of the attacks. The world needs more of this because authoritarians and human rights abusers deserve to have their oppressive efforts thwarted.

Filed Under: hacking, malware, surveillance
Companies: facebook

The Internet Industry's Most High Profile, But Least Successful, Trade Group Dissolves

from the good-riddance dept

While this may feel like Washington DC insider baseball, it's fairly notable that the "big" internet trade/lobbying group, the Internet Association has announced it's shutting down (Emily Birnbaum at Politico had the scoop the night before the official announcement). There will likely be a bunch of post mortems and discussions about this happening just as the big internet companies (who came together to set up IA in the first place) are under such regulatory threats. But, to me, this is good riddance. It was an organization that more often than not made things worse for the internet, rather than better. And that's too bad, because it had a real chance to do the opposite. This is not to say there weren't good people who worked there -- there absolutely were. But as an organization, it missed a ton of opportunities to do the right thing.

The Internet Association was formed in 2012, soon after the SOPA fight. I was asked to meet with some of the folks putting it together at the beginning, and was a bit confused as to what purpose it would serve. There are, already, a few trade groups that represents internet companies, and I wasn't entirely clear on the need for a new one. The story I heard (more or less) was that the "big" trade groups -- including the RIAA, (then) MPAA, NTIA, NCTA had all "professionalized" the trade group space, and that they were seen as much more official and powerful than the more scrappy trade groups representing the internet companies -- like CCIA. While there was also CTA (at that time, still known as CEA), which was bigger and "professional," it was seen as having too broad a coverage, representing not just the internet (on which it actually does a great job), but the wider technology/electronics industry.

However, what struck me at the time of its founding, and in various meetings I had with people at the Internet Association over the years was that they seemed to have no fundamental principles behind their lobbying and advocacy. It seemed to be entirely a political organization. Obviously, any lobbying/trade group is -- perhaps by definition -- a political organization, which is often responsible for figuring out which ways the wind blows on certain regulations. But, still, the more successful trade groups always seem to have some core, fundamental principles that they fight for (even if those core fundamental principles are sometimes silly and misguided -- see: RIAA, MPAA, etc.). And that's what makes their advocacy more powerful. The Internet Association never seemed to really stand for anything.

This all came to a head, most notably, in the Internet Association's about face on FOSTA. As you may recall, there was a pretty unified front against FOSTA from the entire internet, and then suddenly -- almost completely out of the blue -- the Internet Association endorsed it. As I wrote in a big post mortem about how FOSTA became law, much of the blame can be laid at the feet of the Internet Association. It is true that Facebook and Netflix (two giant members at the time, though Netflix later left) decided that FOSTA wasn't a fight they were interested in. This pissed off smaller members of IA tremendously. Days after the announcement, I spoke to someone at a smaller (but still quite successful) internet company, who spent the better part of an hour venting angrily about how the Internet Association screwed them over, and that many other smaller members felt similarly.

I later met with two different executives at IA who both tried to defend the decision as "if we didn't do this, something worse was coming," but no one else seems to believe that (and Congressional staffers told me that wasn't true -- and that they had actually been very close on something that would have been much better). It was, yet again, a political move, rather than a principled one. And, after that, the Internet Association just couldn't be trusted any more.

And as if to just put a huge exclamation point on the idea that the Internet Association was political rather than principled, it pulled this bit of nonsense:

Compounding problems was the decision (after a one year search, in which the organization had no permanent director) to appoint someone with no real background in the internet, or the biggest issues related to internet policy, to be the new executive director. Instead, it was someone who had worked mostly in the telecom space, including at multiple big professionalized trade groups in that space. And, again, you can see the thinking there, given the desire for the Internet Association to punch at that level. But, again, it left the organization without a principled stance, and put directional choices even more in the hands of its members -- who don't necessarily agree on things that much these days. Facebook's power over the organization (as seen in the FOSTA decision) also turned off plenty of companies that are just sick of Facebook ruining the internet for everyone else.

In other words, it was a lobbying group with nothing to actually lobby for, and what little it did lobby for, it did so without conviction. It was just going through the expensive motions. So, good riddance to the Internet Association. There are still other groups that really do stand on their principles and seem to do a damn good job of actually fighting for core ideas around the internet -- like CCIA, CTA, Engine Advocacy, NetChoice, and (the newest entrant), the Chamber of Progress. Those organizations, while trade groups, actually tend to lead on ideas, have strong principled stances, and are able to make a real difference. It's too bad the Internet Association never really was able to do the same.

Filed Under: lobbying, trade groups
Companies: facebook, internet association

DOJ Tells Courts They Don't Need To Explore The Constitutionality Of Section 230 To Toss Donald Trump's Dumb Lawsuits Out

from the better-than-nothing dept

Last month, we noted that the DOJ had announced it was going to intervene in Donald Trump's bombastically silly lawsuits against Facebook, Twitter, and YouTube for suspending his account for violating the websites' terms of service. Those lawsuits have not been going well. While Trump filed them in his home court in Florida, they've all been transferred to California. His decision to use the case to claim Section 230 is unconstitutional only served to wake up the Justice Department, and have them step in to respond to that particular point.

The DOJ has now filed its briefs -- we'll just share the one in the Twitter case since they're all basically the same -- to say (1) it's easy to dismiss this case without bothering to explore the constitutionality of 230, but if it feels otherwise (2) it's blatantly obvious that 230 is constitutional.

On point one:

Section 230(c) immunity is an affirmative defense, so the Court need not consider the constitutionality of that provision unless Plaintiff has first demonstrated that he has otherwise sufficiently alleged valid claims against Defendants. Accordingly, before the Court reaches the constitutional challenge as part of the preliminary injunction motion, it should first consider whether Plaintiff is otherwise likely to succeed on the merits of his claims against Twitter. For example, the Court might decide that Twitter is not a state actor constrained by the First Amendment, in which case Plaintiff could not prevail on count one of the FAC, meaning there would be no need to address the constitutionality of Section 230(c) with regard to that claim. And the Court might similarly decide that Plaintiff is not likely to succeed on the elements of his state law claims.

Another possible avoidance path stems from Plaintiff’s contention that Section 230(c) does not apply to the misconduct alleged. See Motion at 15-18. If the Court were to conclude that Twitter’s acts as alleged by Plaintiff do not fit within the terms of either Section 230(c)(1) or (2), then the statute would not apply, and there would be no occasion for passing on the constitutionality of the statute. Similarly, if the Court were to agree with Plaintiff that Twitter was coerced into suspending Plaintiff’s account, it might determine that immunity under Section 230(c)(2), which applies to certain actions “voluntarily taken in good faith,” is unavailable

That latter paragraph is a bit head-scratching, because it's pretty blatantly obvious that the websites' moderation is clearly covered by 230, but the overarching point stands: there's no need for the court to explore whether or not 230 is constitutional.

But, then, if the court does decide to go down that path, well...

... although Plaintiff’s constitutional challenge is not fully developed, it appears that Plaintiff’s argument is that Section 230(c) is unconstitutional because it supposedly “encourages” Twitter to “censor constitutionally protected speech.” Id. at 9. That argument misunderstands the nature of Section 230(c) and the reach of the Skinner and Hanson decisions. Section 230(c) does not require online service providers to limit or regulate speech by their users. Instead, Section 230(c) allows companies like Twitter to choose to remove content or allow it to remain on their platforms, without facing liability as publishers or speakers for those editorial decisions. See Barnes, 570 F.3d at 1102 (in the context of Section 230(c), “publication involves reviewing, editing, and deciding whether to publish or to withdraw from publication third-party content.”) (emphasis added). Section 230(c) does not reflect a preference for restricting content, much less for restricting content relating to any particular viewpoint.

And further:

As courts in this district have already concluded, the federal regulations in Skinner bear no relationship to Section 230(c). “[N]othing about Section 230 is coercive.” Divino, 2021 U.S. Dist. LEXIS 3245, at *17 (contrasting the regulations in Skinner). Rather, “Section 230 reflects a deliberate absence of government involvement in regulating online speech[.]” Id. “Unlike the regulations in Skinner, Section 230 does not require private entities to do anything, nor does it give the government a right to supervise or obtain information about private activity.” Id. Therefore, Twitter “remains a private forum, not a public forum subject to judicial scrutiny under the First Amendment.” Prager University v. Google LLC, 951 F.3d 991, 995 (9th Cir. 2020). “[C]ourts have uniformly concluded that digital internet platforms that open their property to user-generated content do not become state actors[.]” See id. at 997; see also Atkinson v. Meta Platforms, Inc., No. 20-17489, 2021 U.S. App. LEXIS 34632, at *3 (9th Cir. Nov. 22, 2021) (“Section 230 of the Communications Decency Act does not independently transform Meta Platforms into a government actor for First Amendment purposes.”).

Meanwhile, in that same case, Twitter has also filed its motion to dismiss, and it's worth a read just to understand how bloviatingly pathetic Trump's original complaint is.

Plaintiffs—like all Twitter account holders—agreed to abide by Twitter’s Rules, and yet proceeded to repeatedly violate those Rules. When Twitter responded by suspending or restricting their accounts, Plaintiffs filed this putative nationwide class-action seeking, among other things, an injunction forcing Twitter to carry Plaintiffs’ speech and appointing a court-supervised monitor to oversee all of Twitter’s future content-moderation decisions for the hundreds of millions of Tweets posted on its platform every day. In support of such unprecedented relief, Plaintiffs assert four claims that would upend bedrock principles of constitutional law, disregard standing and other procedural limitations, and stretch Florida consumer protection laws far beyond their geographic, temporal, and substantive limits.

Each of Plaintiffs’ claims is defective for multiple reasons. To summarize just a few: Plaintiffs’ lead claim—that Twitter’s editorial judgments not to disseminate their messages violated Plaintiffs’ First Amendment rights—ignores both that Twitter is a private actor that is not constrained by the federal constitution and that Twitter has its own First Amendment rights to make those judgments. Plaintiffs’ invitation for this Court to invalidate a decades-old federal statute (47 U.S.C. § 230) as violative of the First Amendment fails both because they have no standing to assert such a claim and because their suggestion that the statute is unconstitutional is frivolous. Plaintiffs’ claim under the general consumer protection provision of the Florida Unfair and Deceptive Trade Practices Act (“FDUPTA”) founders both because that provision does not apply in this case due to a binding contractual choice-of-law clause and because the Complaint alleges no facts that support Plaintiffs’ theories of consumer deception. And Plaintiffs’ claim under a new (and recently enjoined) Florida statute that purports to regulate content moderation by certain social media companies (Florida SB 7072) fails at the threshold because all of the conduct challenged in the Complaint occurred before the statute took effect. Plaintiffs’ claims should all be dismissed with prejudice.

Don't hold back now. There's a lot more in the filing, but we'll wait until the judge almost certainly dismisses the case to explore the reasoning...

Filed Under: doj, donald trump, section 230, social media
Companies: facebook, twitter, youtube

Apple's 'Do Not Track' Button Is Privacy Theater

from the privacy-theater-incorporated dept

Earlier this year Apple received ample coverage about how the company was making privacy easier for its customers by introducing a new, simple, tracking opt-out button for users as part of an iOS 14.5 update. Early press reports heavily hyped the concept, which purportedly gave consumers control of which apps were able to collect and monetize user data or track user behavior across the internet. Advertisers (most notably Facebook) cried like a disappointed toddler at Christmas, given the obvious fact that giving users more control over data collection and monetization, means less money for them.

By September researchers had begun to notice that Apple's opt-out system was somewhat performative anyway. The underlying system only really blocked app makers from accessing one bit of data: your phone's ID for Advertisers, or IDFA. There were numerous ways for app makers to track users anyway, so they quickly got to work doing exactly that, collecting information on everything from your IP address and battery charge and volume levels, to remaining device storage, metrics that can be helpful in building personalized profiles of each and every Apple user.

Privacy advocates and the press noted how this was all giving Apple users a false sense of security without really fixing much. Privacy experts and press outlets also repeatedly informed Apple this was happening, but nothing changed. In fact, the Financial Times notes that six months after the feature was introduced, Apple has further softened its stance on the whole effort:

"But seven months later, companies including Snap and Facebook have been allowed to keep sharing user-level signals from iPhones, as long as that data is anonymised and aggregated rather than tied to specific user profiles.

Here's the thing. There's been just an absolute torrent of studies showing how "anonymizing" data is a gibberish term. It only takes a few additional snippets of data to identify "anonymized" users, yet the term is still thrown around by companies as a sort of "get out of jail free" card when it comes to not respecting user privacy. There's an absolute ocean of data floating around the data broker space that comes from apps, OS makers, hardware vendors, and telecoms, and "anonymizing" data doesn't really stop any of them from building detailed profiles on you.

Apple's opt-out button is largely decorative, helping the company brand itself as hyper privacy conscious without actually doing the heavy lifting required of such a shift:

"Lockdown Privacy, an app that blocks ad trackers, has called Apple’s policy “functionally useless in stopping third-party tracking”. It performed a variety of tests on top apps and observed that personal data and device information is still “being sent to trackers in almost all cases."

A lot of folks spend a lot of time trying to tap dance around a fundamental truth: any effort to give consumers more control and clear insight over what's being collected or sold reduces revenues by billions of dollars annually, even if only a fraction of existing users take advantage. And nobody with their face buried deep in the data monetization trough wants that. It's why it's 2021 and the U.S. still doesn't even have a basic, clear privacy law for the internet era. Not even a super clean one mandating basic transparency requirements and working opt-out tools.

So what we get instead is a lot of gibberish and privacy theater by a lot of folks who don't want to take even a tiny hit in revenues in exchange for healthier markets and happier users.

We also get just an endless parade of semantics, like ISP claims they "don't sell access to your data" (no, they just give massive "anonymized" datasets away for free as part of a nebulous, broader arrangement they do get paid for). We get tracking opt-out tools that don't actually opt you out of tracking, or opt you back in any time changes are made. And we get endless proclamations about how everybody supports codifying federal privacy laws from companies that immediately turn around and spend millions of dollars lobbying to ensure even a basic privacy law never sees the light of day.

At some point this combination of feckless oversight, rampant overcollection of data, minimal transparency, and repeated failure to adhere to the basics on data security will result in a privacy scandal that makes the last fives years' worth of scandals look like a grade school picnic. When that happens, we might finally see some traction on at least a basic law that mandates transparency, opt-out tools that actually work, and penalties for lax security. Until that momentum shift happens, the majority of "privacy reform" efforts are going to have a high ratio of meaningless song and dance.

Filed Under: apps, data, do not track, ios, opt-in, opt-out, privacy
Companies: apple, facebook, snap

Documents Shows Just How Much The FBI Can Obtain From Encrypted Communication Services

from the plenty-of-data-but-content-not-so-much dept

There is no "going dark." Consecutive FBI heads may insist there is, but a document created by their own agency contradicts their dire claims that end-to-end encryption lets the criminals and terrorists win.

Andy Kroll has the document and the details for Rolling Stone:

[I]n a previously unreported FBI document obtained by Rolling Stone, the bureau claims that it’s particularly easy to harvest data from Facebook’s WhatsApp and Apple’s iMessage services, as long as the FBI has a warrant or subpoena. Judging by this document, “the most popular encrypted messaging apps iMessage and WhatsApp are also the most permissive,” according to Mallory Knodel, the chief technology officer at the Center for Democracy and Technology.

The document [PDF] shows what can be obtained from which messaging service, with the FBI noting WhatsApp has plenty of information investigators can obtain, including almost real time collection of communications metadata.

WhatsApp will produce certain user metadata, though not actual message content, every 15 minutes in response to a pen register, the FBI says. The FBI guide explains that most messaging services do not or cannot do this and instead provide data with a lag and not in anything close to real time: “Return data provided by the companies listed below, with the exception of WhatsApp, are actually logs of latent data that are provided to law enforcement in a non-real-time manner and may impact investigations due to delivery delays.”

The FBI can obtain this info with a pen register order -- the legal request used for years to obtain ongoing call data on targeted numbers, including numbers called and length of conversations. With a warrant, the FBI can get even more information. A surprising amount, actually. According to the document, WhatsApp turns over address book contacts for targeted users as well as other WhatsApp users who happen to have the targeted person in their address books.

Combine this form of contact chaining with a few pen register orders, and the FBI can basically eavesdrop on hundreds of conversations in near-real time. The caveat, of course, is that the FBI has no access to the content of the conversations. That remains locked up by WhatsApp's encryption. Communications remain "warrant-proof," to use a phrase bandied about by FBI directors. But is it really?

If investigators are able to access the contents of a phone (by seizing the phone or receiving permission from someone to view their end of conversations), encryption is no longer a problem. That's one way to get past the going darkness. Then there's stuff stored in the cloud, which can give law enforcement access to communications despite the presence of end-to-end encryption. Backups of messages might not be encrypted and -- as the document points out -- a warrant will put those in the hands of law enforcement.

If target is using an iPhone and iCloud backups enabled, iCloud returns may contain WhatsApp data, to include message content.

This is a feature of cloud backups -- a way to retrieve messages if something goes wrong with someone's phone or their WhatsApp account. It's also a bug that makes encryption irrelevant. The same goes for Apple's iMessage service. Encryption or no, backups are not encrypted by service providers. In the case of Apple's iMessage, warrants for iCloud backups will give law enforcement the encryption key needed to decrypt the stashed messages.

On the other side, there are truly secure options that the FBI considers dead ends, starting with Signal. Signal retains no user info, which means there's nothing to be had no matter what paperwork the feds produce. But, for the most part, even encrypted messaging and email services generate metadata that can be obtained without a warrant. If investigators want more, warrants can actually result in investigators obtaining a great deal of information about users, their interactions, and their communications. And, as is noted directly above, it can also grant access to communications users mistakenly believed were beyond the reach of law enforcement.

But not everyone using encrypted services is a criminal, no matter what FBI directors say in public. Communications metadata being only a subpoena or pen register order away is concerning, especially for those who use encrypted services not only to maintain their own privacy, but to protect those they communicate with.

“WhatsApp offering all of this information is devastating to a reporter communicating with a confidential source,” says Daniel Kahn Gillmor, a senior staff technologist at the ACLU.

Those who truly understand the protocols and platforms they use for communications will understand the tradeoffs. For everyone else, there's this handy tip sheet, compiled by none other than the FBI, which explains exactly what each service retains and what each service will hand over in response to government paperwork.

It also shows that encryption isn't keeping law enforcement from pursuing investigations. In rare cases, investigators may have zero access to communications. But every communications platform or service creates a digital paper trail investigators can follow until they find something that breaks the case open. "Going dark" -- the idea that law enforcement is helpless in the face of increased use of encryption -- is a lie. And the FBI knows it.

Filed Under: 4th amendment, encryption, fbi, going dark, lawful access, subpoena, warrant
Companies: apple, facebook, meta, whatsapp

Even As Grifters Insist Otherwise, Courts Know That Social Media Are Not State Actors Because Of Section 230

from the stop-it dept

Over the last few years we've heard a lot of nonsense claiming that Section 230 somehow magically turns social media into state actors. This idea, pushed heavily by disgraced law professor Jed Rubenfeld has not fared well in court. As law professor Eric Goldman highlights, multiple courts have been easily rejecting these claims. Notably, they're mostly citing the failed lawsuit from PragerU that insisted that a very light touch moderation (filtering out a very small percentage of their videos from the even smaller percentage of users who turn on "restricted mode") was a form of censorship. The 9th Circuit pointed out that the 1st Amendment says otherwise (which is amusing since Prager himself pretends to be a big supporter of the 1st Amendment).

Either way, three recent cases claiming that social media were state actors all clearly and easily rejected those claims. The first two of these were pro se lawsuits that were mad about social media sites in semi-coherent complaints that the courts construed as liberally as possible to try to find any sort of legitimate claim, and the courts (rightly) found that those claims were barred because the social media sites (YouTube and LinkedIn) were clearly not state actors.

First up, a district court in the Eastern District of Pennsylvania, which made short work of a lawsuit against YouTube filed by a pro-se plaintiff with a variety of different claims about harassment and hacking (that were somehow YouTube's fault?) leading to the suggestion (the complaint is not particularly clear) of a civil rights violation. The court points out that YouTube is not a state actor and thus there's no possible claim here:

Based on the Complaint’s allegations, it appears the named Defendants – a private social media company and its legal department – are not subject to liability under Section 1983. Cf. Prager Univ. v. Google LLC, 951 F.3d 991, 999 (9th Cir. 2020) (affirming the dismissal of a First Amendment claim because YouTube was a private entity and not a state actor); see also Rutenburg v. Twitter, Inc., No. 21-0548, 2021 WL 1338958, at *2 (N.D. Cal. Apr. 9, 2021) (“Federal courts have uniformly rejected attempts to treat similar social media companies [such as Twitter, Facebook, YouTube, and Google] as state actors under Section 1983.”) (collecting cases). Sescey does not allege Defendants are state actors or that they had any connection to a state, county, or local governmental entity. Her Complaint does not allege any facts to show a “close nexus” between the private behavior of YouTube and its legal department and the state itself such that the challenged action here can fairly be treated as an action of the state. Leshko, 423 F.3d at 339. None of Sescey’s allegations support an inference that Defendants are anything other than a privately-run social media company and its internal legal department.

The next one, filed against LinkedIn, requires barely more than a page to dismiss:

The district court properly dismissed Perez’s action because Perez failed to allege facts sufficient to state a plausible claim. See Hebbe v. Pliler, 627 F.3d 338, 341-42 (9th Cir. 2010) (although pro se pleadings are liberally construed, a plaintiff must allege facts sufficient to state a plausible claim); see also Prager U. v. Google LLC, 951 F.3d 991, 996-97 (9th Cir. 2020) (internet media websites are not government actors under the First Amendment);

The other case, also highlighted by Eric Goldman, isn't a pro-se filing like the other two, but at least marginally (in the most marginal of senses) more serious. It was filed on behalf of Cameron Atkinson against Facebook (though the caption is now updated to Meta!) who claims that Facebook broke the law in "censoring" him. The details of the complaint are... really something.

Like many of his fellow citizens and students of the law, the Plaintiff, Cameron L. Atkinson is a thinker who, regardless of whether he is right or wrong, loves to share his thoughts and hear the thoughts of others. He regularly posts on Facebook about political and legal developments with the same civility that he would use in the courtroom or the classroom, seeking to engage in debate with the community of fellow law students and other friends whose respect he has gained.

The Plaintiff, Cameron L. Atkinson, is also an inquiring man who rarely rushes to judgment, often choosing to find out for himself before condemning someone.

Consequently, when Cameron L. Atkinson learned from friends that Facebook was censoring conservatives' posts that mentioned the name of Eric Ciaramella, the alleged Ukranian whistleblower who has provided the impetus for the pending impeachment proceedings against President Donald Trump, he decided to test the scope of Facebook's censorship himself.

Cameron L. Atkinson's first post occurred on the morning of November 11, 2019. He published a post on Facebook that read "Test post: Eric Ciaramella is a hero for blowing the whistle on the Trump administration's treason with Ukraine." See Exhibit 1

Approximately four minutes later, Cameron L. Atkinson published a second test post on Facebook that read "Test post 2: Eric Ciaramella is a dirty lying rat for trying to take down the Trump administration." See Exhibit 1.

Cameron L. Atkinson's object in publishing these posts was to see if Facebook would censor one post, but not the other.

Within 5 hours, Facebook removed both of Cameron L. Atkinson's posts with no warning or notification.

After learning of Facebook's censorship, Cameron L. Atkinson published a third post that read as follows:

I have conflicting thoughts about the naming of Eric Ciaramella, the alleged Ukraine whistleblower. Tattling in the dark shadows destroys public confidence in a matter of serious public interest. On the other hand, the vitriolic nature of our society may very well raise concerns for his safety. However, it may also end up protecting his well-being. Regardless, I think that people should be open to debating the merits of this serious public question.

See Exhibit 2.

Again, less than 5 hours later, Facebook removed Cameron L. Atkinson's third post without notifying him or warning him

The complaint alleged a bunch of violations of things that do not exist:

The Plaintiff claims a violation of his rights under the First Amendment to the United States Constitution, violations of the Communications Decency Act, statutory fraud, theft, a breach of the implied warranty of fair dealing, and violations of the Connecticut Unfair Trade Practices Act (CUPTA)

The case was transferred from Connecticut to California under Facebook's Meta's terms of service, and then tossed out easily by the district court. The appeals court upholds the lower court ruling, but as Goldman notes, the ruling is a bit sloppy (and also they did it as non-precedential for unclear reasons).

But, as the court points out, the 1st Amendment claims are ridiculous because Facebook (shut up, it's not "Meta") is not a state actor:

The district court properly dismissed Atkinson’s First Amendment claim because he did not allege sufficient facts to infer that Meta Platforms is a government actor. Unless certain exceptions apply, the First Amendment only restricts government action. Manhattan Cmty. Access Corp. v. Halleck, 139 S. Ct. 1921, 1928 (2019) (enumerating exceptions). Dismissal is proper when a complaint lacks “a cognizable legal theory” or “sufficient well-pleaded, nonconclusory factual allegations” to state a plausible claim for relief....

Atkinson does not allege plausibly that the federal government “compel[led] [Meta Platforms] to take a particular action,” Halleck, 139 S. Ct. at 1928, or “exercised coercive power,” Blum v. Yaretsky, 457 U.S. 991, 1004 (1982). Allegations of federal coercion do not support the deprivation under color of state law that Atkinson pleaded. See 42 U.S.C. § 1983; Heineke v. Santa Clara Univ., 965 F.3d 1009, 1013 n.3 (9th Cir. 2020) (finding that private actors are only subject to § 1983 liability under a state action theory). Even if such facts could support his claim, Atkinson does not allege federal coercion sufficiently. Instead, his allegations cast Meta Platforms’ decision to adopt community standards as a self-interested business decision.

Atkinson also does not plead sufficient facts to infer that Meta Platforms acted jointly with state governments. Halleck, 139 S. Ct. at 1928. We need not accept as true factual allegations that Atkinson’s submitted exhibit contradicts. Gonzalez v. Planned Parenthood of L.A., 759 F.3d 1112, 1115 (9th Cir. 2014). To the extent that Atkinson’s allegations are not contradicted, they still do not reflect joint action. Absent more, the fact that state officials responded to Meta Platforms’ unsolicited inquiries does not plausibly allege such a degree of “interdependence . . . that [the state] must be recognized as a joint participant” in Meta Platforms’ editorial decisions. Tsao v. Desert Place, Inc., 698 F.3d 1128, 1140 (9th Cir. 2012) (citation omitted). Atkinson does not offer other facts that would make a joint action claim plausible, such as an agreement between state governments and Meta Platforms; state participation in Meta Platforms’ corporate governance; financial ties between state governments and Meta Platforms; or Meta Platforms’ regulation of state activities. Cf. Brentwood v. Tenn. Secondary Sch. Athletic Ass’n, 531 U.S. 288, 298–302 (2001) (explaining state action existed under such circumstances). We cannot infer that state officials “dominate[d]” Meta Platforms’ decision making from Atkinson’s allegations.

Then there's the question, pushed by Rubenfeld and other very foolish people, that Section 230 magically turns websites into state actors. The court says "nuh uh" without much elaboration:

Finally, Section 230 of the Communications Decency Act does not independently transform Meta Platforms into a government actor for First Amendment purposes

Basically, that line of argument is so dumb it doesn't even require any analysis other than saying "no, that's not true."

As for "violating Section 230," well, that's not a thing.

Section 230 does not explicitly create a private right to sue, see 47 U.S.C. § 230, nor do its “language, structure, context, [or] legislative history” implicitly reflect any congressional intent to establish one. Lil’ Man in the Boat, Inc. v. City and County of San Francisco, 5 F.4th 952, 958 (9th Cir. 2021). Section 230 is not “phrased in terms of [individual] persons benefited.” ...

It's somewhat telling how easily and quickly courts are dumping these claims, at the same time that the usual grifters and nonsense-peddlers continue to insist that there's a reasonable legal argument that social media are somehow state actors. They are not and no one should take seriously anyone who claims otherwise.

Filed Under: 1st amendment, section 230, social media, state actors
Companies: facebook, linkedin, youtube