Whatever Problem EARN IT Is Trying To Solve, It Doesn't

from the that-seems-like-a-problem dept

I've already talked about the potential 1st Amendment problems with the EARN IT Act and the potential 4th Amendment problems with it as well. But a recent post by Riana Pfefferkorn at Stanford raises an even bigger issue in all of this: what actual problem is EARN IT trying to solve?

This sounds like a simple question with a potentially simple answer, but the reality, once you start to dig in, suggests that either (1) the backers of EARN IT don't actually know, or (2) if they do know, they know what they actually want is unconstitutional.

Supporters of EARN IT will say, simply, the problem they're trying to solve is the prevalence of child sexual abuse material (CSAM) online. And, that is a real problem (unlike some other moral panics, CSAM is a legitimate, large, and extraordinarily serious problem). But... CSAM is already very, very illegal. So, if you dig in a little further, supporters of EARN IT will say that the problem they're really trying to solve is that... internet companies don't take CSAM seriously enough. But, the law (18 USC 2258A already has pretty strict requirements for websites to report any CSAM they find to NCMEC (the National Center for Missing & Exploited Children) -- and they do. NCMEC reported that it received almost 21.4 million reports of CSAM from websites. Ironically, many supporters of EARN IT point to these numbers as proof that the websites aren't doing enough, while also saying it proves they don't have any incentive to report -- which makes no sense at all.

So... is the problem that those 21.4 million reports didn't result in the DOJ prosecuting enough abusers? If so... isn't the problem somewhere between NCMEC and the DOJ? Because the DOJ can already prosecute for CSAM and Section 230 doesn't get in the way of that (it does not immunize against federal criminal law). And, as Riana noted in her article, this very same Senate Committee just recently heard about how the FBI actually knew about an actual serial child sex abuser named Larry Nasser, and turned a blind eye.

And, if NCMEC is the problem (namely in that it can't process the reports fast enough), then this bill doesn't help at all there either, because the bill doesn't give NCMEC any more funding. And, if the senators are correct that this bill would increase the reports to NCMEC (though it's not clear why that would work), wouldn't that just make it even more difficult for NCMEC to sort through the reports and alert law enforcement?

So... is the problem that companies aren't reporting enough CSAM? If you read the sponsors' myths and facts document, they make this claim -- but, again, the law (with really serious penalties) already requires them to report any CSAM. Taking away Section 230 protections won't change that. Reading between the lines of the "myths and facts" document, they seem to really be saying that the problem is that not every internet service proactively scans every bit of content, but as we've discussed that can't be the problem, because if that is the problem, EARN IT has a massive 4th Amendment problem that will enable actual child sex abusers to suppress evidence!

Basically, if you look step by step through the potential problems that supporters of the bill claim it tries to solve, you immediately realize it doesn't actually solve any of them. And, for nearly all of the potential problems, it seems like there's a much more efficient and effective solution which EARN IT does not do. Riana's post has a handy dandy table walking down each of these paths, but I wanted to make it even clearer, and felt that a table isn't the best way to walk through this. So here is her chart, rewritten (all credit to her brilliant work):

If online services don't report CSAM in violation of 2258A, and the real problem is large-scale, widespread, pervasive noncompliance by numerous providers that knowingly host CSAM without removing or reporting it (NOT just occasional isolated incidents), then there's a very long list of potential remedies:

• Conduct a congressional investigation to determine the extent of the problem
• Hold a hearing to ask DOJ why it has never once brought a 2258A prosecution
• DOJ prosecutes all those providers for illegally hosting CSAM under 2252A as well as violating 2258A’s reporting requirements
• Amend 2258A(e) to increase penalties for noncompliance
• Amend Dodd-Frank to include 2258A compliance in corporate disclosure requirements (akin to Form SD)
• Encourage FTC investigation of noncompliant companies for unfair or deceptive business practices
• Encourage private plaintiffs to file securities-fraud class actions against publicly-traded providers for misleading investors by secretly violating federal reporting duties

If that's the actual problem (which supporters imply, but when you try to get them to say it outright they hem and haw and won't admit it), then it seems like any of the above list would actually be helpful here. And the real question we should be asking is why hasn't the DOJ done anything here?

But what does EARN IT actually do?

• Amend Section 230 instead of enforcing existing law
• Don’t demand that DOJ explain why they aren’t doing their job

Okay, so maybe the supporters will say (as they sometimes admit) that most web sites out there actually do report CSAM under 2258A, but there are still some providers who don't report it and these are occasional, isolated instances of failure to report by multiple providers, OR repeated failure to report by a particular rogue provider (NOT a large-scale problem across the whole tech industry). If anything, that seems more probably than the first version, which doesn't seem to be reported by any facts. However, here again, there are a bunch of tools in the regulator's tool box to deal with this problem:

• Conduct a congressional investigation to determine the extent of the problem
• Hold a hearing to ask DOJ why it has never once brought a 2258A prosecution
• DOJ prosecutes those isolated violations or the particular rogue provider

Again, what it comes down to in this scenario is that the DOJ is not doing it's job. The law is on the books, and the penalties can be pretty stiff (first failure to report is $150,000 and each subsequent failure is another $300,000). If it's true that providers are not doing enough here, such penalties would add up to quite a lot and the question again should be why isn't the DOJ enforcing the law?

But instead of exploring that, here's what EARN IT actually does:

• Amend Section 230 instead of enforcing existing law
• Don’t demand that DOJ explain why they aren’t doing their job

Okay, so next up, Riana points out that maybe it's possible that the DOJ does regular investigations of websites failing to report CSAM in violation of 2258A, but those investigations are consistently resolved without charges or fines and do not become public. Then, there's a pretty simple option for Congress:

• Hold hearings to have DOJ explain why their investigations never result in charges

But, instead, here's what Congress is doing with EARN IT (stop me if you've heard this one before):

• Amend Section 230 instead of enforcing existing law
• Don’t demand that DOJ explain why they aren’t doing their job

Okay, okay, so maybe the reality is that the DOJ does in fact criminally prosecute websites for 2258A violations, but the reason there is no public record of any such prosecution ever is that all such court records are under seal. This would be... odd, first of all, given that the DOJ loves to publicize prosecutions, especially over CSAM. But, again, here's what Congress could do:

• Tell DOJ to move for courts to unseal all sealed records in 2258A cases
• Require DOJ to report data on all 2258A prosecutions since 2258A’s enactment
• Amend 2258A to require regular reporting to Congress by DOJ of enforcement statistics
• Investigate whether providers (especially publicly-traded ones) kept 2258A fines a secret

But, instead, here's what EARN IT does:

• Amend Section 230 instead of enforcing existing law
• Don’t demand that DOJ reveal to Congress its 2258A enforcement details

So, maybe the real problem is simply that the DOJ seems to be ignoring any effort to enforce violations of 2258A. If that's the case, Congress has tools in its toolbox:

• Hold a hearing to ask DOJ why it has never once brought a 2258A prosecution
• Amend 2258A by adding a private right of action so that victims can do the work that DOJ isn’t doing

Instead, EARN IT...

• Amend Section 230 instead of enforcing existing law
• Don’t demand that DOJ explain why they aren’t doing their job

So... that's basically all the possible permutations if the problem is -- as some supporters claim repeatedly -- that companies are regularly violating 2258A and not reporting CSAM that they find. And, in almost every case, the real questions then should be why isn't the DOJ enforcing the law? And there are lots of ways that Congress should deal with that. But EARN IT does literally none of them.

About the only thing that supporters of EARN IT have claimed in response to this point is that, because EARN IT allows for state AGs and civil suits, it is "adding more cops to the beat" to take on failures to report under 2258A. But... that's kinda weird. Because wouldn't it make a hell of a lot more sense to first find out why the existing cops don't bother? Because no one has done that. And, worse, when it comes to the civil suits, this response basically means "the DOJ doesn't care to help victims of CSAM, so we're leaving it up to them to take matters into their own hands." And that doesn't seem like a reasonable solution no matter how you look at it.

If anything, it looks like Congress putting the burden for the DOJ's perpetual failings... on the victims of CSAM. Yikes!

Of course, there are other possible problems here as well, and Riana details them in the chart. In these cases, the problems aren't with failure to report CSAM, but elsewhere in the process. So... if websites do properly report CSAM to NCMEC's CyberTipline, perhaps the problem is that CSAM isn’t being taken down promptly enough or reported to NCMEC “as soon as reasonably possible” as required by 2258A(a)(1)(A)(i).

Well, then, as Riana notes, there are a few things Congress could do:

• Debate whether to insert a firm timeframe into 2258A(a)(1)(A)(i)
• Hold a hearing to ask ICS providers of various sizes why delays happen and whether a specific timeframe is feasible

Instead, what EARN IT actually does is...

• Amend Section 230

Okay, so if companies are reporting to NCMEC in compliance with 2258A, perhaps the problem is the volume of reports is so high that NCMEC is overwhelmed.

Well, then, the possible solutions from Congress would seem to be:

• Hold a hearing to ask NCMEC what it would take to process all the reports they already get
• Appropriate those additional resources to NCMEC

But, what EARN IT does is...

• Amend Section 230 to induce providers to make even more reports NCMEC can’t keep up with
• Give zero additional resources to NCMEC

Okay, so maybe the websites do properly report CSAM to NCMEC, and NCMEC is able to properly alert the DOJ to the CSAM such that the DOJ should be able to go prosecute the actual abusers, but the DOJ doesn’t act on the reports providers make, and doesn’t make its own mandatory reports to Congress about internet crimes against children. That would be horrifying, but again, it would seem like there's a pretty clear course of action for Congress:

• Order GAO to conduct a study on what happens to CyberTips passed by NCMEC to DOJ
• Hold a hearing to ask DOJ why it isn’t acting on tips or filing its required reports
• Appropriate additional resources to DOJ

All of those would help, if this is the problem, but instead, here's what EARN IT actually does:

• Earmark $1 million for IT improvements
• Don’t demand that DOJ explain why they aren’t doing their job

You might sense a pattern here.

And finally, perhaps websites do report CSAM in compliance with 2258A to NCMEC's CyberTipline, and maybe NCMEC does relay important information to the DOJ... and horrifyingly, perhaps federal law enforcement is failing child sex abuse victims just as the FBI turned a blind eye to Larry Nassar’s abuse of dozens of child gymnasts for years.

Well, then it seems fairly obvious what Congress should do:

• Hold a hearing on the FBI’s failure to protect children (this did happen in September 2021)

But here's what EARN IT does in that situation:

• Amend Section 230, effectively delegating enforcement for child sexual abuse to states and victims themselves

As Riana summarizes:

No matter what the problem with online CSAM is, EARN IT isn’t going to fix it. It’s only going to make things worse, both for child victims and for everyone who uses the internet. The truth about EARN IT is that either there isn’t a serious noncompliance problem among providers that’s pervasive enough to merit a new law, but Congress just can’t resist using Section 230 as a political punching bag to harm all internet users in the name of sticking it to Big Tech… or there is a problem, but the DOJ is asleep at the wheel – and EARN IT is a concession that Congress no longer expects them to do their jobs.

Either option should be shameful and embarrassing for the bill’s supporters to admit. Instead, this horrible legislation, if it passes, will be hailed as a bipartisan victory that shows Congress can still come together across the aisle to get things done. Apparently, harming Americans’ rights online while making CSAM prosecutions harder is something both parties can agree on, even in an election year.

So, whatever problem the backers of EARN IT think they're solving for, EARN IT doesn't do it. That seems like it should be a big fucking deal. But, instead of responding to these points, the sponsors claim that people highlighting this "don't care about CSAM."

Filed Under: 2258a, csam, doj, earn it, encryption, fbi, reporting, surveillance
Companies: ncmec

How EARN IT Could Give Abusers A Get Out Of Jail Free Card: By Making Evidence Inadmissible

from the bad-ideas dept

In admitting that his EARN IT Act is really about attacking encryption, Senator Richard Blumenthal said he wouldn't agree to keep encryption out of the bill because he worried that it would give companies a "get-out-of-jail-free card." That's nonsense for multiple reasons, which we explained in that post, but the fact is Blumenthal's bill actually does contain a "get-out-of-jail-free card" that is incredibly damaging. It's one that child sexual abusers may be able to use to suppress any evidence collected against them and which would not just undermine the very point of EARN IT Act, but would make it that much harder to do the thing that needs to be done: stopping such abusers.

We touched on this a little bit in our earlier post about the mistakes senators made during the markup, but it's a little wonky, so it deserves a deeper exploration. Here's a good short description from Kir Nuthi in Slate:

As it stands, most companies that host online content voluntarily turn over huge amounts of potential evidence of child abuse to the National Center for Missing and Exploited Children. Because private companies search for this evidence voluntarily, courts have held that the searches are not subject to the Fourth Amendment. But the EARN IT Actthreatens to disrupt this relationship by using the threat of endless litigation and criminal prosecution to strongly pressure private companies to proactively search for illegal material. Thanks to how the EARN IT Act amends Section 230, companies are more exposed to civil and criminal liability if they don’t follow the government’s “or else” threat and search for child sexual abuse material.

Currently, tech platforms have an obligation to report but not search for suspected instances of child sexual abuse material. That’s why searches today are constitutional—they’re conducted voluntarily. By encouraging and pressuring private sector searches, the EARN IT Act casts doubt on every search—they’d no longer be voluntary. Thus, the Fourth Amendment would apply, and evidence collected without a warrant—all child sexual abuse material in this case, since private parties can’t get a warrant—would be at risk of exclusion from trial.

The Supreme Court has long held that when the government “encourages” private parties to search for evidence, those private parties become “government agents” subject to the Fourth Amendment and its warrant requirement. That means any evidence these companies collect could be ruled inadmissible in criminal trials against child predators because the evidence was procured unconstitutionally.

Put simply, thanks to the EARN IT Act, under theExclusionary Rule, defense attorneys could argue that evidence was collected in violation of the Fourth Amendment and should be excluded from trial. As a result, the bill could lead to fewer convictions of child predators, not more.

In short: under the current setup, companies can search for child sexual abuse material (CSAM) and if they find it they must report it to NCMEC (and remove it). This is good and useful and helps prevent the further spread. But under the 4th Amendment, if the government is mandating a search, then it would require a warrant before the search can happen. So, if the government mandates the search -- and as various senators made clear in both their "myths and facts" document, and in the markup hearing, that's exactly what they intend this bill to do -- then anyone who is charged with evidence found via such a search would have an unfortunately strong response that the evidence was collected under state action, and, as such in order to survive a 4th Amendment review, would require a warrant.

In other words, it hands terrible criminals -- those involved in the abuse of children -- a way to suppress the evidence used against them on 4th Amendment grounds. Under such a regime that would make it more difficult to prosecute actual criminals. But, even worse, it would then create a perverse and dangerous precedent in which companies would be greatly encouraged not to use basic scanning tools to find, remove, and report CSAM content, because in doing so, it would no longer be usable in prosecutions.

So the failure by senators to understand how the 4th Amendment works, means that EARN IT (beyond all its other problems) creates a constitutional mess that is, effectively (and almost literally) a "get-out-of-jail-free card" for criminals.

Filed Under: 4th amendment, csam, earn it, evidence, richard blumenthal, state action, suppression of evidence

The Top Ten Mistakes Senators Made During Today's EARN IT Markup

from the getting-everything-wrong dept

Today, the Senate Judiciary Committee unanimously approved the EARN IT Act and sent that legislation to the Senate floor. As drafted, the bill will be a disaster. Only by monitoring what users communicate could tech services avoid vast new liability, and only by abandoning, or compromising, end-to-end encryption, could they implement such monitoring. Thus, the bill poses a dire threat to the privacy, security and safety of law-abiding Internet users around the world, especially those whose lives depend on having messaging tools that governments cannot crack. Aiding such dissidents is precisely why it was the U.S. government that initially funded the development of the end-to-end encryption (E2EE) now found in Signal, Whatsapp and other such tools. Even worse, the bill will do the opposite of what it claims: instead of helping law enforcement crack down on child sexual abuse material (CSAM), the bill will actually help the most odious criminals walk free.

As with the July 2020 markup of the last Congress’s version of this bill, the vote was unanimous. This time, no amendments were adopted; indeed, none were even put up for a vote. We knew there wouldn’t be much time for discussion because Sen. Dick Durbin kicked off the discussion by noting that Sen. Lindsey Graham would have to leave soon for a floor vote. 

The Committee didn’t bother holding a hearing on the bill before rushing it to markup. The one and only hearing on the bill occurred just six days after its introduction back in March 2020. The Committee thereafter made major (but largely cosmetic) changes to the bill, leaving its Members more confused than ever about what the bill actually does. Today’s markup was a singular low-point in the history of what is supposed to be one of the most serious bodies in Congress. It showed that there is nothing remotely judicious about the Judiciary Committee; that most of its members have little understanding of the Internet and even less of how the, ahem, judiciary actually works; and, saddest of all, that they simply do not care.

Here are the top ten legal and technical mistakes the Committee made today.

Mistake #1: “Encryption Is not Threatened by This Bill”

Strong encryption is essential to online life today. It protects our commerce and our communications from the prying eyes of criminals, hostile authorian regimes and other malicious actors.

Sen. Richard Blumenthal called encryption a “red herring,” relying on his work with Sen. Leahy’s office to implement language from his 2020 amendment to the previous version of EARN IT (even as he admitted to a reporter that encryption was a target). Leahy’s 2020 amendment aimed to preserve companies’ ability to offer secure encryption in their products by providing that a company could not be found in violation of the law because it utilized secure encryption, doesn’t have the ability to decrypt communications, or fails to undermine the security of their encryption (for example, by building in a backdoor for use by law enforcement). 

But while the 2022 EARN IT Act contains the same list of protected activities, the authors snuck in new language that undermines that very protection. This version of the bill says that those activities can’t be an independent basis of liability, but that courts can consider them as evidence while proving the civil and criminal claims permitted by the bill’s provisions. That’s a big deal. EARN IT opens the door to liability under an enormous number of state civil and criminal laws, some of which require (or could require, if state legislatures so choose) a showing that a company was only reckless in its actions—a far lower showing than federal law’s requirement that a defendant have acted “knowingly.” If a court can consider the use of encryption, or failure to create security flaws in that encryption, as evidence that a company was “reckless,” it is effectively the same as imposing liability for encryption itself. No sane company would take the chance of being found liable for transmitting CSAM; they’ll just stop offering strong encryption instead. 

Mistake #2: The Bill’s Sponsors Readily Conceded that EARN IT Would Coerce Monitoring for CSAM

EARN IT’s sponsors repeatedly complained that tech companies aren’t doing enough to monitor for CSAM—and that their goal was to force them to do more. As Sen. Blumenthal noted, free software (PhotoDNA) makes it easy to detect CSAM, and it’s simply outrageous that some sites aren’t even using it. He didn’t get specific but we will: both Parler and Gettr, the alternative social networks favored by the MAGA right, have refused to use PhotoDNA. When asked about it, Parler’s COO told The Washington Post: “I don’t look for that content, so why should I know it exists?" The Stanford Internet Observatory’s David Thiel responded:

We agree completely—morally. So why, as Berin asked when EARN IT was first introduced, doesn’t Congress just directly mandate the use of such easy filtering tools? The answer lies in understanding why Parler and Gettr can get away with this today. Back in 2008, Congress required tech companies that become aware of CSAM to report it immediately to NCMEC, the quasi-governmental clearinghouse that administers the database of CSAM hashes used by PhotoDNA to identify known CSAM. Instead of requiring companies to monitor for CSAM, Congress said exactly the opposite: nothing in 18 U.S.C. § 2258A “shall be construed to require a provider to monitor [for CSAM].”

Why? Was Congress soft on child predators back then? Obviously not. Just the opposite: they understood that requiring tech companies to conduct searches for CSAM would make them state actors subject to the Fourth Amendment’s warrant requirement—and they didn’t want to jeopardize criminal prosecutions. 

Conceding that the purpose of EARN IT Act is to coerce searches for CSAM is a mistake, a colossal one, because it invites courts to rule that searching wasn’t voluntary.

Mistake #3: The Leahy Amendment Alone Won’t Protect Privacy & Security, or Avoid Triggering the Fourth Amendment

While Sen. Leahy’s 2020 amendment was a positive step towards protecting the privacy and security of online communications, and Lee’s proposal today to revive it is welcome, it was always an incomplete solution. While it protected companies against liability for offering encryption or failing to undermine the security of their encryption, it did not protect the refusal to conduct monitoring of user communications. A company offering E2EE products might still be coerced into compromising the security of its devices by scanning user communications “client-side” (i.e., on the device) prior to encrypting sent communications or after decrypting received communications. 

Apple recently proposed such a technology for such client-side scanning, raising concerns from privacy advocates and civil society groups. For its part, Apple assured that safeguards would limit use of the system to known CSAM to prevent the capability from being abused by foreign governments or rogue actors. But the capacity to conduct such surveillance presents an inherent risk of being exploited by malicious actors. Some companies may be able to successfully safeguard such surveillance architecture from misuse or exploitation. However, resources and approaches will vary across companies, and it is a virtual certainty that not all of them will be successful. And if done under coercion, create a risk that such efforts will be ruled state action requiring a warrant under the Fourth Amendment. 

Our letter to the Committee proposes an easy way to expand the Leahy amendment to ensure that companies won’t be held liable for not monitoring user content: borrow language directly from Section 2258A(f).

Mistake #4: EARN IT’s Sponsors Just Don’t Understand the Fourth Amendment Problem

Sen. Blumenthal insisted, repeatedly, that EARN IT contained no explicit requirement not to use encryption. The original version of the bill would, indeed, have allowed a commission to develop “best practices” that would be “required” as conditions of “earning” back the Section 230 immunity tech companies need to operate—hence the bill’s name. But dropping that concept didn’t really make the bill less coercive because the commission and its recommendations were always a sideshow. The bill has always coerced monitoring of user communications—and, to do that, the abandonment or bypassing of strong encryption—indirectly, through the threat of vast legal liability for not doing enough to stop the spread of CSAM. 

Blumenthal simply misunderstands how the courts assess whether a company is conducting unconstitutional warrantless searches as a “government actor.” “Even when a search is not required by law, … if a statute or regulation so strongly encourages a private party to conduct a search that the search is not ‘primarily the result of private initiative,’ then the Fourth Amendment applies.” U.S. v. Stevenson, 727 F.3d 826, 829 (8th Cir. 2013) (quoting Skinner v. Railway Labor Executives' Assn, 489 U.S. 602, 615 (1989)). In that case, the court found that AOL was not a government actor because it “began using the filtering process for business reasons: to detect files that threaten the operation of AOL's network, like malware and spam, as well as files containing what the affidavit describes as “reputational” threats, like images depicting child pornography.” AOL insisted that it “operate[d] its file-scanning program independently of any government program designed to identify either sex-offenders or images of child pornography, and the government never asked AOL to scan Stevenson's e-mail.” Id. By contrast, every time EARN IT’s supporters explain their bill, they make clear that they intend to force companies to search user communications in ways they’re not doing today.

Mistake #2 Again: EARN IT’s Sponsors Make Clear that Coercion Is the Point

In his opening remarks today, Sen. Graham didn’t hide the ball:

"Our goal is to tell the social media companies 'get involved and stop this crap. And if you don't take responsibility for what's on your platform, then Section 230 will not be there for you.' And it's never going to end until we change the game."

Sen. Chris Coons added that he is “hopeful that this will send a strong signal that technology companies … need to do more.” And so on and so forth.

If they had any idea what they were doing, if they understood the Fourth Amendment issue, these Senators would never admit that they’re using liability as a cudgel to force companies to take affirmative steps to combat CSAM. By making intentions unmistakable, they’ve given the most vile criminals exactly what they need to to challenge the admissibility of CSAM evidence resulting from companies “getting involved” and “doing more.” Though some companies, concerned with negative publicity, may tell courts that they conducted searches of user communications for “business reasons,” we know what defendants will argue: the companies’ “business reason” is avoiding the wide, loose liability that EARN IT subjected them to. EARN IT’s sponsors said so.

Mistake #5: EARN IT’s Sponsors Misunderstanding How Liability Would Work 

Except for Sen. Mike Lee, no one on the Committee seemed to understand what kind of liability rolling back Section 230 immunity, as EARN IT does, would create. Sen. Blumenthal repeatedly claimed that the bill requires actual knowledge. One of the bill’s amendments (the new Section 230(e)(6)(A)) would, indeed, require actual knowledge by enabling civil claims under 18 U.S.C. § 2255 “if the conduct underlying the claim constitutes a violation of section 2252 or section 2252A,” both of which contain knowledge requirements. This amendment is certainly an improvement over the original version of EARN IT, which would have explicitly allowed 2255 claims under a recklessness standard. 

But the two other changes to Section 230 clearly don’t require knowledge. As Sen. Lee pointed out today, a church could be sued, or even prosecuted, simply because someone posted CSAM on its bulletin board. Multiple existing state laws already create liability based on something less than actual knowledge of CSAM. As Lee noted, a state could pass a law creating strict liability for hosting CSAM. Allowing states to hold websites liable for recklessness (or even less) while claiming that the bill requires actual knowledge is simply dishonest. All these less-than-knowledge standards will have the same result: coercing sites into monitoring user communications, and into abandoning strong encryption as an obstacle to such monitoring. 

Blumenthal made it clear that this is precisely what he intends, saying: “Other states may wish to follow [those using the “recklessness” standard]. As Justice Brandeis said, states are the laboratories of democracy … and as a former state attorney general I welcome states using that flexibility. I would be loath to straightjacket them in their adoption of different standards.”

Mistake #6: “This Is a Criminal statute, This Is Not Civil Liability”

So said Sen. Lindsey Graham, apparently forgetting what his own bill says. Sen. Dianne Feinstein added her own misunderstanding, saying that she “didn’t know that there was a blanket immunity in this area of the law.” But if either of those statements were true, the EARN IT Act wouldn’t really do much at all. Section 230 has always explicitly carved out federal criminal law from its immunities; companies can already be charged for knowing distribution of child sexual abuse material (CSAM) or child sexual exploitation (CSE) under federal criminal statutes. Indeed, Backpage and its founders were criminally prosecuted even without SESTA’s 2017 changes to Section 230. If the federal government needs assistance in enforcing those laws, it could adopt Sen. Mike Lee’s amendment to permit state criminal prosecutions when the conduct would constitute a violation of federal law. Better yet, the Attorney General could use an existing federal law (28 U.S.C. § 543) to deputize state, local, and tribal prosecutors as “special attorneys” empowered to prosecute violations of federal law. Why no AG has bothered to do so yet is unclear.

What is clear is that EARN IT isn’t just about criminal law. EARN IT expressly carves out civil claims under certain federal statutes, and also under whatever state laws arguably relate to “the advertisement, promotion, presentation, distribution, or solicitation of child sexual abuse material” as defined by federal law. Those laws can and do vary, not only with respect to the substance of what is prohibited, but also the mental state required for liability. This expansive breadth of potential civil liability is part of what makes this bill so dangerous in the first place.

Mistake #7: “If They Can Censor Conservatives, They Can Stop CSAM!”

As at the 2020 markup, Sen. Lee seemed to understand most clearly how EARN IT would work, the Fourth Amendment problems it raises, and how to fix at least some of them. A former Supreme Court Clerk, Lee has a sharp legal mind, but he seems to misunderstand much of how the bill would work in practice, and how content moderation works more generally.

Lee complained that, if Big Tech companies can be so aggressive in “censoring” speech they don’t like, surely they can do the same for CSAM. He’s mixing apples and oranges in two ways. First, CSAM is the digital equivalent of radioactive waste: if a platform gains knowledge of it, it must take it down immediately and report it to NCMEC, and faces stiff criminal penalties if it doesn’t. And while “free speech” platforms like Parler and Gettr refuse to proactively monitor for CSAM (as discussed below), every mainstream service goes out of its way to stamp out CSAM on unencrypted service. Like AOL in the Stevenson case, they do so for business and reputational reasons.

By contrast no website even tries to block all “conservative” speech; rather, mainstream platforms must make difficult judgment calls about taking down politically charged content, such as Trump’s account only after he incited an insurrection in an attempted coup and misinformation about the 2020 election being stolen. Republicans are mad about where tech companies draw such lines.

Second, social media platforms can only moderate content that they can monitor. Signal can’t moderate user content and that is precisely the point: end-to-end-encryption means that no one other than the parties to a communication can see it. Unlike normal communications, which may be protected by lesser forms of “encryption,” the provider isn’t standing in the middle of the communication and it doesn’t have the keys to unlock the messages that it is passing back and forth. Yes, some users will abuse E2EE to share CSAM, but the alternative is to ban it for everyone. There simply isn’t a middle ground.

There may indeed be more that some tech companies could do about content they can see—both public content like social media posts and private content like messages (protected by something less than E2EE). But their being aggressive about, say, misinformation about COVID or the 2020 election has nothing whatsoever to do with the cold, hard reality that they can’t moderate content protected by strong encryption.

It’s hard to tell whether Lee understands these distinctions. Maybe not. Maybe he’s just looking to wave the bloody shirt of “censorship” again. Maybe he’s saying the same thing everyone else is saying, essentially: “Ah, yes, but if only Facebook, Apple and Google didn’t use end-to-end encryption for their messaging services, then they could monitor those for CSAM just like they monitor and moderate other content!” Proposing to amend the bill to require actual knowledge under both state and federal law suggests he doesn’t want this result, but who knows?

Mistake #8: Assuming the Fourth Amendment Won’t Require Warrants If It Applies

Visibility to the provider relates to one important legal distinction not discussed at all today—but that may well explain why the bill’s sponsors don’t seem to care about Fourth Amendment concerns. It’s an argument Senate staffers have used to defend the bill since its introduction. Even if compulsion through vast legal liability did make tech companies government actors, the Fourth Amendment requires a warrant only for searches of material for which users have a reasonable expectation of privacy. Kyllo v. United States, 533 U.S. 27, 33 (2001); see Katz v. United States, 389 U.S. 347, 361 (1967) (Harlan, J., concurring). Courts long held that users had no such expectations for digital messages like email held by third parties. 

But that began to change in 2010. If searches of emails trigger the Fourth Amendment—and U.S. v. Warshak, 631 F.3d 266 (6th Cir. 2010) said they do—searches of private messaging certainly would. The entire purpose of E2EE is to give users rock-solid expectations of privacy in their communications. More recently, the Supreme Court has said that, “given the unique nature of cell phone location records, the fact that the information is held by a third party does not by itself overcome the user's claim to Fourth Amendment protection.” Carpenter v. United States, 138 S. Ct. 2206, 2217 (2018). These cases draw the line Sen. Lee is missing: no, of course users don’t have reasonable expectations of privacy in public social media posts—which is what he’s talking about when he points to “censorship” of conservative speech. EARN IT could avoid the Fourth Amendment by focusing on content providers can see, but it doesn’t, because it’s intended to force companies to be able to see all user communications.

Mistake #9: What They didn’t Discuss: Anonymous Speech

The Committee didn’t discuss how EARN IT would affect speech protected by the First Amendment. No, of course CSAM isn’t protected speech, but the bill would affect lawful speech by law-abiding citizens—primarily by restricting anonymous speech. Critically, EARN IT doesn’t just create liability for trafficking in CSAM. The bill also creates liability for failing to stop communications that “solicit” or “promote” CSAM. Software like PhotoDNA can flag CSAM (by matching cryptographic hashes to known images in NCMEC’s database) but identifying “solicitation” or “promotion” is infinitely more complicated. Every flirtatious conversation between two adult users could be “solicitation” of CSAM—or it might be two adults doing adult things. (Adults sext each other—a lot. Get over it!) But “on the Internet, nobody knows you’re a dog”—and there’s no sure way to distinguish between adults and children. 

The federal government tried to do just that in the Communications Decency Act (CDA) of 1996 (nearly all of which, except Section 230, was struck down) and the Child Online Protection Act (COPA) of 1998. Both laws were struck down as infringing on the First Amendment right to accessing lawful content anonymously. EARN IT accomplishes much the same thing indirectly, the same way it attacks encryption: basing liability on anything less than knowledge means you can be sued for not actively monitoring, or for not age-verifying users, especially when the risks are particularly high (such as when you “should have known” you were dealing with minor users). 

Indeed, EARN IT is even more constitutionally suspect. At least COPA focused on content deemed “harmful to minors.” Instead of requiring age-gating for sites that offered porn and sex-related content (e.g., LGBTQ teen health), EARN IT would affect all users of private communications services, regardless of the nature of the content they access or exchange. Again, the point of E2EE is that the service provider has no way of knowing whether messages are innocent chatter or CSAM. 

EARN IT could raise other novel First Amendment problems. Companies could be held liable not only for failing to age-verify all users—a clear First Amendment violation— but also for failing to bar minors from using E2EE services so that their communications can be monitored or failing to use client-side monitoring on minors’ devices, and even failing to segregate adults from minors so they can’t communicate with each other. 

Without the Lee Amendment, EARN IT leaves states free to base liability on explicitly requiring age-verification or limits on what minors can do. 

Mistake #10: Claiming the Bill Is “Narrowly Crafted”

If you’ve read this far, Sen. Blumenthal’s stubborn insistence that this bill is a “narrowly targeted approach” should make you laugh—or sigh. If he truly believes that, either he hasn’t adequately thought about what this bill really does or he’s so confident in his own genius that he can simply ignore the chorus of protest from civil liberties groups, privacy advocates, human rights activists, minority groups, and civil society—all of whom are saying that this bill is bad policy.

If he doesn’t truly believe what he’s saying, well… that’s another problem entirely.

Bonus Mistake!: A Postscript About the Real CSAM problem

Lee never mentioned that the only significant social media services that don’t take basic measures to identify and block CSAM are Parler, Gettr and other fringe sites celebrated by Republicans as “neutral public fora” for “free speech.” Has any Congressional Republican sent letters to these sites asking why they refuse to use PhotoDNA? 

Instead, Lee did join Rep. Ken Buck in March 2021 to interrogate Apple about its decision to take down the Parler app. Answer: Parler hadn’t bothered setting any meaningful content moderation system. Only after Parler agreed to start doing some moderation of what appeared in its Apple app (but not its website) did Apple reinstate the app.

Filed Under: 1st amendment, 4th amendment, chris coons, content moderation, csam, dianne feinstein, e2ee, earn it, encryption, lindsey graham, mike lee, monitoring, pat leahy, photodna, richard blumenthal, section 230

Senator Wyden: EARN IT Will Make Children Less Safe

from the exactly-right dept

Earlier this week we wrote about the problematic reintroduction of the EARN IT Act and explained how it will make children a lot less safe -- exactly the opposite of what its backers claim. Senator Ron Wyden has now put out a statement that succinctly explains the problems of EARN IT, and exactly how it will do incredible harm to the very children it pretends to protect:

“This sadly misguided bill will not protect children. It will not stop the spread of vile child exploitation material or target the monsters that produce it. And it does not spend a single dollar to invest in prevention services for vulnerable children and youth or help victims and their families by providing evidence-based and trauma-informed resources. Instead, the EARN IT Act threatens the privacy and security of law-abiding Americans by targeting any form of private, secure devices and communication. As a result, the bill will make it easier for predators to track and spy on children and also harm the free speech and free expression of vulnerable groups,” Wyden said. “I have spent my career in the Senate fighting to protect kids and aid victims of abuse, and I will do everything in my power to ensure every single monster responsible for exploiting children or spreading horrific CSAM materials is prosecuted to the fullest extent of the law. But this bill does nothing to turn around the Justice Department’s tragic failure to prioritize child welfare and abuse cases.”

As Wyden notes, he introduced a bill that would put $5 billion towards actually fighting child sexual abuse, but for whatever reason that bill is going nowhere, while EARN IT is on the fast track.

Only one of those bills (Wyden's) actually moves us towards really fighting against child sexual exploitation. The other one grandstands and makes children less safe because it fails to understand technology or the law. Yet which one is Congress gearing up to support?

Filed Under: csam, earn it, ron wyden

Senators' 'Myths & Facts' About EARN IT Is Mostly Myths, Not Facts

from the they're-lying-to-you dept

I already wrote a long post earlier about the very very real problems with the EARN IT Act -- namely that it would make the problem of child sexual abuse material significantly worse by repeating the failed FOSTA playbook, and that it would attack encryption by making it potential "evidence" in a case against a tech company for any CSAM on its site. But with the bill, the sponsors of the bill, Senators Richard Blumenthal and Lindsey Graham, released a "Myth v. Fact" document to try to counter the criticisms of EARN IT. Unfortunately, the document presents an awful lot of "myths" as "facts." And that's a real problem.

The document starts out noting, correctly:

The reporting of CSAM and online child sexual exploitation provides law enforcement with vital information on active predators and ongoing cases of rape and abuse.

More reports, and more accurate information within those reports, means more children freed from one of the most horrific and life-changing crimes imaginable.

But it ignores that taking away Section 230 protections doesn't magically make them do more reporting. It does the opposite. Because now making the effort to find and report CSAM actually puts you at risk of greater liability under EARN IT. The bill literally creates less incentive for a website to build systems to find and report CSAM because merely doing so gives you the knowledge (scienter) necessary under the law to face liability. The bill gets this all exactly backwards.

The document then has the following listed as a "myth":

Given that some tech companies report significant amounts of CSAM to the National Center for Missing and Exploited Children (NCMEC) and provide technical resources to address child exploitation, the tech industry is doing enough to address this crime.

To address it, it lists these facts:

For tech companies that are already taking clear steps to report CSAM, little will change under this bill.

Except, that's not true at all. Because now, if those companies make any mistakes -- which they will because you can't get everything right -- they face potentially crippling liability. The idea that no one will go after them because they report a lot of CSAM is completely divorced from reality. We see companies getting sued all the time in similar circumstances. Under FOSTA now we're seeing companies like Salesforce and Mailchimp being sued because other companies used their services and then sex traffickers used those other services, and somehow that magically makes Salesforce and Mailchimp liable. The same thing would happen under EARN IT.

According to NCMEC’s 2020 statistics on reports of the online exploitation of children, while Facebook issued over 20 million reports that year, in contrast Amazon (which hosts a significant percentage of global commerce and web infrastructure) reported 2,235 cases.

Maybe that's because Facebook is a user-generated content social media platform and Amazon... is not? I mean, I don't even need to say this is "comparing apples to oranges" here because it's "comparing Facebook to Amazon." The two companies do very, very different things that are simply not comparable.

Of course, the underlying (and pretty fucking scary) suggestion here is that Amazon should be scanning every AWS instance for bad stuff, which raises really serious privacy concerns. It's amazing that the very same Senators pushing this bill, which now they're basically saying should require websites to spy on everything, will turn around next week and argue that these companies are "collecting too much information" and are engaged in "surveillance capitalism."

So which is it, Senators? Should Amazon be spying on everyone, or are they not spying hard enough?

There is a sustained problem of underreporting and neglect of legal obligations by some tech companies. During a Senate Judiciary Committee hearing on the EARN IT Act, NCMEC disclosed that it had reported nearly nine times more cases of CSAM material hosted on Amazon to Amazon, than Amazon had found itself, and that Amazon had not taken legally required action on those cases.

Again, this is taken incredibly out of context, and when put back into context means that Amazon isn't spying on all their customers' data. That should be seen as a good thing? Note what this "fact" doesn't say: when Amazon was alerted to CSAM by NCMEC did it remove it? Did it add hashes to the NCMEC database? Because that's what matters here. Otherwise, these Senators are just admitting that they want more surveillance by private companies and less privacy for the public.

Before introducing the EARN IT Act, a bipartisan group of Senators sent detailed questions to more than thirty of the most prominent tech companies. The responses showed that even startups and small firms were willing and able to build safety into their platform using automated tools. Meanwhile, some large companies like Amazon admitted that they were not even using common and free tools to automatically stop CSAM despite substantial and known abuse of their platforms by predators.

They're really throwing Amazon under the bus here. But this "fact" again demonstrates that most internet companies that host user generated content are already doing what is appropriate and finding, reporting, and removing CSAM. The only example they have of a company that is not is Amazon, and that's because Amazon is in a totally different business. They're not a platform for user generated content, they're just a giant computer for other services. Those other services, built on top of Amazon, can (and do!) scan their own systems for CSAM.

This whole "fact" list is basically a category error, in which they lump Amazon in with other companies because whoever wrote this can't find any actual problem out there with actual social media companies.

It is clear that many tech companies will only take CSAM seriously when it becomes their financial interest to do so, and the way to make that a reality is by permitting survivors and state law enforcement to take the companies to court for their role in child exploitation and abuse.

Except this document's own fact check said that every company they asked was doing what was necessary. And, it already is very much in every company's "financial interest" to find, report, and remove CSAM because if you don't you already face significant legal consequences, since hosting CSAM is already very, very much illegal.

The next "myth" listed is:

This bill opens up tech companies to new and unimaginable liability that necessitated CDA Section 230’s unqualified immunities two decades ago

Which is... absolutely true. And we don't need to look any further than what happened with FOSTA to see that this is true. But the Senators deny it. Because they're lying.

The EARN IT Act creates a targeted carve out for the specific, illegal act of possession or distribution of child sexual abuse material.

And FOSTA created "a targeted carve out for the specific, illegal act of human trafficking" but in practice has resulted in a series of totally frivolous lawsuits against ancillary services used by a company that was then used by sex traffickers.

Any tech company that is concerned that its services or applications could be used to distribute CSAM has plenty of tools and options available to prevent this crime without hindering their operations or creating significant costs.

The detection, prevention, and reporting of CSAM is one of the most easily addressed abuses and crimes in the digital era. There are readily accessible, and often free, software and cloud services, such as PhotoDNA, to automate the detection of known CSAM material and report it to NCMEC.

The naming of PhotoDNA is interesting here. It's a Microsoft project (big tech!) that is very important in finding/reporting/removing CSAM. But Microsoft actually limits who can use it, and I've heard of multiple websites that were not allowed to use PhotoDNA. I don't think Techdirt would qualify to use PhotoDNA, for example. In the meantime, Cloudflare actually introduced its own tool that I think came about because Microsoft made it difficult to impossible for many websites to use PhotoDNA.

But just the fact that PhotoDNA and Cloudflare's solution exist and are being used again suggests that "the problem" here doesn't actually exist. As noted in the first post, we don't see companies being sued for CSAM and using Section 230 as a defense, because that's not the problem.

Also, left out of the "fact" is the actual "fact" that PhotoDNA has very real limitations. That article, published a few months ago, notes that (1) Microsoft and NCMEC seem to go out of their way to avoid allowing researchers to study PhotoDNA, (2) contrary to Microsoft/NCMEC claims, the algorithm is able to be reversed (i.e., enabling users to recreate CSAM images from hashes!) (3) it is easily defeatable with minor changes to images and (4) it is subject to false positives. In other words, while PhotoDNA is an important tool for fighting CSAM, it has real problems, and mandating it (as this "fact" suggests is the goal of EARN IT) could create significant (and potentially dangerous) consequences.

The next "myth" is... just weird.

Requiring companies to be on the lookout for child abuse will harm startups and nascent businesses.

No one has made that argument. The actual argument is that adding very serious liability for anyone making any mistake as they're on the lookout for child abuse will do tremendous harm to startups and nascent businesses.

No other type of business in the country is provided such blanket and unqualified immunity for sexual crimes against children.

Except... tech companies aren't given a "blanket and unqualified immunity for sexual crimes against children." This "fact" is just wrong. What Section 230 does it provide immunity for third party speech -- but not if federal crimes are involved, which is certainly the case with CSAM. The whole attempt to blame Section 230 here is just weird. And wrong.

Startups and small businesses have a critical role in the fight against online CSAM. Smaller social media sites and messaging applications, such as Kik Messenger, are routinely used by abusers. The EARN IT Act will ensure that abusers do not flock to small platforms to evade the protections and accountability put in place on larger platforms.

So, now we're blaming Kik? Okay. Except the timing on this is interesting, as just a few days ago the DOJ literally announced that it had arrested a woman for distributing CSAM on Kik, showing again that when law enforcement actually bothers to do so, it can find and arrest those responsible.

Moreover, there are simple, readily accessible, and often free, software and cloud services, such as PhotoDNA, that can be used by any tech company to automate the detection of known CSAM material and report it to NCMEC.

Again, PhotoDNA involves a "qualification" process, and has significant problems. If the point of this bill is to force every website to use PhotoDNA, write that into the law and deal with the fact that a mandated filter raises other Constitutional concerns. Instead, these Senators are basically saying "every website must use PhotoDNA, but we can't legally say that, so wink, wink."

Indeed, it's pretty funny that right after more or less admitting that they're demanding mandatory filters, they claim this is the next "myth":

The EARN IT Act violates the First Amendment.

The "fact" they used to reply to this kinda gives away the game:

Child sexual abuse is not protected speech. Possession of child pornography is a criminal violation and there is no defensible claim that the First Amendment protects child sexual abuse material.

That's correct, but... misleading. No one is concerned about taking down CSAM (again, pretty much every major internet platform already does this as it's already required by law). The concern is that by mandating filters that are not publicly reviewable, you end up taking down other speech. And that other speech may be protected. Again, look at the link above regarding research into PhotoDNA, which suggests that the "false positive" problem with PhotoDNA is very, very real.

And then we get to the encryption stuff with the next "myth":

The EARN IT Act is simply an attempt to ban encryption.

Actually, it seems to be only partially an attempt to ban encryption. The Senators' "facts" on this is just... again, the part that is actually mythical:

The EARN IT Act does not target, limit, or create liability for encryption or privacy services. In fact, in order to ensure the EARN IT Act would not be misconstrued as limiting encryption, specific protections were included in the bill to explicitly state that a court should not consider offering encryption or privacy services as an independent basis for legal liability.

Weasel words. Again, see what I wrote in the last post about the encryption section. It says it can't be the "independent basis" for liability, but it explicitly states that the use of encryption can still be used as evidence against a website under this law. So it very much increases the legal liability for any website that uses encryption, because it will be used against them in court.

Stopping the abuse of children is not at odds with preserving online privacy. Some online platforms have been using automated tools to check images and videos against CSAM databases for more than a decade without endangering privacy or creating consumer concerns. As Facebook has testified to the Senate Judiciary Committee, tech companies can readily implement tools to detect child sexual abuse while offering strong encryption tools.

This is correct, but does not address the point. Of course, you can fight CSAM while preserving privacy, but this bill makes that much more difficult by adding liability risk to anyone who uses encryption (and anyone who tries to go above and beyond in fighting CSAM, but some slips through).

Then there's a "myth" that's actually a fact. Section 230 exempts federal crimes, and CSAM is a federal crime -- and the real issue is that law enforcement tends not to spend much time and resources on fighting the actual creators and distributors of CSAM:

Since CDA 230 already exempts federal crimes, the solution to this problem is increasing resources for law enforcement and hiring more federal prosecutors.

The "facts" the Senators present in response are incredibly misleading.

We support increasing resources for law enforcement officials fighting sex crimes against children. But no amount of money can compensate for the disengagement of the online platforms actually hosting this material.

That second sentence is a non sequitur, since (again...) EARN IT doesn't do anything to stop sites from hosting CSAM. It just opens them up to being sued for trying to stop it!

Hiring more federal investigators cannot replace having companies committed to the fight against child abuse, especially when it comes to monitoring the content posted on online platforms and checking closed groups for abuse.

Again, companies are committed to fighting child abuse, and EARN IT makes it more risky for them to "monitor" the content posted online!

By requiring that only the Department of Justice can bring criminal cases for child sexual exploitation crimes, CDA Section 230 drastically limits the number and types of cases that are brought.

No, it avoids bogus, wasteful lawsuits like the ones that were brought against Salesforce and Mailchimp under FOSTA.

States and survivors have a well-established role in holding offenders accountable, especially with respect to child sexual exploitation, for a reason: under enforcement of child protection laws fails victims and fosters more abuse.

Yes, and they can already hold "offenders accountable" for sexual exploitation. The problem is that this bill distracts from going after actual offenders, and instead blames random internet services that the offenders used for not magically knowing they were being used by offenders.

The EARN Act would ensure that there is more than one cop on the beat by enabling states and civil litigants to seek justice against those who enable child sexual exploitation.

No. It would allow that anyone can go after just about any website for incidental usage by an actual offender, rather than going after the offenders themselves.

This bill is bad and dangerous. It will make the very real problem of CSAM worse and undermine encryption at the same time. This "myth v. fact" sheet reverses the myths and facts in the services of getting bogus "for the children" headlines for Senators desperate to look like they're doing something about a real problem, while they're really moving to make the problem much, much worse.

Filed Under: csam, earn it, encryption, law enforcement, lindsey graham, photodna, richard blumenthal, section 230
Companies: amazon, facebook, kik, microsoft

Senate's New EARN IT Bill Will Make Child Exploitation Problem Worse, Not Better, And Still Attacks Encryption

from the it's-a-bad-bill dept

You may recall the terrible and dangerous EARN IT Act from two years ago, which was a push by Senators Richard Blumenthal and Lindsey Graham to chip away more at Section 230 and to blame tech companies for child sexual abuse material (CSAM). When it was initially introduced, many people noticed that it would undermine both encryption and Section 230 in a single bill. While the supporters of the bill insisted that it wouldn't undermine encryption, the nature of the bill clearly set things up so that you either needed to encrypt everything or to spy on everything. Eventually, the Senators were persuaded to adopt an amendment from Senator Patrick Leahy to more explicitly attempt to exempt encryption from the bill, but it was done in a pretty weak manner. That said, the bill still died.

But, as with 2020, 2022 is an election year, and in an election year some politicians just really want to get their name in headlines about how they're "protecting the children," and Senator Richard Blumenthal loves the fake "protecting the children" limelight more than most other Senators. And thus he has reintroduced the EARN IT Act, claiming (falsely) that it will somehow "hold tech companies responsible for their complicity in sexual abuse and exploitation of children." This is false. It will actually make it more difficult to stop child sexual abuse, but we'll get there. You can read the bill text here, and note that it is nearly identical to the version that came out of the 2020 markup process with the Leahy Amendment, with a few very minor tweaks. The bill has a lot of big name Senators as co-sponsors, and that's from both parties, suggesting that this bill has a very real chance of becoming law. And that would be dangerous.

If you want to know just how bad the bill is, I found out about the re-introduction of the bill -- before it was announced anywhere else -- via a press release sent to me by NCOSE, formerly "morality in media," the busybody organization of prudes who believe that all pornography should be banned. NCOSE was also a driving force behind FOSTA -- the dangerous law with many similarities to EARN IT that (as we predicted) did nothing to stop sex trafficking, and plenty of things to increase the problem of sex trafficking, while putting women in danger and making it more difficult for the police to actually stop trafficking.

Amusingly (?!?) NCOSE's press release tells me both that without EARN IT tech platforms "have no incentive to prevent" CSAM, and that in 2019 tech platforms reported 70 million CSAM images to NCMEC. They use the former to insist that the law is needed, and the latter to suggest that the problem is obviously out of control -- apparently missing the fact that the latter actually shows how the platforms are doing everything they can to stop CSAM on their platforms (and others!) by following existing laws and reporting it to NCMEC where it can be put into a hash database and shared and blocked elsewhere.

But facts are not what's important here. Emotions, headlines, and votes in November are.

Speaking of the lack of facts necessary, with the bill, they also have a "myth v. fact" sheet which is just chock full of misleading and simply incorrect nonsense. I'll break that down in a separate post, but just as one key example, the document really leans heavily on the fact that Amazon sends a lot fewer reports of CSAM to NCMEC than Facebook does. But, if you think for more than 3 seconds about it (and aren't just grandstanding for headlines) you might notice that Facebook is a social media site and Amazon is not. It's comparing two totally different types of services.

However, for this post I want to focus on the key problems of EARN IT. In the very original version of EARN IT, the bill created a committee to study if exempting CSAM from Section 230 would help stop CSAM. Then it shifted to the same form it's in now where the committee still exists, but they skip the part where the committee has to determine if chipping away at 230 will help, and just includes that as a key part of the bill. The 230 part mimics FOSTA (again which has completely failed to do what it claimed and has made the actual problems worse), in that it adds a new exemption to Section 230 that exempts any CSAM from Section 230.

EARN IT will make the CSAM problem much, much worse.

At least in the FOSTA case, supporters could (incorrectly and misleadingly, as it turned out) point to Backpage as an example of a site that had been sued for trafficking and used Section 230 to block the lawsuit. But here... there's nothing. There really aren't* examples of websites using Section 230 to try to block claims of child sexual abuse material*. So it's not even clear what problem these Senators think they're solving (unless the problem is "not enough headlines during an election year about how I'm protecting the children.")

* Update: After publishing this story, a lawyer pointed out a couple of examples of cases, so I'll qualify the original statement to say there really aren't any credible examples of cases where Section 230 blocked legitimate complaints. The main example the lawyer sent over was a case we had written about just a week earlier regarding a clear nuisance lawsuit against Omegle, in which someone was trying to sue the company because creepy people used the site. This is actually a perfect example of why Section 230 is so important, because it kills such nuisance suits quickly. Without it, the range of internet services would be greatly diminished. Another case involves a recent one against Twitter, which again actually demonstrates the importance and benefits of Section 230, in that people are trying to hold Twitter responsible for the actions of users, rather than going after the actual perpetrators.

The best they can say is that companies need the threat of law to report and takedown CSAM. Except, again, pretty much every major website that hosts user content already does this. This is why groups like NCOSE can trumpet "70 million CSAM images" being reported to NCMEC. Because all of the major internet companies actually do what they're supposed to do.

And here's where we get into one of the many reasons this bill is so dangerous. It totally misunderstands how Section 230 works, and in doing so (as with FOSTA) it is likely to make the very real problem of CSAM worse, not better. Section 230 gives companies the flexibility to try different approaches to dealing with various content moderation challenges. It allows for greater and greater experimentation and adjustments as they learn what works -- without fear of liability for any "failure." Removing Section 230 protections does the opposite. It says if you do anything, you may face crippling legal liability. This actually makes companies less willing to do anything that involves trying to seek out, take down, and report CSAM because of the greatly increased liability that comes with admitting that there is CSAM on your platform to search for and deal with.

EARN IT gets the problem exactly backwards. It disincentivizes action by companies, because the vast majority of actions will actually increase rather than decrease liability. As Eric Goldman wrote two years ago, this version of EARN IT doesn't penalize companies for CSAM, it penalizes them for (1) not magically making all CSAM disappear, for (2) knowing too much about CSAM (i.e., telling them to stop looking for it and taking it down) or (3) not exiting the industry altogether (as we saw a bunch of dating sites do post FOSTA).

EARN IT is based on the extremely faulty assumption that internet companies don't care about CSAM and need more incentive to do so, rather than the real problem, which is that CSAM has always been a huge problem and stopping it requires actual law enforcement work focused on the producers of that content. But by threatening internet websites with massive liability if they make a mistake, it actually makes law enforcement's job harder, because they will be less able to actually work with law enforcement. This is not theoretical. We already saw exactly this problem with FOSTA, in which multiple law enforcement agencies have said that FOSTA made their job harder because they can no longer find the information they need to stop sex traffickers. EARN IT creates the exact same problem for CSAM.

So the end result is that by misunderstanding Section 230, by misunderstanding internet company's existing willingness to fight CSAM, EARN IT will undoubtedly make the CSAM problem worse by making it more difficult for companies to track CSAM down and report it, and more difficult for law enforcement to track down an arrest those actually responsible for it. It's a very, very bad and dangerous bill -- and that's before we even get to the issue of encryption!

EARN IT is still very dangerous for encryption

EARN IT supporters claim they "fixed" the threat to encryption in the original bill by using text similar to Senator Leahy's amendment to say that using encryption cannot "serve as an independent basis for liability." But, the language still puts encryption very much at risk. As we've seen, the law enforcement/political class is very quick to want to (falsely) blame encryption for CSAM. And by saying that encryption cannot serve as "an independent basis" for liability, that still leaves open the door to using it as one piece of evidence in a case under EARN IT.

Indeed, one of the changes to the bill from the one in 2020 is that immediately after saying encryption can't be an independent basis for liability it adds a new section that wasn't there before that effectively walks back the encryption-protecting stuff. The new section says: "Nothing in [the part that says encryption isn't a basis for liability] shall be construed to prohibit a court from considering evidence of actions or circumstances described in that subparagraph if the evidence is otherwise admissable." In other words, as long as anyone bringing a case under EARN IT can point to something that is not related to encryption, it can point to the use of encryption as additional evidence of liability for CSAM on the platform.

Again, the end result is drastically increasing liability for the use of encryption. While no one will be able to use the encryption alone as evidence, as long as they point to one other thing -- such as a failure to find a single piece of CSAM -- then they can bring the encryption evidence back in and suggest (incorrectly) some sort of pattern or willful blindness.

And this doesn't even touch on what will come out of the "committee" and its best practices recommendations, which very well might include an attack on end-to-end encryption.

The end result is that (1) EARN IT is attacking a problem that doesn't exist (the use Section 230 to avoid responsibility for CSAM) (2) EARN IT will make the actual problem of CSAM worse by making it much more risky for internet companies to fight CSAM and (3) EARN IT puts encryption at risk by potentially increasing the liability risk of any company that offers encryption.

It's a bad and dangerous bill and the many, many Senators supporting it for kicks and headlines should be ashamed of themselves.

Filed Under: csam, earn it, encryption, fosta, lindsey graham, richard blumenthal, section 230

Report: Client-Side Scanning Is An Insecure Nightmare Just Waiting To Be Exploited By Governments

from the 14-out-of-14-cybersecurity-experts-agree dept

In August, Apple declared that combating the spread of CSAM (child sexual abuse material) was more important than protecting millions of users who've never used their devices to store or share illegal material. While encryption would still protect users' data and communications (in transit and at rest), Apple had given itself permission to inspect data residing on people's devices before allowing it to be sent to others.

This is not a backdoor in a traditional sense. But it can be exploited just like an encryption backdoor if government agencies want access to devices' contents or mandate companies like Apple do more to halt the spread of other content governments have declared troublesome or illegal.

Apple may have implemented its client-side scanning carefully after weighing the pros and cons of introducing a security flaw, but there's simply no way to engage in this sort of scanning without creating a very large and slippery slope capable of accommodating plenty of unwanted (and unwarranted) government intercession.

Apple has put this program on hold for the time being, citing concerns raised by pretty much everyone who knows anything about client-side scanning and encryption. The conclusions that prompted Apple to step away from the precipice of this slope (at least momentarily) have been compiled in a report [PDF] on the negative side effects of client-side scanning, written by a large group of cybersecurity and encryption experts (Hal Abelson, Ross Anderson, Steven M. Bellovin, Josh Benaloh, Matt Blaze, Jon Callas, Whitfield Diffie, Susan Landau, Peter G. Neumann, Ronald L. Rivest, Jeffrey I. Schiller, Bruce Schneier, Vanessa Teague, and Carmela Troncoso). (via The Register)

Here's how that slippery slope looks. Apple's client-side scanning may be targeted, utilizing hashes of known CSAM images, but once the process is in place, it can easily be repurposed.

Only policy decisions prevent the scanning expanding from illegal abuse images to other material of interest to governments; and only the lack of a software update prevents the scanning expanding from static images to content stored in other formats, such as voice, text, or video.

And if people don't think governments will demand more than Apple's proactive CSAM efforts, they haven't been paying attention. CSAM is only the beginning of the list of content governments would like to see tech companies target and control.

While the Five Eyes governments and Apple have been talking about child sex-abuse material (CSAM) —specifically images— in their push for CSS, the European Union has included terrorism and organized crime along with sex abuse. In the EU’s view, targeted content extends from still images through videos to text, as text can be used for both sexual solicitation and terrorist recruitment. We cannot talk merely of “illegal” content, because proposed UK laws would require the blocking online of speech that is legal but that some actors find upsetting.

Once capabilities are built, reasons will be found to make use of them. Once there are mechanisms to perform on-device censorship at scale, court orders may require blocking of nonconsensual intimate imagery, also known as revenge porn. Then copyright owners may bring suit to block allegedly infringing material.

That's just the policy and law side. And that's only a very brief overview of clearly foreseeable expansions of CSS to cover other content, which also brings with it concerns about it being used as a tool for government censorship. Apple has already made concessions to notoriously censorial governments like China's in order to continue to sell products and services there. Additional demands will obviously be made if Apple implements scanning that can be exploited to locate and censor critics of the government.

There's plenty of bad stuff on the technical side, too. CSS is pretty much malware, the report says:

CSS is at odds with the least-privilege principle. Even if it runs in middleware, its scope depends on multiple parties in the targeting chain, so it cannot be claimed to use least-privilege in terms of the scanning scope. If the CSS system is a component used by many apps, then this also violates the least-privilege principle in terms of scope. If it runs at the OS level, things are worse still, as it can completely compromise any user’s device, accessing all their data, performing live intercept, and even turning the device into a room bug.

CSS has difficulty meeting the open-design principle, particularly when the CSS is for CSAM, which has secrecy requirements for the targeted content. As a result, it is not possible to publicly establish what the system actually does, or to be sure that fixes done in response to attacks are comprehensive. Even a meaningful audit must trust that the targeted content is what it purports to be, and so cannot completely test the system and all its failure modes.

Finally, CSS breaks the psychological-acceptability principle by introducing a spy in the owner’s private digital space. A tool that they thought was theirs alone, an intimate device to guard and curate their private life, is suddenly doing surveillance on behalf of the police. At the very least, this takes the chilling effect of surveillance and brings it directly to the owner’s fingertips and very thoughts.

While the report does offer some suggestions on how to make scanning less exploitable, the downsides are too numerous to conclude this can somehow be done safely. Given how many intrusive surveillance programs have already been justified with concerns about terrorism or the spread of illicit material, CSS -- no matter how implemented -- with become a tempting tool for governments to exploit.

In a world where our personal information lies in bits carried on powerful communication and storage devices in our pockets, both technology and laws must be designed to protect our privacy and security, not intrude upon it. Robust protection requires technology and law to complement each other. Client-side scanning would gravely undermine this, making us all less safe and less secure.

Despite this comprehensive report warning against the implementation of client-side scanning, there's a chance Apple may still roll its version out. And once it does, the pressure will be on other companies to do at least as much as Apple is doing to combat CSAM. The only upside is that if governments decide scanning should be used for reasons other than Apple intends, it has the power to shut its system down.

Filed Under: client side scanning, csam, iphones, security
Companies: apple

Apple Recognizes It Jumped Too Quickly On Its CSAM Detection System; Delays Implementation

from the good dept

Sometimes speaking out works. A month ago, Apple announced a series of new offerings that it claimed would be useful in fighting back against CSAM (child sexual abuse material). This is a real problem, and it's commendable that Apple was exploring ways to fight it. However, the major concern was how Apple had decided to do this. Despite the fact that a ton of experts have been working on ways to deal with this extremely challenging problem, Apple (in Apple fashion) went it alone and just jumped right in the deep end, causing a lot more trouble than necessary -- both because their implementation had numerous serious risks that Apple didn't seem to account for, and (perhaps more importantly) because the plan could wipe away years of goodwill in conversations between technologists, security professionals, human rights advocates and more in trying to seek solutions that better balance the risks.

Thankfully, with much of the security community, the human rights community, and others calling attention to Apple's dangerous approach, the company has now announced a plan to delay the implementation, gather more information, and actually talk to experts before deciding how to move forward. Apple put (in tiny print...) an update on the page where it announced these features.

Update as of September 3, 2021: Previously we announced plans for features intended to help protect children from predators who use communication tools to recruit and exploit them and to help limit the spread of Child Sexual Abuse Material. Based on feedback from customers, advocacy groups, researchers, and others, we have decided to take additional time over the coming months to collect input and make improvements before releasing these critically important child safety features.

It's good that the company has finally realized that it moved too quickly on this and hadn't necessarily understood the ramifications of its decision. It remains to be seen whether the company will actually do more in terms of realizing how dangerous its approach was, or if it will simply look to make a few cosmetic changes to the system.

Notably, this announcement came out just as Scientific American released an interesting article that warns that one of the child safety features -- one that had received less concern than the others -- might harm children more than it helps. This is the "communication safety in messages" feature that would scan iMessages of kids under 13, blur messages that the system deemed sexually explicit, and alert parents if the kid sends or opens such a message.

There were some initial concerns about this -- especially regarding LGTBQ children whose parents might not be understanding. However, many of those initial concerns were quieted by the details of the program -- including the fact that it was opt-in and was designed to be transparent to both the kids and the parents what was happening (so no sneaky surveillance or surprise alerts). It's also specifically designed for child accounts that are set up in Family Sharing.

However, as the SciAm article notes, this system still should raise concerns, because it would make kids think they're always being watched:

In fact, even by having this feature, we are teaching young people that they do not have a right to privacy. Removing young people’s privacy and right to give consent is exactly the opposite of what UNICEF’s evidence-based guidelines for preventing online and offline child sexual exploitation and abuse suggest. Further, this feature not only risks causing harm, but it also opens the door for wider intrusions into our private conversations, including intrusions by government.

We need to do better when it comes to designing technology to keep the young safe online. This starts with involving the potential victims themselves in the design of safety systems. As a growing movement around design justice suggests, involving the people most impacted by a technology is an effective way to prevent harm and design more effective solutions. So far, youth haven’t been part of the conversations that technology companies or researchers are having. They need to be.

Again, there are real concerns here, and parents obviously want to protect their children. But over and over again we've seen the way you do that is by teaching them how to handle dangerous situations, rather than adding yet another layer of surveillance. The surveillance teaches them not only that they have no privacy, but similarly takes away their own agency in learning how to deal with difficult situations themselves.

Filed Under: children, client side scanning, csam, scanning, security
Companies: apple

Researchers Who Built Similar System Explain Why Apple's CSAM Scanning System Is Dangerous

from the it's-not-good dept

Jonathan Mayer, a Princeton University professor and former chief technologist at the FTC, is one of the smartest people I know. Every time I've spoken with him I feel like I learn something. He's now written a quite interesting article for the Washington Post noting how he, and a graduate researcher at Princeton, Anunay Kulshrestha, actually built a CSAM scanning system similar to the one that Apple recently announced, which has security experts up in arms over the risks inherent to the approach.

Mayer and Kulshretha note that while Apple is saying that people worried about their system are misunderstanding it, they are not. They know what they're talking about -- and they still say the system is dangerous.

We wrote the only peer-reviewed publication on how to build a system like Apple’s — and we concluded the technology was dangerous. We’re not concerned because we misunderstand how Apple’s system works. The problem is, we understand exactly how it works.

Mayer and Kulshretha are certainly not making light of the challenges and problems associated with stopping CSAM. It's why they started their project to see if there was an effective way to identify CSAM even in end-to-end encrypted systems. And they even built a system to do just that. And what they found is that the risks are simply too great.

We were so disturbed that we took a step we hadn’t seen before in computer science literature: We warned against our own system design, urging further research on how to mitigate the serious downsides. We’d planned to discuss paths forward at an academic conference this month.

That dialogue never happened. The week before our presentation, Apple announced it would deploy its nearly identical system on iCloud Photos, which exists on more than 1.5 billion devices. Apple’s motivation, like ours, was to protect children. And its system was technically more efficient and capable than ours. But we were baffled to see that Apple had few answers for the hard questions we’d surfaced.

The potential dangers that Mayer and Kulshretha are exactly what many had warned about when Apple announced its plans:

After many false starts, we built a working prototype. But we encountered a glaring problem.

Our system could be easily repurposed for surveillance and censorship. The design wasn’t restricted to a specific category of content; a service could simply swap in any content-matching database, and the person using that service would be none the wiser.

A foreign government could, for example, compel a service to out people sharing disfavored political speech. That’s no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials.

We spotted other shortcomings. The content-matching process could have false positives, and malicious users could game the system to subject innocent users to scrutiny.

That's already pretty damning. But there's some other even more damning information that has come out as well. As we noted in our earlier posts (and as Mayer and Kulshretha noted in their research), the risk of false positives is extremely high. And late last week, the hypothetical became a lot more real. Someone reverse engineered the NeuralHash algorithm that Apple is using and put it on Github.

It did not take long for people to point that, first of all, there are collisions with totally unrelated images:

Now, you might say that since that second image is not really an image, maybe it's not that big a deal? Well, about that...

And, of course, one of the issues with any hash-based system is the idea that subtle changes to the image would output a totally different hash, thus making it easy for some to simply route around the scanning. Apple had suggested its system could defeat that, but...

This is part of the reason that we highlighted earlier that security researchers are so up in arms about this. Apple seemingly ignored so much of the research and conversations that were happening about these approaches, and just barged right in announcing that it had a solution without exploring the tradeoffs and difficulties associated with it -- leaving that for security experts to point out afterwards.

Apple is trying to downplay these findings, saying that it expected the collisions at least, and that it's system would also do a separate server side hashing comparison which would stop the false collisions. Though, as Bruce Schneier points out, if this was "expected," then why wasn't it discussed in the initial details that were released? Similarly, I have yet to see a response to the flip side issue of changing the images in a way that fool NeuralHash while still looking the same.

I know Apple keeps wanting to insist that it's thought through all of this, but it doesn't seem to have thought through any of how the security community would see all of this, and it's after-the-fact scrambling is not exactly reassuring.

Filed Under: backdoors, client side scanning, csam, encryption, hash clash, jonathan mayer, surveillance
Companies: apple