Facebook AI Moderation Continues To Suck Because Moderation At Scale Is Impossible

from the car-washes-and-mass-shootings dept

For several years now, we've been beating the idea that content moderation at scale is impossible to get right, otherwise known as Masnick's Impossibility Theorem. The idea there is not that platforms shouldn't do any form of moderation, or that they shouldn't continue to try to improve the method for moderation. Instead, this is all about expectations setting, partially for a public that simply wants better content to show up on their various devices, but even more so for political leaders that often see a problem happening on the internet and assume that the answer is simply "moar tech!".

Being an internet behemoth, Facebook catches a lot of heat for when its moderation practices suck. Several years ago, Mark Zuckerberg announced that Facebook had developed an AI-driven moderation program, alongside the claim that this program would capture "the vast majority" of objectionable content. Anyone who has spent 10 minutes on Facebook in the years since realizes how badly Facebook failed towards that goal. And, as it turns out, failed in both directions.

By that I mean that, while much of our own commentary on all this has focused on how often Facebook's moderation ends up blocking non-offending content, a recent Ars Technica post on just how much hate speech makes its way onto the platform has some specific notes about how some of the most objectionable content is misclassified by the AI moderation platform.

Facebook’s internal documents reveal just how far its AI moderation tools are from identifying what human moderators were easily catching. Cockfights, for example, were mistakenly flagged by the AI as a car crash. “These are clearly cockfighting videos,” the report said. In another instance, videos livestreamed by perpetrators of mass shootings were labeled by AI tools as paintball games or a trip through a carwash.

It's not entirely clear to me just why the AI system is seeing mass shootings and animals fighting and thinking its paintball or carwashes, though I unfortunately have some guesses and they aren't fun to think about. Either way, this... you know... sucks! If the AI you're relying on to filter out extreme and violent content labels a mass shooting as a trip through the carwash, well, that really should send us back to the drawing board, shouldn't it?

It's worse in other countries, as the Ars post notes. There are countries where Facebook has no database of racial slurs in native languages, meaning it cannot even begin blocking such content on the site, via AI or otherwise. Polled Facebook users routinely identify hate on the platform as its chief problem, but the company seems to be erring in the opposite direction.

Still, Facebook’s leadership has been more concerned with taking down too many posts, company insiders told WSJ. As a result, they said, engineers are now more likely to train models that avoid false positives, letting more hate speech slip through undetected.

Which may actually be the right thing to do. I'm not prepared to adjudicate that point in this post. But what we can say definitively is that Facebook has an expectations setting problem on its hands. For years it has touted its AI and human moderators as the solution to the most vile content on its platform... and it doesn't work. Not at scale at least. And outside of America and a handful of other Western nations, barely at all.

It might be time for the company to just say so and tell the public and its representatives that this is going to take a long, long while before the company gets this anywhere close to right.

Filed Under: ai, content moderation, impossibility
Companies: facebook

Arlo Makes Live Customer Service A Luxury Option

from the basic-competency-is-extra dept

The never-ending quest for improved quarterly returns means that things that technically shouldn't be luxury options, inevitably wind up being precisely that. We've shown how a baseline expectation of privacy is increasingly treated as a luxury option by hardware makers and telecoms alike. The same thing also sometimes happens to customer service; at least when companies think they can get away with it.

"Smart home" and home security hardware vendor Arlo, for example, has announced a number of new, not particularly impressive subscription tiers for its internet-connected video cameras. The changes effectively involve forcing users to pay more money every month if they ever want to talk to a live customer service representative. From Stacey Higginbotham:

"This week, Arlo launched what I generously think of as its pay-for-customer-service enticement for its smart home camera products. As of Oct. 4, customers without a subscription who’ve had their devices for more than 90 days no longer get phone support. And after one year, they lose access to live chat support."

If you don't pay Arlo more money for actual customer service, you're relegated to cobbling together support solutions from the company's forums, an automated website chat bot, or elsewhere. Given the cost of Arlo products, the decision to make speaking to an actual human being a $3 to $15 monthly add on is fairly ludicrous:

"Arlo’s customer support framework now requires a $2.99 to $14.99 per month Arlo subscription, a free trial plan, or the device to be within 90 days of purchase for phone support. Then you’re downgraded to chat support for the remainder of the year.

After that, absent a plan, Arlo customers with problems will only have access to a virtual assistant or the public forums. That means no phone support and no chat. This feels pretty punitive for a product that can cost between $130 and $300 depending on the device."

Even U.S. telecom giants, the poster children for atrocious U.S. customer service, haven't meaningfully pursued making live customer support a premium option (though they have tinkered with innately providing worse support to folks with low credit scores). Arlo's choice comes amidst higher shipping costs and supply chain issues during COVID, but the decision to try and recover those higher costs by making basic competency a luxury tier will likely come back to bite it in an IOT space that's only getting more competitive.

Filed Under: customer support, home security, iot, subscription fees
Companies: arlo

Delta Proudly Announces Its Participation In The DHS's Expanded Biometric Collection Program

from the WE-LOVE-YOU-GOVERNMENT-[girlish-screams] dept

Via Travel & Leisure comes this warning -- one the online magazine has decided to portray as exciting news.

Delta Air Lines is expanding its partnership with the Transportation Security Administration with its use of facial recognition technology making getting through airport security even quicker.

The airline is implementing a "digital identity experience" at its hub in Atlanta, offering customers with TSA PreCheck and a Delta SkyMiles number the chance to pass through security and board their flight without having to pull out a boarding pass or their ID.

Ah, the "digital identity experience." That's apparently DeltaSpeak for "biometric collection and facial recognition deployment." That's been the DHS's plan all along. It may have pretended it only wanted to post up at ports of entry (i.e., international airports) but it is going to roll this out to as many airports as possible.

How do we know "digital identity experience" is Delta PR? Because this "article" is largely a regurgitation of Delta's own press release about its increased biometric collection/domestic surveillance efforts.

First unveiled in Detroit security checkpoints in early 2021, Delta’s digital identity experience is an industry first in exclusive partnership with TSA PreCheck. The experience is expanding to Atlanta, offering customers a more efficient way to navigate the airport – without showing a paper boarding pass or a physical government ID. With just one look at a camera, customers who qualify and opt in can easily and efficiently check a bag, pass through the TSA PreCheck security line and board their plane.

You'll notice this "experience" is first being offered to PreCheck participants. This means people who have already registered and paid the government to buy back some of their Constitutional rights are the first to be invited to help expand Custom and Border Protection's biometric database.

Delta is just the "partner." The TSA just mans the front end. The data and subsequent scans belong to the CBP.

Once a customer reaches a camera at the airport, their image is encrypted and sent to U.S. Customs and Border Protection’s (CBP) facial biometric matching service via a secure channel with no accompanying biographic data. CBP then verifies a customer’s identity against government holdings and sends back an indicator to allow the customer to proceed.

There are several things wrong with all of this, starting with Travel & Leisure's cheery stenography of cheery Delta's up-sale of government intrusion. While no one expects a leisure mag to get serious about the implications of expanding biometric collections and facial recognition programs to encompass US citizens traveling domestically, the article could have used less of the Delta's GO TEAM USA jargon and more actual facts: like how facial recognition AI -- all of it -- is fundamentally flawed. Or maybe just point out the unjustified but steady encroachment of surveillance tech into airports serving millions of travelers the government has no reason to suspect are up to no good.

And Travel & Leisure may want to vet its sources. After all, Delta is more than happy to pretend things aren't the way they are as it pitches in to make traveling a worse experience for its customers.

Atlanta’s domestic terminal south security checkpoint is the first in the U.S. that will be converted to computed tomography-automated screening lane (CT-ASL) systems - making the world’s busiest airport even more efficient as travelers connect to destinations around the world.

That's simply not true. Increased efficiency may be the end goal but a recently released report by the DHS Inspector General says CT scanners are actually slowing down the screening process.

TSA deployed CT systems to airport passenger screening checkpoints that did not meet minimum throughput requirements. TSA’s February 2018 Operational Requirements Document identified the need for a CT system capable of screening, on average, 200 items per hour to successfully perform the mission. However, we determined TSA purchased 300 CT systems capable of screening an average of 170 items per hour — 15 percent less than the minimum requirement, and less than the AT X-ray system capability of approximately 354 items per hour.

This is a human centipede. The TSA says some shit. Delta Airlines reposts it with its own happy spin. And, at the tail end of it, consumer-facing sites are swallowing everything and dumping it on to web pages without bothering to question the sources or provide any information that might counterbalance the government's assertions about how more intrusive collections make America a better place to call home.

Filed Under: biometric collection, facial recognition, security theater, surveillance, tsa
Companies: delta

Daily Deal: The Python, Git, And YAML Bundle

from the good-deals-on-cool-stuff dept

The Python, Git, And YAML Bundle has 9 courses to help you learn all about Python, YAML, and Git. Five courses cover Python programming from the beginner level to more advanced concepts. Three courses cover Git and how to use it for your personal projects. The final course introduces you to the YAML fundamentals. The bundle is on sale for $29.

Note: The Techdirt Deals Store is powered and curated by StackCommerce. A portion of all sales from Techdirt Deals helps support Techdirt. The products featured do not reflect endorsements by our editorial team.

Filed Under: daily deal

Report: Client-Side Scanning Is An Insecure Nightmare Just Waiting To Be Exploited By Governments

from the 14-out-of-14-cybersecurity-experts-agree dept

In August, Apple declared that combating the spread of CSAM (child sexual abuse material) was more important than protecting millions of users who've never used their devices to store or share illegal material. While encryption would still protect users' data and communications (in transit and at rest), Apple had given itself permission to inspect data residing on people's devices before allowing it to be sent to others.

This is not a backdoor in a traditional sense. But it can be exploited just like an encryption backdoor if government agencies want access to devices' contents or mandate companies like Apple do more to halt the spread of other content governments have declared troublesome or illegal.

Apple may have implemented its client-side scanning carefully after weighing the pros and cons of introducing a security flaw, but there's simply no way to engage in this sort of scanning without creating a very large and slippery slope capable of accommodating plenty of unwanted (and unwarranted) government intercession.

Apple has put this program on hold for the time being, citing concerns raised by pretty much everyone who knows anything about client-side scanning and encryption. The conclusions that prompted Apple to step away from the precipice of this slope (at least momentarily) have been compiled in a report [PDF] on the negative side effects of client-side scanning, written by a large group of cybersecurity and encryption experts (Hal Abelson, Ross Anderson, Steven M. Bellovin, Josh Benaloh, Matt Blaze, Jon Callas, Whitfield Diffie, Susan Landau, Peter G. Neumann, Ronald L. Rivest, Jeffrey I. Schiller, Bruce Schneier, Vanessa Teague, and Carmela Troncoso). (via The Register)

Here's how that slippery slope looks. Apple's client-side scanning may be targeted, utilizing hashes of known CSAM images, but once the process is in place, it can easily be repurposed.

Only policy decisions prevent the scanning expanding from illegal abuse images to other material of interest to governments; and only the lack of a software update prevents the scanning expanding from static images to content stored in other formats, such as voice, text, or video.

And if people don't think governments will demand more than Apple's proactive CSAM efforts, they haven't been paying attention. CSAM is only the beginning of the list of content governments would like to see tech companies target and control.

While the Five Eyes governments and Apple have been talking about child sex-abuse material (CSAM) —specifically images— in their push for CSS, the European Union has included terrorism and organized crime along with sex abuse. In the EU’s view, targeted content extends from still images through videos to text, as text can be used for both sexual solicitation and terrorist recruitment. We cannot talk merely of “illegal” content, because proposed UK laws would require the blocking online of speech that is legal but that some actors find upsetting.

Once capabilities are built, reasons will be found to make use of them. Once there are mechanisms to perform on-device censorship at scale, court orders may require blocking of nonconsensual intimate imagery, also known as revenge porn. Then copyright owners may bring suit to block allegedly infringing material.

That's just the policy and law side. And that's only a very brief overview of clearly foreseeable expansions of CSS to cover other content, which also brings with it concerns about it being used as a tool for government censorship. Apple has already made concessions to notoriously censorial governments like China's in order to continue to sell products and services there. Additional demands will obviously be made if Apple implements scanning that can be exploited to locate and censor critics of the government.

There's plenty of bad stuff on the technical side, too. CSS is pretty much malware, the report says:

CSS is at odds with the least-privilege principle. Even if it runs in middleware, its scope depends on multiple parties in the targeting chain, so it cannot be claimed to use least-privilege in terms of the scanning scope. If the CSS system is a component used by many apps, then this also violates the least-privilege principle in terms of scope. If it runs at the OS level, things are worse still, as it can completely compromise any user’s device, accessing all their data, performing live intercept, and even turning the device into a room bug.

CSS has difficulty meeting the open-design principle, particularly when the CSS is for CSAM, which has secrecy requirements for the targeted content. As a result, it is not possible to publicly establish what the system actually does, or to be sure that fixes done in response to attacks are comprehensive. Even a meaningful audit must trust that the targeted content is what it purports to be, and so cannot completely test the system and all its failure modes.

Finally, CSS breaks the psychological-acceptability principle by introducing a spy in the owner’s private digital space. A tool that they thought was theirs alone, an intimate device to guard and curate their private life, is suddenly doing surveillance on behalf of the police. At the very least, this takes the chilling effect of surveillance and brings it directly to the owner’s fingertips and very thoughts.

While the report does offer some suggestions on how to make scanning less exploitable, the downsides are too numerous to conclude this can somehow be done safely. Given how many intrusive surveillance programs have already been justified with concerns about terrorism or the spread of illicit material, CSS -- no matter how implemented -- with become a tempting tool for governments to exploit.

In a world where our personal information lies in bits carried on powerful communication and storage devices in our pockets, both technology and laws must be designed to protect our privacy and security, not intrude upon it. Robust protection requires technology and law to complement each other. Client-side scanning would gravely undermine this, making us all less safe and less secure.

Despite this comprehensive report warning against the implementation of client-side scanning, there's a chance Apple may still roll its version out. And once it does, the pressure will be on other companies to do at least as much as Apple is doing to combat CSAM. The only upside is that if governments decide scanning should be used for reasons other than Apple intends, it has the power to shut its system down.

Filed Under: client side scanning, csam, iphones, security
Companies: apple