Every Streaming Company Not Named Apple Receives A Lousy Grade On Privacy

from the unimportant-afterthoughts dept

While streaming providers and hardware companies see significantly higher consumer satisfaction rates that traditional cable TV, their privacy practices still leave something to be desired. That's according to a new breakdown of streaming service privacy policies by Common Sense Media, which doled out terrible grades to pretty much everybody not named Apple:

"Our privacy evaluations of the top 10 streaming apps indicate that all streaming apps (except Apple TV+) have privacy practices that put consumers' privacy at considerable risk including selling data, sending third‐party marketing communications, displaying targeted advertisements, tracking users across other sites and services, and creating advertising profiles for data brokers."

Of course their privacy practices suck because being terrible on privacy is perfectly legal in a country that can't manage to pass even a basic privacy law for the internet era. Not a law cracking down on the dodgy behavior by third party data brokers. Not a law that implements some basic transparency requirements so consumers know what's being collected and who it's being sold to. And not a basic law that implements something vaguely resembling real accountability for companies that can't be bothered, time and time again, to properly secure their networks, devices, and servers from intrusion.

The firm took a deep dive into the data collection and sharing practices of 10 different streaming services and hardware vendors, from Netflix and Disney+ to Roku. They closely examined what data was being collected, how much of that collection was transparent to the end user, and where that data was sent. They also took a close look at device security using the Consumer Reports' Digital Standard, a promising new metric that attempts to standardize security practices so they can be included in product reviews.

Some of the failures were downright ugly, like making no real exceptions for the data collection of children. Many of the issues revealed weren't the end of the world, but they make it repeatedly clear that companies aren't being transparent about what is collected, and often enjoy making opting out of data collection and monetization as cumbersome and annoying as possible:

"The Roku Streaming Stick+ set‐up process did not display notice of privacy settings or choices a user can make about sharing data. Only after the Roku device has completed setup can users navigate to the "Settings" menu, and choose to opt in to the single privacy setting "Limit Ad Tracking" which is not enabled by default. This setting is worded in such a way that it may be misleading that opting in to limiting a worse practice is actually opting out of use of your data for that worse practice, which is not a principle of privacy by design."

There's a consistent drumbeat of arguments that tend to focus on how it's simply impossible for our Luddite Congress to pass a useful privacy law. But again, it wouldn't be at that difficult to write a clear law, with input from major stakeholders, that simply mandates clear communication on what's being collected and how to opt out of it. But we can't even do that. Instead we've got complete apathy toward the privacy and security internet shit show on the federal level, which created a discordant wave of less than impressive and often convoluted state-level laws.

Filed Under: data collection, privacy, streaming
Companies: apple

Apple Recognizes It Jumped Too Quickly On Its CSAM Detection System; Delays Implementation

from the good dept

Sometimes speaking out works. A month ago, Apple announced a series of new offerings that it claimed would be useful in fighting back against CSAM (child sexual abuse material). This is a real problem, and it's commendable that Apple was exploring ways to fight it. However, the major concern was how Apple had decided to do this. Despite the fact that a ton of experts have been working on ways to deal with this extremely challenging problem, Apple (in Apple fashion) went it alone and just jumped right in the deep end, causing a lot more trouble than necessary -- both because their implementation had numerous serious risks that Apple didn't seem to account for, and (perhaps more importantly) because the plan could wipe away years of goodwill in conversations between technologists, security professionals, human rights advocates and more in trying to seek solutions that better balance the risks.

Thankfully, with much of the security community, the human rights community, and others calling attention to Apple's dangerous approach, the company has now announced a plan to delay the implementation, gather more information, and actually talk to experts before deciding how to move forward. Apple put (in tiny print...) an update on the page where it announced these features.

Update as of September 3, 2021: Previously we announced plans for features intended to help protect children from predators who use communication tools to recruit and exploit them and to help limit the spread of Child Sexual Abuse Material. Based on feedback from customers, advocacy groups, researchers, and others, we have decided to take additional time over the coming months to collect input and make improvements before releasing these critically important child safety features.

It's good that the company has finally realized that it moved too quickly on this and hadn't necessarily understood the ramifications of its decision. It remains to be seen whether the company will actually do more in terms of realizing how dangerous its approach was, or if it will simply look to make a few cosmetic changes to the system.

Notably, this announcement came out just as Scientific American released an interesting article that warns that one of the child safety features -- one that had received less concern than the others -- might harm children more than it helps. This is the "communication safety in messages" feature that would scan iMessages of kids under 13, blur messages that the system deemed sexually explicit, and alert parents if the kid sends or opens such a message.

There were some initial concerns about this -- especially regarding LGTBQ children whose parents might not be understanding. However, many of those initial concerns were quieted by the details of the program -- including the fact that it was opt-in and was designed to be transparent to both the kids and the parents what was happening (so no sneaky surveillance or surprise alerts). It's also specifically designed for child accounts that are set up in Family Sharing.

However, as the SciAm article notes, this system still should raise concerns, because it would make kids think they're always being watched:

In fact, even by having this feature, we are teaching young people that they do not have a right to privacy. Removing young people’s privacy and right to give consent is exactly the opposite of what UNICEF’s evidence-based guidelines for preventing online and offline child sexual exploitation and abuse suggest. Further, this feature not only risks causing harm, but it also opens the door for wider intrusions into our private conversations, including intrusions by government.

We need to do better when it comes to designing technology to keep the young safe online. This starts with involving the potential victims themselves in the design of safety systems. As a growing movement around design justice suggests, involving the people most impacted by a technology is an effective way to prevent harm and design more effective solutions. So far, youth haven’t been part of the conversations that technology companies or researchers are having. They need to be.

Again, there are real concerns here, and parents obviously want to protect their children. But over and over again we've seen the way you do that is by teaching them how to handle dangerous situations, rather than adding yet another layer of surveillance. The surveillance teaches them not only that they have no privacy, but similarly takes away their own agency in learning how to deal with difficult situations themselves.

Filed Under: children, client side scanning, csam, scanning, security
Companies: apple

Colorado Transportation Officals Asked Navigation App Providers To Plant False Information. Worse, The Providers Complied.

from the slippery-slope-of-faking-slippery-slopes dept

Well, this isn't cool. Colorado transportation officials fed bogus information to map apps to make an open road appear to be closed.

Hoping to keep traffic from rerouting to a smaller road after a larger highway was closed due to rockslides, the Colorado Department of Transportation did this:

[T]he Colorado Department of Transportation marked the road as closed on its travelers update site http://www.cotrip.org because of a “safety closure due to mudslide,” and Pitkin County Undersheriff Alex Burchetta said Wednesday afternoon the county sent out an alert about 3:30 p.m. as such, based on that information.

However, there were not any mudslides and the messaging “evolved” and was changed by CDOT, Burchetta said. A CDOT spokesperson confirmed there were no slides.

That affected the DOT's own site, which is itself problematic. Drivers depend on that information being accurate. Falsifying reports for the purpose of controlling traffic flow shouldn't be considered acceptable.

But that wasn't the only travel information outlet affected by the DOT's shady traffic shaping.

Gregg Miller, a CDOT business process architect, was tasked with contacting the navigation services when agency officials were desperately trying to prevent motorists from flooding Highway 82 during the closure of Interstate 70 through Glenwood Canyon because of mudslides and the ensuing damage.

Traffic levels were hitting an estimated 7,000 to 9,000 vehicles per day during the week of Aug. 1 compared to a normal load of 1,000 vehicles per day, one official estimated.

John Lorme, CDOT director of maintenance and operations, directed Miller via email on Aug. 4 at 11:49 a.m. to get the roads closed on the navigation services.

“I need this to show closed to traffic on the mapping apps, soonest,” Lorme wrote. “I will assume responsibility. All locals understand what’s going on. It’s the (commercial vehicle) and (recreational vehicle) traffic that is creating hazardous conditions.”

CDOT is a "trusted partner" with multiple navigation app providers, allowing it to directly feed traffic information to these companies. But there's nothing trustworthy about feeding false information to popular consumer apps. It seems if the DOT wanted to close a road or limit its traffic, it had plenty of options that didn't involve delivering false information to drivers via map apps and the DOT's own website.

Making this worse was the DOT's decision to maintain the illusion of a road closure on consumer apps while updating its own site to reflect the actual facts.

Miller was successful in getting Google, Waze, Apple and TomTom to show Highway 82 as closed on Aug. 4. However, CDOT executive director Shoshana Lew insisted that evening that the agency keep Highway 82 marked as open on cotrip.org, the agency’s real-time road status app.

This doesn't fix the problem. Drivers are more likely to rely on navigation apps than government websites when dealing with travel complications. Efforts like this diminish trust -- both of the apps drivers use and the government that's supposed to be serving them.

Just as worrying was these companies' agreement to participate in the ruse. Communications obtained with public records requests appear to show Google and Apple knew they were being asked to plant false information in their map offerings.

The next morning, Miller wrote to his supervisor, CDOT chief engineer Stephen Harelson, to express his concerns. Miller said maintenance and operations personnel had directed him to contact Google, Waze and Apple the prior day to ask them to “show Independence Pass closed for traffic routing purposes for the entire month of August.”

“We are currently listed as a ‘trusted partner’ with these services and while they questioned this (request), I explained to them that CDOT is concerned about the traffic levels on the road and they need to be closed,” Miller wrote. “They did it but questioned why COTRIP showed Independence Pass as open.”

Google's statement appears to indicate it's willing to plant fake information if asked to do so by government agencies.

"When official changes are made to restrict certain routes, we update our directions accordingly.”

Apparently that includes showing a road is closed when it actually isn't and listing a nonexistent hazard (mudslide) as the reason for the (fake) closure.

Obviously, nothing can really prevent government officials from straight up lying about road conditions to map app providers. But this fiasco involved not only the planting of false information by government officials, but the active participation of navigation app providers. This is a huge abuse of trust by all parties involved -- something that could very well lead to drivers ignoring road closure warnings in the future and putting themselves in danger.

Filed Under: cdot, colorado, fake information, navigation, traffic, traffic routing, transportation
Companies: apple, google, tomtom, waze

Apple's Dedication To Privacy Is Missing When It Comes To Its Workforce, Employees Say

from the consistently-inconsistent dept

For a while it's been clear that Apple has been hoping to use "we care about privacy" as a marketing advantage in an internet economy that consistently... doesn't. For example Apple has stood up to the FBI when it comes to backdooring encryption. And in 2020 Apple unveiled several new privacy features and policies it hoped would differentiate it from its other "big tech" contemporaries repeatedly under fire in the press and in Congress. Granted there's been ongoing hints this dedication isn't exactly consistent, but it's the thought that counts, I guess.

But Apple employees say the company's outward-facing dedication to privacy isn't reflected in house. This week The Verge reported on how the company's decision to force Apple employees to link their personal Apple ID and work Apple accounts are creating significant privacy headaches for employees. Personal Apple ID accounts are obviously filled with all manner of sensitive personal and financial data that an employee might not want shared with an employer. And the requirement that employees fuse their personal and work lives in such a fashion is causing significant internal turmoil, the report claims:

"...some employees say the company isn’t doing enough to protect their personal privacy and, at times, actively seeks to invade it for security reasons. Employees have been asked to install software builds on their phones to test out new features prior to launch — only to find the builds expose their personal messages. Others have found that when testing new products like Apple’s Face ID, images are recorded every time they open their phones. “If they did this to a customer, people would lose their goddamn minds,” says Ashley Gjøvik, a senior engineering program manager."

Apple devices don’t support using multiple Apple IDs, and while you can technically use several phones (say, an Android), that winds up being cumbersome and difficult if you want to actually get work done. The forced integration is just a bizarre disconnect from the company's recent pro-privacy branding. Especially given the requirements have had a stunning privacy impact on the very people Apple expects to build privacy-friendly products and services:

So in short Apple is demanding that employees fuse extremely personal accounts with oceans of sensitive data with their work accounts. While at the same time conducting significant internal surveillance, and making it extremely clear employees should have absolutely no expectation of privacy (very much the status quo at most companies):

"Underpinning all of this is a stringent employment agreement that gives Apple the right to conduct extensive employee surveillance, including “physical, video, or electronic surveillance” as well as the ability to “search your workspace such as file cabinets, desks, and offices (even if locked), review phone records, or search any non-Apple property (such as backpacks, purses) on company premises."

The story makes it very clear Apple hasn't tried very hard to make any of this any easier. Untangling your personal and work data when you leave Apple also winds up being a nightmare, given that Apple bans employees from wiping their company-owned devices (including the personal data that gets tangled up with work) when they leave the company, opening them to legal action for violating their employment agreement if they do.

So in short Apple's marketing insists that when it comes to privacy the company is ethical and different, but when it comes to the people actually tasked with building the products that adhere to those standards -- these values suddenly and mysteriously disappear.

Filed Under: apple id, employment, privacy, work life balance, work phone
Companies: apple

Researchers Who Built Similar System Explain Why Apple's CSAM Scanning System Is Dangerous

from the it's-not-good dept

Jonathan Mayer, a Princeton University professor and former chief technologist at the FTC, is one of the smartest people I know. Every time I've spoken with him I feel like I learn something. He's now written a quite interesting article for the Washington Post noting how he, and a graduate researcher at Princeton, Anunay Kulshrestha, actually built a CSAM scanning system similar to the one that Apple recently announced, which has security experts up in arms over the risks inherent to the approach.

Mayer and Kulshretha note that while Apple is saying that people worried about their system are misunderstanding it, they are not. They know what they're talking about -- and they still say the system is dangerous.

We wrote the only peer-reviewed publication on how to build a system like Apple’s — and we concluded the technology was dangerous. We’re not concerned because we misunderstand how Apple’s system works. The problem is, we understand exactly how it works.

Mayer and Kulshretha are certainly not making light of the challenges and problems associated with stopping CSAM. It's why they started their project to see if there was an effective way to identify CSAM even in end-to-end encrypted systems. And they even built a system to do just that. And what they found is that the risks are simply too great.

We were so disturbed that we took a step we hadn’t seen before in computer science literature: We warned against our own system design, urging further research on how to mitigate the serious downsides. We’d planned to discuss paths forward at an academic conference this month.

That dialogue never happened. The week before our presentation, Apple announced it would deploy its nearly identical system on iCloud Photos, which exists on more than 1.5 billion devices. Apple’s motivation, like ours, was to protect children. And its system was technically more efficient and capable than ours. But we were baffled to see that Apple had few answers for the hard questions we’d surfaced.

The potential dangers that Mayer and Kulshretha are exactly what many had warned about when Apple announced its plans:

After many false starts, we built a working prototype. But we encountered a glaring problem.

Our system could be easily repurposed for surveillance and censorship. The design wasn’t restricted to a specific category of content; a service could simply swap in any content-matching database, and the person using that service would be none the wiser.

A foreign government could, for example, compel a service to out people sharing disfavored political speech. That’s no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials.

We spotted other shortcomings. The content-matching process could have false positives, and malicious users could game the system to subject innocent users to scrutiny.

That's already pretty damning. But there's some other even more damning information that has come out as well. As we noted in our earlier posts (and as Mayer and Kulshretha noted in their research), the risk of false positives is extremely high. And late last week, the hypothetical became a lot more real. Someone reverse engineered the NeuralHash algorithm that Apple is using and put it on Github.

It did not take long for people to point that, first of all, there are collisions with totally unrelated images:

Now, you might say that since that second image is not really an image, maybe it's not that big a deal? Well, about that...

And, of course, one of the issues with any hash-based system is the idea that subtle changes to the image would output a totally different hash, thus making it easy for some to simply route around the scanning. Apple had suggested its system could defeat that, but...

This is part of the reason that we highlighted earlier that security researchers are so up in arms about this. Apple seemingly ignored so much of the research and conversations that were happening about these approaches, and just barged right in announcing that it had a solution without exploring the tradeoffs and difficulties associated with it -- leaving that for security experts to point out afterwards.

Apple is trying to downplay these findings, saying that it expected the collisions at least, and that it's system would also do a separate server side hashing comparison which would stop the false collisions. Though, as Bruce Schneier points out, if this was "expected," then why wasn't it discussed in the initial details that were released? Similarly, I have yet to see a response to the flip side issue of changing the images in a way that fool NeuralHash while still looking the same.

I know Apple keeps wanting to insist that it's thought through all of this, but it doesn't seem to have thought through any of how the security community would see all of this, and it's after-the-fact scrambling is not exactly reassuring.

Filed Under: backdoors, client side scanning, csam, encryption, hash clash, jonathan mayer, surveillance
Companies: apple

Apple's New Scanning Tools Raising More Concerns, Even Inside Apple

from the take-a-step-back dept

Last week, we wrote about our concerns about Apple's newly announced scanning efforts that the company claimed were to protect children. Lots of security experts raised concerns about how this was being rolled out -- and none of the complaints were meant to take away from the very real and legitimate concerns about child sexual abuse. Security guru Alex Stamos wrote one of the most thoughtful threads about the whole thing, noting (as with so many of these issues) that there are no easy answers here. I highly recommend you read the entire thread, but here's a little snippet:

Similar to many of these debates involving nuanced tech policy issues with no easy answers, a big part of the problem here is, as Stamos notes in his thread, there are tons of conversations happening about the nuances and tradeoffs, and even though Apple's approach is not as disastrous and dangerous as it could have been (i.e., clearly a lot of thought was put by the team at Apple into minimizing many -- though not all -- of the risks here), this approach was still done without talking to the many, many people who have been trying to find a reasonable balance here. And that messes a lot of stuff up.

Stamos, along with computer science professor/security guy Matt Green, have now published a good piece in the NY Times highlighting their concerns. The article notes there is less concern about the iMessage child safety features (Apple's initial description of that seemed much more concerning, but the details show why it's not that bad). But the photo scanning on the phone raises a lot of concerns:

But the other technology, which allows Apple to scan the photos on your phone, is more alarming. While Apple has vowed to use this technology to search only for child sexual abuse material, and only if your photos are uploaded to iCloud Photos, nothing in principle prevents this sort of technology from being used for other purposes and without your consent. It is reasonable to wonder if law enforcement in the United States could compel Apple (or any other company that develops such capacities) to use this technology to detect other kinds of images or documents stored on people’s computers or phones.

While Apple is introducing the child sexual abuse detection feature only in the United States for now, it is not hard to imagine that foreign governments will be eager to use this sort of tool to monitor other aspects of their citizens’ lives — and might pressure Apple to comply. Apple does not have a good record of resisting such pressure in China, for example, having moved Chinese citizens’ data to Chinese government servers. Even some democracies criminalize broad categories of hate speech and blasphemy. Would Apple be able to resist the demands of legitimately elected governments to use this technology to help enforce those laws?

Another worry is that the new technology has not been sufficiently tested. The tool relies on a new algorithm designed to recognize known child sexual abuse images, even if they have been slightly altered. Apple says this algorithm is extremely unlikely to accidentally flag legitimate content, and it has added some safeguards, including having Apple employees review images before forwarding them to the National Center for Missing and Exploited Children. But Apple has allowed few if any independent computer scientists to test its algorithm.

The computer science and policymaking communities have spent years considering the kinds of problems raised by this sort of technology, trying to find a proper balance between public safety and individual privacy. The Apple plan upends all of that deliberation. Apple has more than one billion devices in the world, so its decisions affect the security plans of every government and every other technology company. Apple has now sent a clear message that it is safe to build and use systems that directly scan people’s personal phones for prohibited content.

In a separate thread, Stamos has a suggested path forward for Apple, which involves pumping the brakes quite a bit on some of these features.

Meanwhile, Reuters revealed on Thursday that inside Apple there are widespread concerns as well.

Apple employees have flooded an Apple internal Slack channel with more than 800 messages on the plan announced a week ago, workers who asked not to be identified told Reuters. Many expressed worries that the feature could be exploited by repressive governments looking to find other material for censorship or arrests, according to workers who saw the days-long thread.

Past security changes at Apple have also prompted concern among employees, but the volume and duration of the new debate is surprising, the workers said. Some posters worried that Apple is damaging its leading reputation for protecting privacy.

The article notes that many of the concerns are coming from outside of the security team at Apple -- suggesting that the concerns are more about perception than they are technical. But, really, this highlights the same problem that Stamos noted earlier: Apple's standard operating procedure of doing everything alone, and then also doing "surprise" announcements regarding products. That's great for a new gadget in your pocket. It's not so great for dealing with a massively challenging and very legitimate problem with no easy answers, where getting things even a little wrong can have significant negative consequences.

Unlike many companies that rush out offerings that do more harm than good, I do think that Apple did think this through internally with lots of smart and thoughtful people. But these are problems and challenges that go beyond just one company -- and Apple's famously insular approach is exactly the wrong thing for this sort of challenge.

Filed Under: backdoors, csam, encryption, icloud, iphones, privacy, scanning, security, surveillance
Companies: apple

Apple Undermines Its Famous Security 'For The Children'

from the this-is-dangerously-dumb dept

Apple is somewhat famous for its approach to security on its iPhones. Most famously, Apple went to court to fight the FBI's demand that they effectively insert a backdoor into its on-phone encryption (by being able to force an update to the phone). Apple has tons of goodwill in the security community (and the public) because of that, though not in the law enforcement community. Unfortunately, it appears that Apple is throwing away much of that good will and has decided to undermine the security of its phone... "for the children" (of course).

This week, Apple announced what it refers to as "expanded protections for children." The company has been receiving lots of pressure from certain corners (including law enforcement groups who hate encryption), claiming that its encryption was helping hide child sexual abuse material (CSAM) on phones (and in iCloud accounts). So Apple's plan is to introduce what's generally called "client-side scanning" to search for CSAM on phones as well as a system that scans iCloud content for potentially problematic content. Apple claims that it's doing this in a manner that is protective of privacy. And, to be fair, this clearly isn't something that Apple rolled out willy-nilly without considering the trade-offs. It's clear from Apple's detailed explanations of the new "safety" features, that it is trying to balance the competing interests at play here. And, obviously, stopping the abuse of children is an important goal.

The problem is that, even with all of the balancing Apple has done here, it's definitely moved down a very dangerous, and very slippery slope towards using this approach for other things.

Apple's brief description of its new offerings are as follows:

Apple is introducing new child safety features in three areas, developed in collaboration with child safety experts. First, new communication tools will enable parents to play a more informed role in helping their children navigate communication online. The Messages app will use on-device machine learning to warn about sensitive content, while keeping private communications unreadable by Apple.

Next, iOS and iPadOS will use new applications of cryptography to help limit the spread of CSAM online, while designing for user privacy. CSAM detection will help Apple provide valuable information to law enforcement on collections of CSAM in iCloud Photos.

Finally, updates to Siri and Search provide parents and children expanded information and help if they encounter unsafe situations. Siri and Search will also intervene when users try to search for CSAM-related topics.

Some of the initial concerns about these descriptions -- including fears that, say, LGBTQ+ children might be outed to their parents -- have been somewhat (though not entirely) alleviated with the more detailed explanation. But that doesn't mean there aren't still very serious concerns about how this plays out in practice and what this means for Apple's security.

First, there's the issue of client-side scanning. As an EFF post from 2019 explains, client-side scanning breaks end-to-end encryption. In the EFF's latest post about Apple's announcement, it includes a quick description of how this introduces a backdoor:

We’ve said it before, and we’ll say it again now: it’s impossible to build a client-side scanning system that can only be used for sexually explicit images sent or received by children. As a consequence, even a well-intentioned effort to build such a system will break key promises of the messenger’s encryption itself and open the door to broader abuses.

All it would take to widen the narrow backdoor that Apple is building is an expansion of the machine learning parameters to look for additional types of content, or a tweak of the configuration flags to scan, not just children’s, but anyone’s accounts. That’s not a slippery slope; that’s a fully built system just waiting for external pressure to make the slightest change. Take the example of India, where recently passed rules include dangerous requirements for platforms to identify the origins of messages and pre-screen content. New laws in Ethiopia requiring content takedowns of “misinformation” in 24 hours may apply to messaging services. And many other countries—often those with authoritarian governments—have passed similar laws. Apple’s changes would enable such screening, takedown, and reporting in its end-to-end messaging. The abuse cases are easy to imagine: governments that outlaw homosexuality might require the classifier to be trained to restrict apparent LGBTQ+ content, or an authoritarian regime might demand the classifier be able to spot popular satirical images or protest flyers.

We’ve already seen this mission creep in action. One of the technologies originally built to scan and hash child sexual abuse imagery has been repurposed to create a database of “terrorist” content that companies can contribute to and access for the purpose of banning such content. The database, managed by the Global Internet Forum to Counter Terrorism (GIFCT), is troublingly without external oversight, despite calls from civil society. While it’s therefore impossible to know whether the database has overreached, we do know that platforms regularly flag critical content as “terrorism,” including documentation of violence and repression, counterspeech, art, and satire.

It's actually difficult to find any security experts who support Apple's approach here. Alec Muffett sums it up in a single tweet:

This is the very slippery slope. If we somehow believe that governments won't demand Apple cave on a wide variety of other types of content, you haven't been paying attention. Of course, Apple can claim that it will stand strong against such demands, but now we're back to being entirely dependent on trusting Apple.

As noted above, there were some initial concerns about the parent notifications, but as EFF's description notes, the rollout here does include some level of consent by users before their parents are notified, but it's still quite problematic:

In these new processes, if an account held by a child under 13 wishes to send an image that the on-device machine learning classifier determines is a sexually explicit image, a notification will pop up, telling the under-13 child that their parent will be notified of this content. If the under-13 child still chooses to send the content, they have to accept that the “parent” will be notified, and the image will be irrevocably saved to the parental controls section of their phone for the parent to view later. For users between the ages of 13 and 17, a similar warning notification will pop up, though without the parental notification.

Similarly, if the under-13 child receives an image that iMessage deems to be “sexually explicit”, before being allowed to view the photo, a notification will pop up that tells the under-13 child that their parent will be notified that they are receiving a sexually explicit image. Again, if the under-13 user accepts the image, the parent is notified and the image is saved to the phone. Users between 13 and 17 years old will similarly receive a warning notification, but a notification about this action will not be sent to their parent’s device.

This means that if—for instance—a minor using an iPhone without these features turned on sends a photo to another minor who does have the features enabled, they do not receive a notification that iMessage considers their image to be “explicit” or that the recipient’s parent will be notified. The recipient’s parents will be informed of the content without the sender consenting to their involvement. Additionally, once sent or received, the “sexually explicit image” cannot be deleted from the under-13 user’s device.

Whether sending or receiving such content, the under-13 user has the option to decline without the parent being notified. Nevertheless, these notifications give the sense that Apple is watching over the user’s shoulder—and in the case of under-13s, that’s essentially what Apple has given parents the ability to do.

It is also important to note that Apple has chosen to use the notoriously difficult-to-audit technology of machine learning classifiers to determine what constitutes a sexually explicit image. We know from years of documentation and research that machine-learning technologies, used without human oversight, have a habit of wrongfully classifying content, including supposedly “sexually explicit” content. When blogging platform Tumblr instituted a filter for sexual content in 2018, it famously caught all sorts of other imagery in the net, including pictures of Pomeranian puppies, selfies of fully-clothed individuals, and more. Facebook’s attempts to police nudity have resulted in the removal of pictures of famous statues such as Copenhagen’s Little Mermaid. These filters have a history of chilling expression, and there’s plenty of reason to believe that Apple’s will do the same.

There remains a real risk of false positives in this kind of system. There's a very worth reading blog post explaining how automated matching technologies fail, often in catastrophic ways. You really need to read that entire post as brief excerpts wouldn't do it justice -- but as it notes, the risk of false positives here are very high, and the cost of such false positives can be catastrophic. Obviously, CSAM is also catastrophic, so you can see how there is a real challenge in balancing those interests, but there are legitimate concerns that with this approach it's unclear if the balance is properly calibrated.

Obviously, Apple is trying to walk a fine line here. No one wants to be supporting CSAM distribution. But, once again, all the pressure on Apple feels like people blaming the tool for (serious) abuses by its users, and in demanding a "solution," opening up a very dangerous situation. If there were some way to guarantee that these technologies wouldn't be abused, or mess up, you could kind of see how this makes sense. But history has shown time and time again that neither of these things is really possible. Opening up this hole in Apple's famous security means more demands are coming.

Filed Under: client side scanning, csam, encryption, for the children, parent notification, privacy, security
Companies: apple

FTC Formally Embraces Right To Repair As Movement Goes Mainstream

from the you-should-own-the-things-you-buy dept

One of the bigger bright spots in the last few years of often thorny or downright ridiculous policy debates has been the continued ascension of the right to repair movement. Whether it's Apple's wasteful restrictions or bullying of independent repair shops, Sony and Microsoft's efforts to monopolize game console repair, or John Deere's efforts to drive up repair costs for tractor owners, "right to repair" as a movement was born out of a genuine and bipartisan public annoyance at repair monopolies, obnoxious DRM, and self-service restrictions related to tools, documentation, and parts.

And despite Apple and friends' best attempt to smear the movement as some dangerous and diabolical cabal only of use to sexual predators, its popularity shows no signs of slowing down. There's legislation pending on both the federal level and in two-dozen states. Prompted by an FTC report showing industry opposition to the movement is largely fluff and nonsense, the Biden administration recently issued an executive order urging the FTC to do more. And now the FTC, with a bipartisan vote of 5-0, has adopted a new policy paper (pdf) and says it will take tougher action against illegal repair restrictions:

"The Federal Trade Commission today unanimously voted to ramp up law enforcement against repair restrictions that prevent small businesses, workers, consumers, and even government entities from fixing their own products. The policy statement adopted today is aimed at manufacturers’ practices that make it extremely difficult for purchasers to repair their products or shop around for other service providers to do it for them. By enforcing against restrictions that violate antitrust or consumer protection laws, the Commission is taking important steps to restore the right to repair."

The problem, of course, is the same problem facing the FTC on numerous fronts, from privacy to safety in bleach labeling. Namely that the agency's authority is generally restricted under the FTC act to tackling corporate practices that are clearly "unfair and deceptive." The agency also continually suffers from budget and staffing shortcomings (by well-lobbied Congressional design), so tackling the full scope of a problem like this often isn't logistically or financially possible.

Still, the FTC argued that there's a lot more it can do to lend markets and consumers a hand on the right to repair front, whether that's doing a better job enforcing existing warranty laws, engaging in better coordination with state and local policymakers, or doing a better job holding companies that attempt to monopolize repair accountable under antitrust law:

"These types of restrictions can significantly raise costs for consumers, stifle innovation, close off business opportunities for independent repair shops, create unnecessary electronic waste, delay timely repairs and undermine resiliency,” FTC Chairperson Lina Khan said during the meeting. “The FTC has a range of tools it can use to root out unlawful repair restrictions. And today's policy statement would commit us to move forward on this issue with new vigor."

Ideally you'd still want a comprehensive right to repair law to shore up shortcomings in FTC authority. Much like privacy laws, that's generally opposed by a broad coalition of cross-industry lobbyists who'd very much like to keep nickel and diming customers on a whole range of fronts, be it the phone industry or medical device manufacturing. But the more companies like Apple, John Deere, or the auto industry try to fight against reform using stupid, fear-mongering arguments, the more attention these shitty practices receive, the more annoyed the public gets, and the more bipartisan support develops.

That's a major reason why right to repair went from a niche concern just a few years ago, to seeing consistent coverage via the likes of CBS News. That's a positive development any way you slice it, and a nice contrast to many of the thornier and seemingly insurmountable tech policy debates that tend to consume DC, get mired in debate, and seemingly go nowhere.

Filed Under: ftc, right to repair
Companies: apple, john deere

Social Tech Loses Appeal To Apple Over 'Memoji' Trademark Because Suing Apple Isn't Using The Mark In Commerce

from the trademark-tautology dept

For what it's worth, lawsuits against Apple over emojis are not entirely unheard of. You may recall that Apple was sued by a woman who claimed it was copyright infringement for Apple to have emojis that represent more diverse communities, for instance. But for a truly fun story about Apple being sued, and winning its defense, over emojis, well, you have to go to a case between Apple and a company called Social Tech.

Social Tech sued Apple in 2018, alleging Apple's Memoji personalized emoji feature infringed its trademark covering its app with the same name. U.S. District Judge Vince Chhabria ruled for Apple in 2019.

Indeed. And Social Tech appealed that ruling. So why did the court find for Apple in 2019? Well, because Social Tech couldn't demonstrate that it was actually using the mark in commerce. This is where we'll need to dig into some details.

In 2016 Social Tech filed an intent-to-use trademark application for "Memoji" for use in apps and software. The Trademark Office granted a Notice of Allowance, which essentially gives the go ahead to the applicant to put the mark in actual commerce, after which the UPSTO would approve the mark. Social Tech basically did nothing for 2 years after that, other than to ask for an extension on the Notice of Allowance. Meanwhile, another company called Lucky Bunny LLC filed its own trademark application for "Memoji" for the same class of goods. That application was suspended due to the Social Tech application status. In the summer of 2018, Apple acquired Lucky Bunny and its assets, including the suspended trademark application. In June, Apple announced the acquisition and released a public test version of its new mobile OS that included Lucky Bunny's Memoji software.

And this is where it gets weird. When I said above that Social Tech did nothing for two years before Apple's announcement, that's not entirely true. Social Tech did write a business plan, fund itself internally for $100k, and write up some promotional material. What it didn't do was write a single line of code... until Apple announced the acquisition. Then, and only then, did Social Tech's interest in actually using the Memoji mark in commerce go into overdrive.

The appeal was decided by the court in Apple's favor as a result of all of this, but the details laid out in the judgement are striking.

During the three weeks after Apple’s announcement, Social Tech’s co-founder and president, Samuel Bonet, exchanged a series of emails with a software developer to accelerate the timing of the application’s development. In the first of these emails on June 7, Bonet described the circumstances to be “life changing” and concluded the email with: “Time to get paid, gentlemen.” In a series of subsequent emails, Bonet regularly followed up with the developer on the application’s progress. On June 12, Bonet wrote: “the app needs to erase the background AND the body . . . Of course this may take a little work to get perfect, but as long as we can get close initially, we can start to test and put in commerce.” On the evening of June 13, Bonet wrote to the developer: “[i]n other news . . . the initial letter has been sent to Apple. The process has begun. Peace and wealth!”

Bonet continued to follow up on the application’s progress over the next several days, noting that “the editing feature [was] vital” to “satisfy the ‘editing’ requirement of the trademark.” On June 18, Bonet wrote that Social Tech would release the application for Android in the Google Play Store first, proclaiming: “We are lining up all of our information, in preparation for a nice lawsuit against Apple, Inc! We are looking REALLY good. Get your Lamborghini picked out!”

It goes on from there, but the general gist is that Social Tech was very specifically attempting to finally get its mark pushed through simply in order to file these trademark suits and "get paid." And that, sadly for Social Tech, does not satisfy the requirement for using the mark in commerce. And, since the entire enterprise of putting a broken app in commerce was done simply to get the trademark for "Memoji" and file the lawsuit, those don't actually count either.

As a result, the court found for Apple.

The panel held that mere adoption of a mark without bona fide use in commerce, in an attempt to reserve rights for the future, is insufficient to establish rights in the mark under the Lanham Act. Use in commerce requires use of a genuine character, in a way sufficiently public to identify or distinguish the marked goods in an appropriate segment of the public mind. Considering the totality of the circumstances, including relevant non-sales activities, the panel agreed with the district court that the evidence in the record showed that Social Tech’s use of the MEMOJI mark was not bona fide in commerce. Accordingly, Apple was entitled to cancellation of Social Tech’s trademark registration.

When it comes to intellectual property, it's nice to see grifting not work out for the grifters once in a while.

Filed Under: emoji, lawsuits, memoji, shakedown, trademark
Companies: apple, lucky bunny, social tech

How Sketchy Data Scavengers Are Using Hatred Of 'Big Tech' To Attack Plans To Make The Web More Private

from the be-careful-what-you-wish-for dept

We warned that this was likely back when Google announced plans to phase out third-party cookies in Chrome (something all the other major browser makers had already done): that this would be used to attack Google as being anti-competitive, even as it was pro-privacy. Privacy and competition do not need to be in conflict, but they can be. And what's happening now is that more sketchy ad companies are abusing the constant drumbeat and fear over "Big Tech" to attack privacy protections -- but that behind the scenes story is getting missed as people are more focused on more breaking news about how Google has decided to push back its move on 3rd party cookies for two more years.

Issie Lapowsky, over at Protocol, has a must read story on how sketchy ad and data brokers have crashed the W3C, riding a wave of anti-big tech feelings to push for worse solutions for everyone as it comes to privacy (of course, Facebook is on the wrong side of this as well -- it's basically all the sketchy companies and Facebook against all the other companies). It's quite a story.

On one side are engineers who build browsers at Apple, Google, Mozilla, Brave and Microsoft. These companies are frequent competitors that have come to embrace web privacy on drastically different timelines. But they've all heard the call of both global regulators and their own users, and are turning to the W3C to develop new privacy-protective standards to replace the tracking techniques businesses have long relied on.

On the other side are companies that use cross-site tracking for things like website optimization and advertising, and are fighting for their industry's very survival. That includes small firms like Rosewell's, but also giants of the industry, like Facebook.

Much of the story focuses on James Rosewell, who runs a "data analytics" company, and joined the W3C apparently just to throw a wrench in any real effort to beef up privacy protections as a core element of the internet.

Rosewell has become one of this side's most committed foot soldiers since he joined the W3C last April. Where Facebook's developers can only offer cautious edits to Apple and Google's privacy proposals, knowing full well that every exchange within the W3C is part of the public record, Rosewell is decidedly less constrained. On any given day, you can find him in groups dedicated to privacy or web advertising, diving into conversations about new standards browsers are considering.

Rather than asking technical questions about how to make browsers' privacy specifications work better, he often asks philosophical ones, like whether anyone really wants their browser making certain privacy decisions for them at all. He's filled the W3C's forums with concerns about its underlying procedures, sometimes a dozen at a time, and has called upon the W3C's leadership to more clearly articulate the values for which the organization stands.

His exchanges with other members of the group tend to have the flavor of Hamilton and Burr's last letters — overly polite, but pulsing with contempt. "I prioritize clarity over social harmony," Rosewell said.

Rosewell and his companions are leaning hard on the idea that if privacy is actually protected, then only Google, Apple, and Microsoft will control the internet. But that's bullshit -- and thankfully, the privacy-focused browser Brave has people willing to call it out as such:

"They use cynical terms like: 'We're here to protect user choice' or 'We're here to protect the open web' or, frankly, horseshit like this," said Pete Snyder, director of privacy at Brave, which makes an anti-tracking browser. "They're there to slow down privacy protections that the browsers are creating."

The article also notes that these more sketchy data brokers (and Facebook) are not just looking to derail the more privacy protective approaches of the browser makers, but to create new standards that would enshrine the surveillance aspect of the web. And, again, they're playing on the general sentiment against "big tech" to try to push that across.

That's only the beginning of the article, it then goes into great detail on the history here (including the fight over "Do Not Track"), and how Rosewell and his colleagues are trying to throw sand in the gears of the W3C to basically wipe out any real movement towards better standards on privacy. And also how Facebook is quietly cheering on Rosewell and folks.

It may sound like a boring and wonky discussion, but it's hugely important for understand the future of the open internet and privacy, and how competition questions are impacted by these things. All of this is important, but the various things -- privacy, competition, content moderation, etc -- are often looked at as separate buckets. But bad choices in one will inevitably impact the others as well. For those who care about privacy online, this story (which goes into a lot more detail) is a must read.

Filed Under: 3rd party cookies, data brokers, privacy, standards, tracking, web browsing
Companies: apple, facebook, google, w3c