Lessons From Making Internet Companies Liable For User's Speech: You Get Less Speech, Less Security And Less Innovation

from the not-good dept

Stanford's Daphne Keller is one of the world's foremost experts on intermediary liability protections and someone we've mentioned on the website many times in the past (and have had her on the podcast a few times as well). She's just published a fantastic paper presenting lessons from making internet platforms liable for the speech of its users. As she makes clear, she is not arguing that platforms should do no moderation at all. That's a silly idea that no one who has any understanding of these issues thinks is a reasonable idea. The concern is that as many people (including regulators) keep pushing to pin liability on internet companies for the activities of their users, it creates some pretty damaging side effects. Specifically, the paper details how it harms speech, makes us less safe, and harms the innovation economy. It's actually kind of hard to see what the benefit side is on this particular cost-benefit equation.

As the paper notes, it's quite notable how the demands from people about what platforms should do keeps changing. People keep demanding that certain content gets removed, while others freak out that too much content is being removed. And sometimes it's the same people (they want the "bad" stuff -- i.e., stuff they don't like -- removed, but get really angry when the stuff they do like is removed). Perhaps even more importantly, the issues for why certain content may get taken down are the same issues that often involve long and complex court cases, with lots of nuance and detailed arguments going back and forth. And yet, many people seem to think that private companies are somehow equipped to credibly replicate that entire judicial process, without the time, knowledge or resources to do so:

As a society, we are far from consensus about legal or social speech rules. There are still enough novel and disputed questions surrounding even long-standing legal doctrines, like copyright and defamation, to keep law firms in business. If democratic processes and court rulings leave us with such unclear guidance, we cannot reasonably expect private platforms to do much better. However they interpret the law, and whatever other ethical rules they set, the outcome will be wrong by many people’s standards.

Keller then looked at a variety of examples involving intermediary liability to see what the evidence says would happen if we legally delegate private internet platforms into the role of speech police. It doesn't look good. Free speech will suffer greatly:

The first cost of strict platform removal obligations is to internet users’ free expression rights. We should expect over-removal to be increasingly common under laws that ratchet up platforms’ incentives to err on the side of taking things down. Germany’s new NetzDG law, for example, threatens platforms with fines of up to &euro'50 million for failure to remove “obviously” unlawful content within twenty-four hours’ notice. This has already led to embarrassing mistakes. Twitter suspended a German satirical magazine for mocking a politician, and Facebook took down a photo of a bikini top artfully draped over a double speed bump sign.11 We cannot know what other unnecessary deletions have passed unnoticed.

From there, the paper explores the issue of security. Attempts to stifle terrorists' use of online services by pressuring platforms to remove terrorist content may seem like a good idea (assuming we agree that terrorism is bad), but the actual impact goes way beyond just having certain content removed. And the paper looks at what the real world impact of these programs have been in the realm of trying to "counter violent extremism."

The second cost I will discuss is to security. Online content removal is only one of many tools experts have identified for fighting terrorism. Singular focus on the internet, and overreliance on content purges as tools against real-world violence, may miss out on or even undermine other interventions and policing efforts.

The cost-benefit analysis behind CVE campaigns holds that we must accept certain downsides because the upside—preventing terrorist attacks—is so crucial. I will argue that the upsides of these campaigns are unclear at best, and their downsides are significant. Over-removal drives extremists into echo chambers in darker corners of the internet, chills important public conversations, and may silence moderate voices. It also builds mistrust and anger among entire communities. Platforms straining to go “faster and further” in taking down Islamist extremist content in particular will systematically and unfairly burden innocent internet users who happened to be speaking Arabic, discussing Middle Eastern politics, or talking about Islam. Such policies add fuel to existing frustrations with governments that enforce these policies, or platforms that appear to act as state proxies. Lawmakers engaged in serious calculations about ways to counter real-world violence—not just online speech—need to factor in these unintended consequences if they are to set wise policies.

Finally, the paper looks at the impact on innovation and the economy and, again, notes that putting liability on platforms for user speech can have profound negative impacts.

The third cost is to the economy. There is a reason why the technology-driven economic boom of recent decades happened in the United States. As publications with titles like “How Law Made Silicon Valley” point out, our platform liability laws had a lot to do with it. These laws also affect the economic health of ordinary businesses that find customers through internet platforms—which, in the age of Yelp, Grubhub, and eBay, could be almost any business. Small commercial operations are especially vulnerable when intermediary liability laws encourage over-removal, because unscrupulous rivals routinely misuse notice and takedown to target their competitors.

The entire paper weighs in at a neat 44 pages and it's chock full of useful information and analysis on this very important question. It should be required reading for anyone who thinks that there are easy answers to the question of what to do about "bad" content online, and it highlights that we actually have a lot of data and evidence to answer the questions that many legislators seem to be regulating based on how they "think" the world would work, rather than how the world actually works.

Current attitudes toward intermediary liability, particularly in Europe, verge on “regulate first, ask questions later.” I have suggested here that some of the most important questions that should inform policy in this area already have answers. We have twenty years of experience to tell us how intermediary liability laws affect, not just platforms themselves, but the general public that relies on them. We also have valuable analysis and sources of law from pre-internet sources, like the Supreme Court bookstore cases. The internet raises new issues in many areas—from competition to privacy to free expression—but none are as novel as we are sometimes told. Lawmakers and courts are not drafting on a blank slate for any of them.

Demands for platforms to get rid of all content in a particular category, such as “extremism,” do not translate to meaningful policy making—unless the policy is a shotgun approach to online speech, taking down the good with the bad. To “go further and faster” in eliminating prohibited material, platforms can only adopt actual standards (more or less clear, and more or less speech-protective) about the content they will allow, and establish procedures (more or less fair to users, and more or less cumbersome for companies) for enforcing them.

On internet speech platforms, just like anywhere else, only implementable things happen. To make sound policy, we must take account of what real-world implementation will look like. This includes being realistic about the capabilities of technical filters and about the motivations and likely choices of platforms that review user content under threat of liability.

This is an important contribution to the discussion, and highly recommended. Go check it out.

Filed Under: cda 230, daphne keller, dmca 512, free speech, innovation, intermediary liability, section 230, security

Yet Another Study Shows The Internet Of Things Is A Privacy And Security Dumpster Fire

from the the-dumber-the-better dept

Day in and day out, it's becoming increasingly clear that the smart home revolution simply isn't all that smart.

Security analysts like Bruce Schneier have been sounding the alarm bells for years now about the lax to nonexistent security and privacy standards inherent in the internet of broken things space. From refrigerators that leak your Gmail credentials to Barbie dolls that can be easily hacked to spy on kids, it's increasingly clear that dumber technology is often the smarter solution. Not only do many of these devices actually make us less secure, their lack of real security has resulted in their use in historically large DDoS attacks.

Study after study shows it's a problem that's not really getting better. For example, despite a decade of reports about the lack of real security and privacy standards in smart TVs, Consumer Reports recently found that most smart TVs remain impressively open to attack and abuse. And a new study out of the UK by Which? studied 19 different smart gadgets and found a "staggering level of corporate surveillance of your home" by devices that routinely hoovered up consumer data, then funneled it out to dozens of partner companies -- often without clear consumer permission:

"Many apps ask for your exact location when they don’t actually need it for the product or service to work. Far too often, specific information is requested about you when the justification seems arguable at best. Then there’s the galaxy of other companies busily working in the background of your smart gadgets. During our testing we saw more than 20 other operators involved behind the scenes, including marketing companies. When we used a smart TV for just 15 minutes, it connected with a staggering 700 distinct addresses on the internet.

You'll recall that a few years ago, the revelation that there was now a search engine specifically built to provide easy access to poorly secured webcams resulted in all manner of consternation about the problem of default usernames and passwords and devices with paper-mache-grade security. But despite flimsy webcam security being such a hot topic for years, many vendors still haven't gotten the message:

"We’re also concerned over how companies secure your data. In a separate test together with other consumer organisations, we found a flaw in this wireless security camera’s app (provided by a company called Sricam), which meant that we could access more than 200,000 passwords and device IDs for other ieGeek cameras. We could then see live video feeds of other users, and talk to those users via the camera’s microphone (which we didn’t do). ieGeek/Sricam fixed this flaw in late March 2018, but we’ve subsequently found and disclosed other critical vulnerabilities with the camera and app."

Security analysts like Bruce Schneier have clearly illustrated why there's no incentive to fix these problems:

"The market can't fix this because neither the buyer nor the seller cares. The owners of the webcams and DVRs used in the denial-of-service attacks don't care. Their devices were cheap to buy, they still work, and they don't know any of the victims of the attacks. The sellers of those devices don't care: They're now selling newer and better models, and the original buyers only cared about price and features. There is no market solution, because the insecurity is what economists call an externality: It's an effect of the purchasing decision that affects other people. Think of it kind of like invisible pollution."

The reality is we're collectively more interested in making money and obsessing over the latest gadget than addressing the problem. And while there's some very good ongoing efforts to create some basic security and privacy standards in the IOT space, the prevailing attitude among IOT users and vendors alike that this is all somebody else's problem. Folks like Schneier have been warning for a while that it's likely going to take a mass casualty event (caused by hacked infrastructure) to finally motivate some changes in the internet of broken things space.

Filed Under: internet of things, privacy, security

New Gear: The NSA Collection

from the approved-for-release dept

Recently, the folks at Government Attic filed a FOIA request that garnered a very cool response: a collection of posters made by the NSA in the 1950s and 60s to remind its employees about security. It wasn't long before we got some requests to put them on t-shirts in the Techdirt Gear store and so... that's exactly what we've done!

You can now get 24 of the NSA's posters (with more coming soon) on premium t-shirts, hoodies and mugs from Teespring. Check out our store for the full NSA collection, or click the images below to go directly to the ones you like most.

Filed Under: gear, nsa, security, security posters, surveillance, t-shirts

President Trump Thinks Basic Phone Security Is Simply Too Inconvenient

from the who-needs-updates dept

For the past year much has been made of the President's unwillingness to adhere to anything close to reasonable security when using his mobile phones. Whereas the Defense Information Systems Agency (DISA) and the National Security Agency usually work in concert providing state leaders with "hardened" devices that are heavily encrypted, routinely updated, and frequently swapped out, Trump has refused to use these more secure DMCC-S devices (effectively a Samsung Galaxy S4 device utilizing Samsung's Knox security architecture) because they apparently infringe on his ability to Tweet.

Just a few months ago, Senators sent a letter expressing concern that Trump's mobile phone practices were leaving the President open to potential hacking by foreign entities:

"The President of the United States stands alone as the single-most valuable intelligence target on the planet. Given the apparent lack of progress the Administration has made since initial reports in 2016 of the President’s poor operational security, it appears the only thing standing between the Office of the President and the next national security nightmare is a combination of President Trump’s personal restraint and sheer luck."

Eventually, the President was convinced to use two iPhones: one locked down specifically for Twitter, and the other specifically tasked with making phone calls. Even here reports have suggested that Trump has struggled to adhere to these restrictions, often making personal calls on his unsecured Samsung Galaxy III. This week a report from Politico highlights again how White House and IT security staff have recommended that these devices be routinely swapped out, an idea the President continues to resist:

"While aides have urged the president to swap out the Twitter phone on a monthly basis, Trump has resisted their entreaties, telling them it was “too inconvenient,” the same administration official said.

The president has gone as long as five months without having the phone checked by security experts. It is unclear how often Trump’s call-capable phones, which are essentially used as burner phones, are swapped out."

And yeah, that's a serious problem. While these devices appear to be the equivalent of burner phones, five months is forever in security world, giving attackers ample opportunities to compromise the microphone, camera, or other hardware embedded in Trump's devices. It's a particularly notable problem in the wake of things like the SS7 vulnerablity -- a flaw mobile carriers routinely try to downplay that opens the door to remote surveillance and a wide variety of attacks on most modern smartphones and mobile networks.

Needless to say, the President's abysmal privacy and security practices likely make a delicious potential target for hostile foreign powers, something you'd think Trump would realize given his breathless hyperventilation regarding a certain home e-mail server. Meanwhile, the President's disdain for routine security comes incongruently as his FCC moves to block many Chinese hardware vendors from doing business in the States, a protectionist move the administration insists is exclusively focused on protecting national security.

Filed Under: disa, donald trump, secure phone, security, tweeting

Bill Introduced To Prevent Government Agencies From Demanding Encryption Backdoors

from the pushing-back-from-the-top-down dept

The FBI continues its push for a solution to its "going dark" problem. Joined by the DOJ, agency head Christopher Wray has suggested the only way forward is a legislative or judicial fix, gesturing vaguely to the thousands of locked phones the FBI has gathered. It's a disingenuous push, considering the tools available to the agency to crack locked devices and obtain the apparently juicy evidence hidden inside.

The FBI hasn't been honest in its efforts or its portrayal of the problem. Questions put to the FBI about its internal efforts to crack locked devices are still unanswered. The only "new" development isn't all that new: Ray Ozzie's "key escrow" proposal may tweak a few details but it's not that far removed in intent from the Clipper Chip that kicked off the first Crypto War. It's nothing more than another way to make device security worse, with the only beneficiary being the government.

The FBI's disingenuousness has not gone unnoticed. Efforts have been made over the last half-decade to push legislators towards mandating government access, but no one has been willing to give the FBI what it wants if it means making encryption less useful. A new bill [PDF], introduced by Zoe Lofgren, Thomas Massie, Ted Poe, Jerry Nadler, Ted Lieu, and Matt Gaetz would codify this resistance to government-mandated backdoors.

The two-page bill has sweeping safeguards that uphold security both for developers and users. As the bill says, “no agency may mandate or request that a manufacturer, developer, or seller of covered products design or alter the security functions in its product or service to allow the surveillance of any user of such product or service, or to allow the physical search of such product, by any agency.”

This bill would protect companies that make encrypted mobile phones, tablets, desktop and laptop computers, as well as developers of popular software for sending end-to-end encrypted messages, including Signal and WhatsApp, from being forced to alter their products in a way that would weaken the encryption. The bill also forbids the government from seeking a court order that would mandate such alterations. The lone exception is for wiretapping standards required under the 1994 Communications for Law Enforcement Act (CALEA), which itself specifically permits providers to offer end-to-end encryption of their services.

The Secure Data Act shouldn't be needed but the FBI and DOJ have forced the hand of legislators. Rather than take multiple hints dropped by the previous administration, the agencies have only increased the volume of their anti-encryption rhetoric in recent months. Maybe the agencies felt they'd have the ear of the current administration and Congressional majority, but investigations involving the president and his staff have pretty much killed any "law and order" leanings the party normally retains. This bill may see widespread bipartisan support simply because it appears to be sticking it to the Deep State. Whatever. We'll take it. Hopefully, this makes a short and direct trip to the Oval Office for a signature.

Filed Under: backdoors, congress, doj, encryption, fbi, going dark, jerry nadler, matt gaetz, responsible encryption, secure data act, security, ted lieu, ted poe, thomas massie, zoe lofgren

Police Drop Charges Filed Against 19-Year-Old Who Downloaded Public Documents From Gov't FOI Portal

from the access-not-specifically-authorized-is-not-'unauthorized-access' dept

Last month, we covered the incredible case of an unnamed 19-year-old who was facing criminal charges for downloading publicly-available documents from a government Freedom of Information portal. The teen had written a script to fetch all available documents from the Nova Scotia's government FOI site -- a script that did nothing more than increment digits at the end of the URL to find everything that had been uploaded by the government.

The government screwed up. It uploaded documents to the publicly-accessible server that hadn't been redacted yet. It was a very small percentage of the total haul -- 250 of the 7,000 docs obtained -- but the government made a very big deal out of it after discovering they had been accessed.

The government complained to the police and had the downloader hauled in to face unauthorized access charges, claiming he had "exploited a vulnerability" to obtain unredacted files. But no exploit was used. It was the government that left unredacted documents in a publicly-accessible space. Nevertheless, the teen's home was raided, his family accosted, and several electronic devices seized -- including those of family members. The 19-year-old's younger brother was even detained by officers while walking to school.

Government officials claimed the teen "stole" documents, and pushed for criminal charges which could have resulted in a ten-year sentence for downloading documents from a government portal designed to facilitate the downloading of documents.

Fortunately, Nova Scotia law enforcement has decided there's nothing to pursue in this case.

In an email to CBC News, Halifax police Supt. Jim Perrin did not mention what kind of information police were given from the province, but he said it was a "high-profile case that potentially impacted many Nova Scotians."

"As the investigation evolved, we have determined that the 19-year-old who was arrested on April 11 did not have intent to commit a criminal offence by accessing the information," Perrin said in the email.

The "information" the province "handed over" was probably nothing more than the belated recognition that pursuing criminal charges had accomplished nothing more than exposing the government's careless handling of citizens' personal information and its willingness to find a scapegoat to burden with its failure. The government also revealed 11 other IP addresses had accessed the same unredacted documents, which only further solidified the government's complicity in public access to unredacted personal info.

Prosecutors would have struggled to prove intent -- something law enforcement likely recognized shortly after taking up the case. And this would have been a case they couldn't ignore, not with government officials making lots of noise about hacking that never took place and "unauthorized" access that was plainly authorized by their inability or unwillingness to properly secure documents that hadn't been vetted or redacted. It's already suffered a PR black eye. This move to disengage simply reduces the chance of further injury.

Filed Under: canada, foi, foia, public documents, security

Princeton Project Aims To Secure The Internet Of Broken, Shitty Things

from the Barbie-needs-a-better-firewall dept

Year after year, we're installing millions upon millions of "internet of things" devices on home and business networks that have only a fleeting regard for security or privacy. The width and depth of manufacturer incompetence on display can't be understated. Thermostats that prevent you from actually heating your home. Smart door locks that make you less secure. Refrigerators that leak Gmail credentials. Children's toys that listen to your kids' prattle, then (poorly) secure said prattle in the cloud. Cars that could, potentially, result in your death.

The list goes on and on, and it grows exponentially by the week, especially as such devices are quickly compromised and integrated into massive new botnets. And as several security experts have noted, nobody in this chain of dysfunction has the slightest interest in doing much about this massive rise in "invisible pollution":

"The market can't fix this because neither the buyer nor the seller cares. Think of all the CCTV cameras and DVRs used in the attack against Brian Krebs. The owners of those devices don't care. Their devices were cheap to buy, they still work, and they don't even know Brian. The sellers of those devices don't care: they're now selling newer and better models, and the original buyers only cared about price and features. There is no market solution because the insecurity is what economists call an externality: it's an effect of the purchasing decision that affects other people. Think of it kind of like invisible pollution."

One core part of the problem is that IOT device makers refuse to provide much control or transparency over what their internet-connected devices actually do once online. Often the tools and device interfaces provided to the end user are comically simple, providing you with virtually no data on how much bandwidth your devices are consuming, or what data they're transferring back to the cloud (frequently unencrypted). As a result, many normal people are participating in historically massive DDOS attacks or having their every behavior monitored without having the slightest idea it's actually occurring.

To that end Princeton's computer science department has launched a research program called the IOT Inspector they hope will provide users with a little more insight into what IOT devices are actually up to. The researchers behind the project say they spent some time analyzing fifty different common IOT devices, and like previous studies found that security and privacy in these devices was a total shitshow. Sending private user data unencrypted back to the cloud was common:

Unfortunately, many of the devices we have examined lack even these basic security or privacy features. For example, the Withings Smart Blood Pressure Monitor included the brand of the device and the string “blood pressure” in unencrypted HTTP GET request headers. This allows a network eavesdropper to (1) learn that someone in a household owns a blood pressure monitor and (2) determine how frequently the monitor is used based on the frequency of requests. It would be simple to hide this information with SSL."

As were devices that immediately began chatting with all manner of partner services whether the user wants them to or not:

Samsung Smart TV: During the first minute after power-on, the TV talks to Google Play, Double Click, Netflix, FandangoNOW, Spotify, CBS, MSNBC, NFL, Deezer, and Facebook—even though we did not sign in or create accounts with any of them.

Again, user control and transparency is almost always an afterthought. Obviously, the creation of some unified standards is one solution. As is creating routers and hardware that alert users to when their devices have been compromised. Smarter networks and hardware are going to need to be a cornerstone of any proposed solution, the researchers note:

We are experimenting with machine learning-based DDoS detection using features using IoT-specific network behaviors (e.g., limited number of endpoints and regular time intervals between packets). Preliminary results indicate that home gateway routers or other network middleboxes could automatically detect local IoT device sources of DDoS attacks with high accuracy using low-cost machine learning algorithms.

Of course better standards are going to need to be built on the backs of a joint collaboration between governments, companies, consumers and researchers. And while we've seen mixed results on that front so far, efforts like this (and the Consumer Reports' open source attempt to make privacy and security an integral part of product reviews) are definitely a step in the right direction.

Filed Under: iot, princeton, security
Companies: princeton

Software Legend Ray Ozzie Thinks He Can Safely Backdoor Encryption; He's Very Wrong

from the and-dangerous dept

There have been ongoing debates for a while now about the stupidity of backdooring encryption, with plenty of experts explaining why there's no feasible way to do it without causing all sorts of serious consequences (some more unintended than others). Without getting too deep into the weeds, the basic issue is that cryptography is freaking difficult and if something goes wrong, you're in a lot of trouble very fast. And it's very, very easy for something to go wrong. Adding in a backdoor to encryption is, effectively, making something go wrong... on purpose. In doing so, however, you're introducing a whole host of other opportunities for many, many things to go wrong, blowing up the whole scheme and putting everyone's information at risk. So, if you're going to show up with a "plan" to backdoor encryption, you better have a pretty convincing argument for how you avoid that issue (because the reality is you can't).

For at least a year (probably more) the one name that has kept coming up over and over as one of the few techies who insists that the common wisdom on backdooring encryption is wrong... is Ray Ozzie. Everyone notes that he's Microsoft's former Chief Software Architect and CTO, but some of us remember him from way before that when he created Lotus Notes and Groove Networks (which was supposed to be the nirvana of collaboration software). In recent months his name has popped up here and there, often by FBI/DOJ folks seeking to backdoor encryption, as having some possible ways forward.

And, recently, Wired did a big story on his backdoor idea, where he plays right into the FBI's "nerd harder" trope, by saying exactly what the FBI wants to hear, and which nearly every actual security expert says is wrong:

Ozzie, trim and vigorous at 62, acknowledged off the bat that he was dealing with a polarizing issue. The cryptographic and civil liberties community argued that solving the problem was virtually impossible, which “kind of bothers me,” he said. “In engineering if you think hard enough, you can come up with a solution.” He believed he had one.

This, of course, is the same sort of thing that James Comey, Christopher Wray and Rod Rosenstein have all suggested in the past few years: "you techies are smart, if you just nerd harder, you'll solve the problem." Ozzie, tragically, is giving them ammo. But he's not delivering the actual goods.

The Wired story details his plan which is not particularly unique. It takes concepts that others have proposed (and which have been shown to not be particularly secure) and puts a fresh coat of paint on them. Basically, the vendor of a device has a private key that it needs to keep secret, and under some "very special circumstances" it can send an employee into the dark chamber to do the requisite dance, retrieve the code, and give it to law enforcement. That's been suggested many times, and it's been explained many times why that opens up all sorts of dangerous scenarios that could put everyone at risk. The one piece that does seem different is that Ozzie wants a sort of limitation on the possible damage his system does if it goes wrong (in one particular way), which is that under his system if the backdoor is used, it can only be used on one phone and then it disables that phone forever:

Ozzie designed other features meant to ­reassure skeptics. Clear works on only one device at a time: Obtaining one phone’s PIN would not give the authorities the means to crack anyone else’s phone. Also, when a phone is unlocked with Clear, a special chip inside the phone blows itself up, freezing the contents of the phone thereafter. This prevents any tampering with the contents of the phone. Clear can’t be used for ongoing surveillance, Ozzie told the Columbia group, because once it is employed, the phone would no longer be able to be used.

So, let's be clear. That piece isn't what's useful in "reassuring skeptics." That piece is the only thing that really appears to be that unique about Ozzie's plan. And it hasn't done much to reassure skeptics. As the report notes, when Ozzie laid this out at a special meeting of super smart folks in the field, it didn't take long for one to spot a hole:

The most dramatic comment came from computer science professor and cryptographer Eran Tromer. With the flair of Hercule Poirot revealing the murderer, he announced that he’d discovered a weakness. He spun a wild scenario involving a stolen phone, a second hacked phone, and a bank robbery. Ozzie conceded that Tromer found a flaw, but not one that couldn’t be fixed.

"Not one that couldn't be fixed." But it took this guy just hearing about the system to find the flaw. There are more flaws. And they're going to be catastrophic. Because that's how cryptogrpahy works. Columbia computer science professor and all around computer security genius Steve Bellovin (who was also at that meeting) highlights how Tromer's flaw-spotting shows why Ozzie's plan is a fantasy with dangerous consequences:

Ozzie presented his proposal at a meeting at Columbia—I was there—to a diverse group. Levy wrote that Ozzie felt that he had "taken another baby step in what is now a two-years-and-counting quest" and that "he'd started to change the debate about how best to balance privacy and law enforcement access". I don't agree. In fact, I think that one can draw the opposite conclusion.

At the meeting, Eran Tromer found a flaw in Ozzie's scheme: under certain circumstances, an attacker can get an arbitrary phone unlocked. That in itself is interesting, but to me the important thing is that a flaw was found. Ozzie has been presenting his scheme for quite some time. I first heard it last May, at a meeting with several brand-name cryptographers in the audience. No one spotted the flaw. At the January meeting, though, Eran squinted at it and looked at it sideways—and in real-time he found a problem that everyone else had missed. Are there other problems lurking? I wouldn't be even slightly surprised. As I keep saying, cryptographic protocols are hard.

Bellovin also points out -- as others have before -- that there's a wider problem here: how other countries will use whatever stupid example the US sets for much more nefarious purposes:

If the United States adopts this scheme, other countries, including specifically Russia and China, are sure to follow. Would they consent to a scheme that relied on the cooperation of an American company, and with keys stored in the U.S.? Almost certainly not. Now: would the U.S. be content with phones unlockable only with the consent and cooperation of Russian or Chinese companies? I can't see that, either. Maybe there's a solution, maybe not—but the proposal is silent on the issue.

And we're just getting started on how many experts are weighing in on just how wrong Ozzie is. Errata Security's Rob Graham pulls no punches pointing out that:

He's only solving the part we already know how to solve. He's deliberately ignoring the stuff we don't know how to solve. We know how to make backdoors, we just don't know how to secure them.

Specifically, Ozzie's plan relies on the idea that companies can keep their master private key safe. To support that this is possible, Ozzie (as the FBI has in the past) points to the fact that companies like Apple already keep their signing keys secret. And that's true. But that assumes incorrectly that signing keys and decryption keys are the same thing and can be treated similarly. They're not and they cannot be. The security protocols around signing keys are intense, but part of that intensity is built around the idea that you almost never have to use a signing key.

A decryption key is a different story altogether, especially with the FBI blathering on about thousands of phones it wants to dig its digital hands into. And, as Graham notes, you quickly run into a scaling issue, and with that scale, you ruin any chance of keeping that key secure.

Yes, Apple has a vault where they've successfully protected important keys. No, it doesn't mean this vault scales. The more people and the more often you have to touch the vault, the less secure it becomes. We are talking thousands of requests per day from 100,000 different law enforcement agencies around the world. We are unlikely to protect this against incompetence and mistakes. We are definitely unable to secure this against deliberate attack.

And, even worse, when that happened, we wouldn't even know.

If Ozzie's master key were stolen, nothing would happen. Nobody would know, and evildoers would be able to freely decrypt phones. Ozzie claims his scheme can work because SSL works -- but then his scheme includes none of the many protections necessary to make SSL work.

What I'm trying to show here is that in a lab, it all looks nice and pretty, but when attacked at scale, things break down -- quickly. We have so much experience with failure at scale that we can judge Ozzie's scheme as woefully incomplete. It's not even up to the standard of SSL, and we have a long list of SSL problems.

And so Ozzie's scheme relies on an impossibility. That you could protect a decryption key that has to be used frequently, the same way that a signing key is currently protected. And that doesn't work. And when it fails, everyone is seriously fucked.

Graham's article also notes that Ozzis is -- in true nerd harder fashion -- focusing on this as a technological problem, ignoring all the human reasons why such a system will fail and such a key won't be protected.

It focuses on the mathematical model but ignores the human element. We already know how to solve the mathematical problem in a hundred different ways. The part we don't know how to secure is the human element.

How do we know the law enforcement person is who they say they are? How do we know the "trusted Apple employee" can't be bribed? How can the law enforcement agent communicate securely with the Apple employee?

You think these things are theoretical, but they aren't.

Cryptography expert (and professor at Johns Hopkins), Matt Green did a fairly thorough tweetstorm debunking of Ozzie's plan as well. He also points out, as Graham does, the disaster scenario of what happens when (not if) the key gets out. But, an even bigger point that Green makes is that Ozzie's plan relies on a special chip in every device... and assumes that we'll design that chip to work perfectly and never get broken. And that's ridiculous:

Green and Graham also both point to the example of GrayKey, the recently reported on tool that law enforcement has been using to crack into all supposedly encrypted iPhones. Already, someone has hacked into the company behind GrayKey and leaked some of the code.

Put it all together and:

Suddenly the fawning over Ozzie's plan doesn't look so good any more, does it? And, again, these are the problems that everyone who has dug into why backdoors are a bad idea have pointed out before:

Green expanded some of his tweets into a blog post as well, which is also worth reading. In it, he also points out that even if we acknowledge the difference between signing keys and decryption keys, companies aren't even that good at keeping signing keys safe (and those are almost certainly going to be more protected that decryption keys since they need to be access much less frequently):

Moreover, signing keys leak all the time. The phenomenon is so common that journalists have given it a name: it’s called “Stuxnet-style code signing”. The name derives from the fact that the Stuxnet malware — the nation-state malware used to sabotage Iran’s nuclear program — was authenticated with valid code signing keys, many of which were (presumably) stolen from various software vendors. This practice hasn’t remained with nation states, unfortunately, and has now become common in retail malware.

And he also digs deeper into the point he made in his tweetstorm about how on the processor side, not even Apple has been able to keep its secure chip from being broken -- yet Ozzie's plan is based almost entirely on the idea that such an unbreakable chip would be available:

The richest and most sophisticated phone manufacturer in the entire world tried to build a processor that achieved goals similar to those Ozzie requires. And as of April 2018, after five years of trying, they have been unable to achieve this goal — a goal that is critical to the security of the Ozzie proposal as I understand it.

Now obviously the lack of a secure processor today doesn’t mean such a processor will never exist. However, let me propose a general rule: if your proposal fundamentally relies on a secure lock that nobody can ever break, then it’s on you to show me how to build that lock.

Update: We should add that the criticisms raised here are not new either. Back in February we wrote about a whitepaper by Riana Pfefferkorn making basically all of these same points that the folks quoted above are making. In other words, it's a bit bizarre that Wired wrote this article as if Ozzie is doing something new and noteworthy.

So that's a bunch of experts highlighting why Ozzie's plan is silly. But, from the policy side it's awful too. Because having Ozzie going around and spouting this debunked nonsense, but with his pedigree, simply gives the "going dark" and "responsible encryption" pundits something to grasp onto to claim they were right all along, even though they weren't. They've said for years that the techies just need to nerd harder, and they will canonize Ray Ozzie as the proof that they were right... even though they're not and his plan doesn't solve any of the really hard problems.

And, as we noted much earlier in this post, cryptography is one of those areas where the hard problems really fucking matter. And if Ozzie's plan doesn't even touch on most of the big ones, it's no plan at all. It's a Potemkin Village that law enforcement types will parade around for the next couple of years insisting that backdoors can be made safely, even though Ozzie's plan is not safe at all. I am sure that Ray Ozzie means well -- and I've got tremendous respect for him and have for years. But what he's doing here is actively harmful -- even if his plan is never implemented. Giving the James Comeys and Chris Wrays of the worlds some facade they can cling to to say that this can be done is only going to create many more problems.

Filed Under: encryption, encryption is hard, going dark, key escrow, matthew green, nerd harder, ray ozzie, responsible encryption, rob graham, security, steven bellovin

A Casino Was Hacked Thanks To The Internet Of Broken Things & A Fish Tank Thermometer

from the somebody-might-want-to-get-on-this dept

For years we've documented how the internet of broken things industry and evangelists have contributed to a global privacy and security shitshow. The rush to connect everything from tea kettles to Barbie dolls to the internet without including even basic privacy or security standards has resulted in a massive security problem few seem interested in actually fixing. As a result we're not only less secure and more at risk for privacy violations, but these devices are now routinely contributing to some of the most devastating DDoS attacks history has ever seen.

A year or so ago Bruce Schneier penned what was probably the best explanation of why nothing in the IOT chain of dysfunction seems to improve:

"The market can't fix this because neither the buyer nor the seller cares. Think of all the CCTV cameras and DVRs used in the attack against Brian Krebs. The owners of those devices don't care. Their devices were cheap to buy, they still work, and they don't even know Brian. The sellers of those devices don't care: they're now selling newer and better models, and the original buyers only cared about price and features. There is no market solution because the insecurity is what economists call an externality: it's an effect of the purchasing decision that affects other people. Think of it kind of like invisible pollution."

Instead of fixing their products, vendors simply move on to marketing the next best thing. And consumers continue to gobble them up, creating millions of millions of new attack vectors into homes and businesses around the world annually. Obviously this "invisible pollution" continues to have a very real and visible impact. Case in point: Nicole Eagan, the CEO of cybersecurity firm Darktrace, says hackers are increasingly targeting unprotected IOT devices including air conditioners, toys, and surveillance cameras to get into corporate networks.

She noted how one bank that decided to skimp on security cameras actually wound up being hacked after those cameras were quickly compromised by attackers. Speaking at the WSJ CEO Council Conference, she also shared an anecdote about how one big casino client had their customers' financial histories stolen thanks to an internet-of-broken things aquarium thermostat:

"Eagan gave one memorable anecdote about a case Darktrace worked on in which a casino was hacked via a thermometer in an aquarium in the lobby. The attackers used that to get a foothold in the network," she said. "They then found the high-roller database and then pulled that back across the network, out the thermostat, and up to the cloud."

It's understandable that people are wary of regulating this sector lest it stifle innovation or create unforeseen, additional problems. But it's pretty clear we're going to need a massive collaboration between the public, companies, and government if we want to avoid some potentially calamitous and fatal outcomes (especially if and when essential infrastructure is targeted). That's why what the open source IOT security and privacy standards organizations like Consumer Reports have been cooking up desperately need all the public and private sector support they can get.

Filed Under: casino, cybersecurity, iot, security, thermometer

Another Company Blows Off Breach Notification For Months, Lies About Affected Customers When It's Exposed

from the trust-no-one dept

Another day, another security breach. Another day, another security breach handled badly by the company leaking data. Another day, another security researcher being treated like garbage for attempting to report it. Etc. Etc.

The victim perpetrator here is Panera Bread. Researcher Dylan Houlihan informed Panera Bread its online ordering service was leaking data. This notification happened months ago.

In August 2017, I reported a vulnerability to Panera Bread that allowed the full name, home address, email address, food/dietary preferences, username, phone number, birthday and last four digits of a saved credit card to be accessed in bulk for any user that had ever signed up for an account. This includes my own personal data! Despite an explicit acknowledgement of the issue and a promise to fix it, Panera Bread sat on the vulnerability and, as far as I can tell, did nothing about it for eight months.

Houlihan emailed Mike Gustavision -- then Panera's head of security -- about the vulnerability. Like many other discovered data leaks, all a user had to do was alter digits in company's online ordering site to view other people's personal information. Users did not even need a Panera account to do this.

Houlihan's notification attempt was greeted with derision by Panera's security head. [Click for a larger version.]

Dylan,

My team received your emails however it was very suspicious and appeared scam in nature therefore was ignored. If this is a sales tactic I would highly recommend a better approach as demanding a PGP key would not be a good way to start off. As a security professional you should be aware that any organization that has a security practice would never respond to a request like the one you sent. I am willing to discuss whatever vulnerabilities you believe you have found but I will not be duped, demanded for restitution/bounty or listen to a sales pitch.

Eventually, Gustavision provided a PGP key and allowed Houlihan to send him info on the site's vulnerability. But, as Houlihan points out, this is no way to treat someone reporting a possible breach. Not only was the immediate response needlessly combative, the company's response to the notification was to do nothing until it was publicized by other security researchers.

This was contrary to Gustavision's statements to Houlihan, which claimed Panera's security team was "working on a response." That was the claim last August. Houlihan continued to check the site since his own information was included in what was exposed and nothing changed until April of this year, eight months after being notified.

Somehow, Panera was magically on top of the situation when it went mainstream. After Brian Krebs spoke to the company's CIO about the breach, Panera briefly took its site offline for maintenance. It then declared it had fixed the hole within two hours of notification, glossing over the fact it had been notified eight months earlier and done nothing. It also downplayed the problem as only affecting a small portion of Panera customers.

Almost minutes after this story was published, Panera gave a statement to Fox News (no link will be provided) downplaying the severity of this breach, stating that only 10,000 customer records were exposed.

In essence, it lied to press outlets seeking comment. Security researchers noted the problem hadn't even been completely fixed yet.

Almost in an instant, multiple sources — especially @holdsecurity — pointed out that Panera had basically “fixed” the problem by requiring people to log in to a valid user account at panerabread.com in order to view the exposed customer records (as opposed to letting just anyone with the right link access the records).

And it was far, far bigger than Panera publicly claimed. Krebs initially estimated the exposed records at 7 million. Additional research by Krebs showed multiple divisions of Panera were affected by the same vulnerability (like its online catering service). After examining APIs used by Panera's online services, Krebs estimates close to 37 million records have been exposed.

What will Panera learn from this? Whatever it does learn won't spread to other companies, that's for certain. Breach after breach has shown us companies are willing to shoot the messenger, cover up the damage, ignore repeated notifications, and obfuscate when breaches are finally exposed. Panera didn't handle breach notification worse than other companies have. It just did as little as possible until forced to confront the problem. This mindset is shared by far too many entities. They love scooping up personal data, but not the security responsibility that comes with it.

Filed Under: data breach, leaks, security
Companies: panera