Facebook Screws Up Again
from the please-make-it-stop dept
Another day, another Facebook privacy scandal.
This time around, a "senior Facebook employee" has informed security expert Brian Krebs that Facebook has been storing the passwords of "hundreds of millions" of Facebook (and Instagram) users in plain text (aka unencrypted). This is a fundamental security error that no company should ever make, yet it's been a pretty common occurrence for tech companies where security and privacy are commonly seen as an afterthought. According to Krebs, the passwords were accessible to around 20,000 Facebook employees for the better part of the last decade:
"The Facebook source said the investigation so far indicates between 200 million and 600 million Facebook users may have had their account passwords stored in plain text and searchable by more than 20,000 Facebook employees. The source said Facebook is still trying to determine how many passwords were exposed and for how long, but so far the inquiry has uncovered archives with plain text user passwords in them dating back to 2012.
My Facebook insider said access logs showed some 2,000 engineers or developers made approximately nine million internal queries for data elements that contained plain text user passwords."
On the "plus side," this latest scandal is slightly less terrible than past scandals like the Cambridge Analytica fracas. In those instances, the scandals made it clear Facebook routinely viewed consumer privacy as a distant afterthought as it looked to monetize every brain fart of its userbase. In this case, insiders told Motherboard that this does appear to have been a bug, and that the majority of Facebook passwords are usually encrypted:
"A current Facebook employee told Motherboard that "it sucks."
"Obviously we don’t store them in plaintext ‘normally,’" the employee, who has a technical role, told Motherboard. "Logged in plaintext in some unique weird cases we found and fixed and are talking about." Motherboard granted multiple sources in this story anonymity to speak more candidly about a security incident.
"It should’ve never happened," they said.
Still, given Facebook's resources and the volume of security talent they have on staff, the fact that it happened at all is grossly embarrassing. The scandal comes right on the heels of Facebook's other recent scandals -- like its cavalier sharing of user health and real estate data -- and is only compounding a scandal-ridden 2018 for the company. Krebs stated that as many as 600 million of the company's 2.7 billion users could be affected by the company's latest screw up, though, thus far, Facebook has yet to notify any of the impacted users.
Facebook was quick to issue a blog post amusingly entitled "keeping passwords secure," before confirming that Facebook failed to do precisely that. Throughout the post Facebook's Pedro Canahuati downplays the scope of the threat, while remaining somewhat murky on how many people were actually impacted:
"To be clear, these passwords were never visible to anyone outside of Facebook and we have found no evidence to date that anyone internally abused or improperly accessed them. We estimate that we will notify hundreds of millions of Facebook Lite users, tens of millions of other Facebook users, and tens of thousands of Instagram users. Facebook Lite is a version of Facebook predominantly used by people in regions with lower connectivity."
Given that this data was available to 20,000 employees over a period of roughly seven years, the claim that they've found "no evidence" of abuse should be of cold comfort. The company, meanwhile, continued to insist that consumer privacy is among its top priorities:
"In the course of our review, we have been looking at the ways we store certain other categories of information — like access tokens — and have fixed problems as we’ve discovered them. There is nothing more important to us than protecting people’s information, and we will continue making improvements as part of our ongoing security efforts at Facebook."
At this point it's fundamentally obvious that has never actually been true. And while that may be true now that the company is staring at looming regulation and mammoth fines all around the globe, at this point Facebook would need to be able to go a week without a major privacy scandal before any sentient being would take those claims at face value. In the interim, if you're not using a decent password manager and unique passwords on every website you visit, you might just want to get on that.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: passwords, plaintext, privacy
Companies: facebook
Reader Comments
Subscribe: RSS
View by: Time | Thread
Password logging
The "plain text" file was presumably some debugging or webserver log. Elsevier recently made a similar mistake with Kibana, even making all such recorded passwords public.
https://news.ycombinator.com/item?id=19423770
[ link to this | view in chronology ]
Re: Password logging
From the linked article: "Facebook is probing a series of security failures in which employees built applications that logged unencrypted password data for Facebook users and stored it in plain text"
Karl is likely wrong about this part:
The quote does not support that statement. It says they don't normally store them in plaintext. If they're at all competent, which remains to be seen, they'd be stored hashed, not encrypted. The important difference is that Facebook cannot reverse a hash to learn someone's password.
[ link to this | view in chronology ]
Hmm... let's see.
How many people are on Facebook? Google says around 1 billion. That's 1,000 million, of which between 20-60 percent had this happen to them. That sure sounds like "normally" to me. (Especially when you consider the first rule of data breaches: it's always bigger than they realize at first!)
And yet Facebook officially claims that:
War is peace! Freedom is slavery! Ignorance is strength!
[ link to this | view in chronology ]
Re:
I don't think that's necessarily what's going on here, though I'm far from sure that Facebook deserves the benefit of the doubt. What it sounds like, based on the anonymous dev who's quoted, is that passwords were written in cleartext to an error log file, presumably along with lots of other information about the user's environment and what they were doing. Devs (2000 of them, apparently) queried those log files, and the results of those queries included the passwords. But there is (allegedly) no evidence that those passwords were misused. Believe it or not, it's at least facially plausible, and it's reasonably consistent.
[ link to this | view in chronology ]
Re:
Fixed that for you.
[ link to this | view in chronology ]
Re: Re:
Yes, this is also true. But I meant it the way I wrote it: it's very common for the damage to be larger than they actually realize at first.
[ link to this | view in chronology ]
Re:
If this isn't enough reason to scuttle fb and have zuck plucked from his poshness and arrested, then it'll probably never happen.
[ link to this | view in chronology ]
Password encryption vs. hashing
Small nitpick: passwords aren't usually encrypted, but hashed, so that the password can still be compare to the one the user signed up with, but the original password can't be extracted out of it. (Or at least not in an easy way.)
[ link to this | view in chronology ]
Re: Password encryption vs. hashing
too bad "hashing" isn't just a one way version or another word for "encryption" where the user still has access to the public key portion to encrypt and compare words to the original (without being able to decode the original)... Glad nobody is being pendantic...
[ link to this | view in chronology ]
Re: Re: Password encryption vs. hashing
Umm... it isn't. Encryption is reversible, while hashing is not. The use of the term "public key" implies the existence of a private key, but there is no private key with a hash.
With the way hashes work, reducing an input of arbitrary size to an output of a fixed size, (the input is usually larger than the output, though of course with passwords this isn't always the case,) it can be trivially shown via the Pigeonhole Principle that there exist an infinite number of inputs that will hash to the same output. While it's true that the purpose of cryptographic hash design is making it as difficult as possible to find such "colliding" inputs, they are mathematically required to exist. And therefore, if there are infinite different inputs that can yield the same hash output, it is impossible to "decrypt" a hash and say "this is the (singular) input that it came from."
Hashing is not in any way "another word for encryption," and claiming that it is makes people who actually understand the principles involved cringe at your ignorance.
[ link to this | view in chronology ]
"Still, given Facebook's resources and the volume of security talent they have on staff, the fact that it happened at all is grossly embarrassing"
There's a couple of things I can think of here, though. First, Facebook is very much a developer-led organisation that has experienced incredible, unprecedented growth at various points in its short history. It would not be unusual for security to have taken a backseat in such an environment. If security is less a part of the corporate culture than adding new features and trying to find ways to monetise users, they probably won't have been going back to patch holes opened by bad process or design until a breach is noticed. That's just the way a lot of these companies work.
Second, unless I'm mistaken the lion's share of the issue is reported to be affecting Facebook Lite, a version of the app directly targeted at people in developing countries. So, reading between the lines, the issue was probably introduced during some push to expand global reach, but then not fixed because a lower per-user income metric meant they weren't a priority. Meaning that it's likely that certain parts of the company weren't getting the resources and talent you assume they did.
I could be wrong with all of the above, but having worked for larger companies I can certainly say that it's the norm for boring things like security and painful things like best practices to get the most corner cutting, especially in departments not considered the main revenue generators.
[ link to this | view in chronology ]
Re:
On top of it given how they handled user data it's no wonder they'd fail at other parts as well jut for the profits.
[ link to this | view in chronology ]
True story bro
This is a 100% true statement. Yet this:
Is a misinterpretation of that statement. Facebook wants to protect the data they harvest because it is valuable, user privacy is barely a tertiary concern because FB never releases information, they sell it. You cannot sell what has been given away for free through malfeasance or blunder or intentionally. They on count us being gullible idiots to misinterpret their words, then pat themselves on the back when we do.
[ link to this | view in chronology ]
but sockpuppets...
But how else could Facebook manipulate and deceive the majority of users if the weren't able to steal individual users identities to post 'sock puppet' type information...
Maybe it's just an off color remark or a leading subject (how about those Swasticas?), maybe it's an ad for an opposing candidate or side.
How can Facebook keep the 'discussions' flowing if they aren't allowed to inject their own opinions (under their users names/accounts... but I'm SURE this has NEVER EVER EVER happened, we pinkey swear)...
[ link to this | view in chronology ]
Re: but sockpuppets...
"the weren't able to steal individual users identities to post 'sock puppet' type information"
Do you have any actual evidence this has happened?
I mean, it's dumb to think that the people who have root access to the servers and databases themselves would need a user password in the first place, but I'm sure if you're going to accuse them of such things you must know it happened, right?
[ link to this | view in chronology ]
Re: Re: but sockpuppets...
It makes sense that fb would want employees to be able to access user accounts for whatever purpose fb wants. It does not seem that this was an erroneous error, but calculated access.
[ link to this | view in chronology ]
Re: Re: Re: but sockpuppets...
Maybe fb would take a block of these accounts and offer for enticement to procure sale of data. Their history of mishandling their own users' data would make anyone suspicious of this. You let enough rain fall on a company that put off protecting itself with paint and it'll start rusting fast.
[ link to this | view in chronology ]
Re: Re: Re: Re: but sockpuppets...
Maybe a lot of things, you can make up whatever shit you want at this point. But the fact remains you don't have to make up wild stories about how people get into accounts.
Every single organisation you deal with, no matter how big or small, has somebody who can access the data collected by that company at a root level. If this surprises you, you are exceeding uninformed about how things work. No conspiracy theory is required - whatever company you have dealt with someone has a means by which they could have accessed your data. Some companies are better than others at reducing the number of people for whom this is possible, but someone always has the potential.
[ link to this | view in chronology ]
El Reg's take on this in its usual snarky manner.
[ link to this | view in chronology ]
Re:
"antisocial network's latest Zuck-up"
I love it!
[ link to this | view in chronology ]
Re: Re:
Fb will just have to zuck it up and keep on zucking!
[ link to this | view in chronology ]
Alternatively Titled...
...Another Day Ending in "Y"...
[ link to this | view in chronology ]
Oh look, Masnick shilling for Facebook again...
... wait, where'd all the usual trolls go?
[ link to this | view in chronology ]
Re: Oh look, Masnick shilling for Facebook again...
Masnick didn't write this.
If he had, you can be sure it would have been littered with excuses, downplaying of the incident, or something about how GDPR somehow caused this.
[ link to this | view in chronology ]
Re: Re: Oh look, Masnick shilling for Facebook again...
Found the troll!
Seriously, we get it, the thought that Masnick lives makes you piss in your own oatmeal every morning...
[ link to this | view in chronology ]
Why did facebook ever have the passwords to begin withto begin with? The usual way of storing psswords is to encrypt a block of 0's using the password as the key (at least on unix and I'm pretty sure on VMS). the only thing that is stored is the encrypted block of zeroes. The password used to encrypt it never gets stored anywhere. When someone logs in and enters a password, the password then encrypts a block of zeroes, and THAT is what gets compared against the stored password.
This has been common as far as I can remember, which like 30 years, so why did facebook ever have the passwords to begin with? Not only should they not have the passwords stored in plaintext, they should not have the passwords stored in an encrypted form that they can decrypt.
[ link to this | view in chronology ]
Re:
The passwords at issue were probably captured in plaintext from within password handling code via an internal development mechanism (such as a debugging/trace log, for instance). As a result, the plaintext store the article refers to was not their "Master Password Table", but some debug.log file dumped onto a server's hard disk.
[ link to this | view in chronology ]
Re: Re:
Facebook should never have had the plaintext passwords even for debugging, since that would not be needed for any debugging. There is simply no excuse at all for ever having anyone's password stored anywhere in any way.
[ link to this | view in chronology ]
So Facebook internal employees had access to all those accounts, with everything in them. This would have to be including those accounts that were set to Private/Friends only, and not subject to sharing, so if some of those photos ended up being publicly exposed (we know how people are about such things), what recourse do end-users have? Civil suit for privacy violation?
[ link to this | view in chronology ]
Re: So Facebook internal employees had access to all those accou
I hate to be the bearer of bad news, but regardless of the platform you are using, there will be people that have access to everything. Aside from the planet9 operating system, I know of no other operating system that doesn't require some form of admin or superuser that has total access to everything, for maintenence reasons, if nothing else. And, how else do you think they obtain files requested by law enforcement?
So, if you have an account on any platform anywhere, you should assume that some people other than you can access anything you upload or post. Whether or not they are allowed to is a different question, but lack of permission never has been much of an impediment.
[ link to this | view in chronology ]
Re: Re: So Facebook internal employees had access to all those a
OOOOOr.. what if there is some internal grand scheme here by a very lofty employee who has been secretly selling data to very questionable types, but very wealthy questionable types and that employee was setting up buyers with passwords to a certain amount of users' data in order to make a tidy sum under the table? Could it be?
[ link to this | view in chronology ]
Re: Re: Re: So Facebook internal employees had access to all tho
It could be. But, there are a great many other stories that make a lot more sense and have a lot more evidence to support them.
[ link to this | view in chronology ]
Re: Re: So Facebook internal employees had access to all those a
"I hate to be the bearer of bad news, but regardless of the platform you are using, there will be people that have access to everything."
Yes, this is true.
The sys admin will have access to the shadow password file in which hashed passwords are kept. The sys admin will not have access to any passwords in the clear because there are none on the system.
[ link to this | view in chronology ]
Re: Re: Re: So Facebook internal employees had access to all tho
That is true, in a well designed system with security built in from the ground up, at least. Although that doesn't preclude him from changing a user password or bypassing security in other ways.
Now, what about people with greater access than he does?
[ link to this | view in chronology ]
Re:
"So Facebook internal employees had access to all those accounts, with everything in them"
Yes, Facebook employees have the root passwords for all server and databases they control, meaning that someone in the organisation has access to everything at a fundamental base level.
If this scares you, you need to stop dealing with any company that has an IT staff.
[ link to this | view in chronology ]