Report Exposes Flaws In Link Shorteners That Reveal Sensitive Info About Users And Track Their Offline Movements

from the sna.fu dept

Fri, Apr 15th 2016 9:38am — Tim Cushing

URL shorteners: not just for malware/spam delivery anymore!

TL;DR: short URLs produced by bit.ly, goo.gl, and similar services are so short that they can be scanned by brute force. Our scan discovered a large number of Microsoft OneDrive accounts with private documents. Many of these accounts are unlocked and allow anyone to inject malware that will be automatically downloaded to users’ devices. We also discovered many driving directions that reveal sensitive information for identifiable individuals, including their visits to specialized medical facilities, prisons, and adult establishments.

The Freedom to Tinker Foundation has just released a study it compiled over the last 18 months -- one in which it scanned thousands of shortened URLs and discovered what they unintentionally revealed. Microsoft's OneDrive -- which uses link-shortening -- could be made to reveal documents uploaders never intended to share with the public. Worse, Freedom to Tinker discovered a small percentage of brute-forced URLs linked to documents with "write" privileges enabled.

Around 7% of the OneDrive folders discovered in this fashion allow writing. This means that anyone who randomly scans bit.ly URLs will find thousands of unlocked OneDrive folders and can modify existing files in them or upload arbitrary content, potentially including malware.

And, because Microsoft's automatic virus/malware scanning for OneDrive contents is less than robust, it wouldn't take much for any random person to wreak havoc on any number of devices with access to those contents.

OneDrive “synchronizes” account contents across the user’s OneDrive clients. Therefore, the injected malware will be automatically downloaded to all of the user’s machines and devices running OneDrive.

Fortunately for OneDrive users, the scanning method deployed by FTTF no longer works as of March 2016. But this doesn't necessarily mean the accounts are completely secure -- just that one avenue for attack/access has been closed.

Just as disturbing -- but for different reasons -- is the automatic link shortening tied to Google Maps. The links could be manipulated to discover all sorts of inferential information about people's private activities… or at least the activities they never thought they were sharing with the world. The directions and searches uncovered by FTTF's scanning activity potentially reveal plenty of sensitive information about Google Maps users.

Our sample random scan of these URLs yielded 23,965,718 live links, of which 10% were for maps with driving directions. These include directions to and from many sensitive locations: clinics for specific diseases (including cancer and mental diseases), addiction treatment centers, abortion providers, correctional and juvenile detention facilities, payday and car-title lenders, gentlemen’s clubs, etc. The endpoints of driving directions often contain enough information (e.g., addresses of single-family residences) to uniquely identify the individuals who requested the directions. For instance, when analyzing one such endpoint, we uncovered the address, full name, and age of a young woman who shared directions to a planned parenthood facility.

The same privacy concerns associated with the indiscriminate use of automatic license plate readers by law enforcement and warrantless access to cell site location info are present here: the reconstruction of people's lives via the "tracking" of their movements. In this case, however, the information generated is more "voluntary" than either of the other listed collections, which are far more passive than searching for directions using a web service provided by a company with an unquenchable thirst for data.

The good news is that the method deployed for the report no longer works for Google Maps-shortened links. But, once again, that does not mean the problems with link shorteners have been eliminated. FTTF points out that the March 2016 change by Microsoft (which claims it had nothing to do with FTTF reporting the vulnerability to it) only affects links generated after that date. Any previous short URLs are still vulnerable to traversal scans.

Google, however, did make a more of a serious attempt to prevent abuse of its shortened links.

All newly generated goo.gl/maps URLs have 11- or 12-character tokens, and Google deployed defenses to limit the scanning of the existing URLs.

While this news should be of concern to users of these services, it definitely has to be great news for law enforcement and intelligence agencies. So much for "going dark." Vulnerabilities in web services apparently provide access to otherwise "locked" cloud storage contents and Google Maps -- at least until it was fixed -- generating tons of location data for the taking.

It's also worth pointing out that the method used by Freedom to Tinker to complile this report is basically the same method used by Andrew "Weev" Auernheimer to expose AT&T users' email addresses: altering URLs to uncover data presumed to be hidden. Of course, AT&T's vindictiveness resulted in a 3.5 year prison sentence for Auernheimer. No legal threats have been made towards FTTF, but the sad thing is that security research is inherently risky, as you can never tell whether the entity affected will respond with a bug fix or a police report -- not until after they've been informed.

Url-Shortener (PDF)Url-Shortener (Text)

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: link shorteners, onedrive, privacy, vulnerabilities
Companies: google, microsoft

38 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

pegr, 15 Apr 2016 @ 9:42am

You likely won't remember, but...
Hi Mike!

You likely won't remember a couple of years ago when I chastised you for using URL shorteners. That's when you started putting "[url]" at the end of your links. That said, I just have one comment.

Told ya! :)
[ link to this | view in thread ]
Richard (profile), 15 Apr 2016 @ 9:45am

Feymann
as you can never tell whether the entity affected will respond with a bug fix or a police report -- not until after they've been informed.

This attitude is not new.

Richard Feynmann encountered it at Los Alamos 70 years ago when he explained how the safes could be cracked.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 10:00am

Re: Feynmann
It may not be new, but it is still a problem. If security research were less risky, you might have more white hats working on it and disclosing the information to the affected vendors. As is, an unknown number of technically competent researchers are not reporting, and may even be avoiding attempting to discover, vulnerabilities because they do not want to deal with the hostile responses. When I see people contemplating an attempt to find a vulnerability, I actively remind them to consider the potential for hostile responses before they embark on the research.

Personally, I make it a point not to explore things that look like they might be weak, because I cannot prove the weakness without it looking like a hack attack if someone is monitoring the target.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 10:29am

If anyone is a software developer, a great alternative for sharing location is Geo Uri which requires no internet service. Adding support for it in your apps would go a long way to helping solve this problem.

Ex. geo:40.68924,-74.04454

* https://en.wikipedia.org/wiki/Geo_URI_scheme
[ link to this | view in thread ]
Jack, 15 Apr 2016 @ 10:29am

Re: Re: Feynmann
When your choices are a. Report to company, possibly go to jail or b. Report to THT, make thousands of dollars - it isn't hard to deduce why so we have so many persistent issues with vulnerabilities.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 10:38am

Re:
geo:40.68924,-74.04454
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 10:57am

Re: You likely won't remember, but...
I'm not seeing the privacy risk there? If someone followed the link they would see... a Techdirt article?
[ link to this | view in thread ]
Bob Webster (profile), 15 Apr 2016 @ 11:04am

It's hard to believe so many people consider the use of shortened URLs a security measure. It is not, and was never intended to be. A URL is exposed, by definition, whether long or shortened. A shortened URL is a convenience, not a security tool. Some people misuse base64 encoding for "security" as well, but it does not mean we should get rid of base64 encoding.
[ link to this | view in thread ]
PRMan (profile), 15 Apr 2016 @ 11:15am

Lengthening?
The article suggests lengthening as the solution. Really?

The whole point is that these are short enough for Twitter.

I suggest in 7-character bit.ly, they start using the whole keyspace instead of a 1 at the front of another 6-character keyspace. That would help.

But it was odd to me that all the suggestions were pointed toward the shortening providers, as if the users had no responsibility whatsoever.

How about:

1) Don't put your documents in the cloud.
2) If you do put your documents in the cloud, OneDrive isn't secure.
3) Don't make shortened URLs to your documents. And don't share document URLs online.
[ link to this | view in thread ]
Jeremy Lyman (profile), 15 Apr 2016 @ 11:21am

Loose Lips
"the address, full name, and age of a young woman who shared directions"
Not necessarily. When I share links to directions I hardly ever put where I am in them. Most times they contain where my friend is starting and where they want to go. So this may be the address of a young woman whose friend punched her information into Google and exposed her to a privacy violation. Which is arguably a bit worse.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 11:37am

Uh
How the hell is this a flaw in link shorteners?
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 11:44am

Re: Lengthening?
Users are dumb. Developers have a responsibility to protect their users, especially if they are providing a service.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 11:49am

Re: Loose Lips
Why link to directions? Unless you're sending a parade route the only thing they need is a destination. They can figure out their own route using tools they choose.
[ link to this | view in thread ]
nasch (profile), 15 Apr 2016 @ 11:51am

Re: Lengthening?
The article suggests lengthening as the solution. Really?

I must have missed that part. Where did he suggest that is the solution?
[ link to this | view in thread ]
nasch (profile), 15 Apr 2016 @ 11:54am

Scanned
TL;DR: short URLs produced by bit.ly, goo.gl, and similar services are so short that they can be scanned by brute force.

"Scanned" seems a strange description, wouldn't "guessed" be more accurate?
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 11:54am

Re: Re: Lengthening?
The first sentence.
[ link to this | view in thread ]
Jeremy Lyman (profile), 15 Apr 2016 @ 11:55am

Re: Uh
The only flaw I can think of is their permanence or inability to remove a link. Besides the standard "can't see where the hell this thing is taking me" issue.

Otherwise, yes this is a PSA. Short urls can be guessed!
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 11:55am

Re: Scanned
Nah, it's the internet. Personally I would have used hacked or cyber-pillaged.
[ link to this | view in thread ]
Jeremy Lyman (profile), 15 Apr 2016 @ 12:06pm

Re: Re: Loose Lips
Off the top of my head I can think of exercise routes on trails or roads that hit a certain mileage, or a detour that takes a road not naturally recommended. Sometimes addresses can be geolocated incorrectly so you might use lat/long coordinates instead. I'm sure there are other reasons beyond just generally being helpful.

The main point is that it's something people do, whether there's a good reason or not, and something that google specifically encourages with a checkbox.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 12:06pm

I would be interested to know if Facebook picture URLs are guessable by a nation state.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 12:13pm

Re: Re: Re: Loose Lips
Maybe I' not parsing your previous comment correctly, but it sounds like you said you do things that in the next sentence you consider a privacy violation. Or do you always ask for consent first before entering other's location data into a public service?

It's totally okay if you received informed consent to use data in that way, but if these are things we consider important shouldn't we strive to protect everyone's data not just our own?
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 12:16pm

Re: Lengthening?
1) Don't put your documents in the cloud.

That requires that people cam either get a fixed IP address for their home connection, or use a dynamic DNS services to allow use of a private server. Also, if using windows it is a right pain to set up to use SSH for a secure connection to a home server. It almost as if the the proprietary operating system vendors and ISPs are doing their best to ensure that private individuals cannot have any privacy.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 12:17pm

Re: Re: Lengthening?
May I suggest:https://syncthing.net/
[ link to this | view in thread ]
Uriel-238 (profile), 15 Apr 2016 @ 12:24pm

So I can still use shorteners...
...to link to YouTube videos and Wikipedia articles, but might not want to trust other people's shortened links. Yes?

Of course, the only reason I have generally to use short URLs is if I were posting links on Twitter.

For anything else with less sparse space, I use DDG Bangs. Exempli gratia:

https://duckduckgo.com/?q=techdirt+!w
https://duckduckgo.com/?q=techdirt+moe+anthropomorphic+m ascot+!i

Why doesn't Techdirt have a moe-tan anthro mascot? Do you not love Japan?
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 12:32pm

Re: So I can still use shorteners...
Don't trust a URL that requires no authentication to view to be private. It's not really that complicated.
[ link to this | view in thread ]
Jeremy Lyman (profile), 15 Apr 2016 @ 12:33pm

Re: Re: Re: Re: Loose Lips
Ah, I see! Sure, my behavior could be considered that way. I was just using my experience to point out that the personally identifiable information (pii) they gathered wasn't necessarily submitted by that person.

I don't generally use url shorteners or route people to medical facilities, so my friends should be okay. But it's something that most people probably don't consider; these services take a snapshot of your private browsing history (whatever information is currently in url parameters) and make it publicly accessible and discover-able.
[ link to this | view in thread ]
nasch (profile), 15 Apr 2016 @ 1:24pm

Re: Re: Re: Lengthening?
This? "TL;DR: short URLs produced by bit.ly, goo.gl, and similar services are so short that they can be scanned by brute force." That doesn't suggest a solution, it describes a problem.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 1:47pm

Re: Re: Re: Re: Lengthening?
Time to go back to school?
[ link to this | view in thread ]
nasch (profile), 15 Apr 2016 @ 2:03pm

Re: Re: Re: Re: Re: Lengthening?
No idea what you're talking about.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2016 @ 2:20pm

Re: Re: Re: Re: Re: Re: Lengthening?
That's my point.
[ link to this | view in thread ]
Rich Kulawiec, 15 Apr 2016 @ 2:47pm

"link shorteners" are scams and abuse magnets
I've had a blacklist-on-sight policy in place for them for years and strongly recommend the same to everyone else. Most of them are scams; most of them are run by spammer and phishers and equally filthy scumbags; and NONE of them does even a token job of abuse control. (Of course they don't: abuse is their business model.)
[ link to this | view in thread ]
Uriel-238 (profile), 15 Apr 2016 @ 3:59pm

Not the guy who usually brings up the obvious joke
...Short URLs produced by bit.ly, goo.gl, and similar services are so short that they can be scanned by brute force.

So size does matter.
[ link to this | view in thread ]
John Fenderson (profile), 17 Apr 2016 @ 9:25am

Link shorteners are a terrible idea
The flaw being discussed here is one problem with them. However, there are (at least) two additional problems that are at least as bad:

1. They eliminate transparency. It's impossible to know where the link goes without clicking on it. This is 100% unacceptable and makes it trivially easy to trick people into clicking malicious links.

2. They unnecessarily involve a third party who now has access to which IP addresses are requesting which links.

Personally, I never follow links that go through a shortener and I urge others to follow the same practice.
[ link to this | view in thread ]
John Fenderson (profile), 17 Apr 2016 @ 9:27am

Re: Re: Re: Re: Re: Re: Re: Lengthening?
Nasch isn't the only one who doesn't know what you're trying to say. Why not just say it?
[ link to this | view in thread ]
John Fenderson (profile), 17 Apr 2016 @ 9:32am

Re: Re: Lengthening?
"That requires that people cam either get a fixed IP address for their home connection, or use a dynamic DNS services to allow use of a private server"

Not as much as you might think. While it's true that your dynamic IP address can change, in practice it rarely actually does. And it almost never changes unless you disconnect and reconnect.

I know a lot of people who run an SSH server from their home machines for remote access, but don't have a dedicated IP and don't use DDNS services.

"if using windows it is a right pain to set up to use SSH for a secure connection to a home server."

This is just not true. Perhaps it used to be years ago, but setting this up on Windows now is pretty simple. It takes about 5 minutes.
[ link to this | view in thread ]
Wendy Cockcroft, 18 Apr 2016 @ 5:54am

Re: Link shorteners are a terrible idea
I get what you're saying, John, but when the link is really long it's handy to have a shortener. What if the shortening service scanned the URL for malware, then added a title attribute so that, when you hovered your mouse over it, the actual destination would display in a little box?

E.g. my Medium account.
[ link to this | view in thread ]
John Fenderson (profile), 18 Apr 2016 @ 7:37am

Re: Re: Link shorteners are a terrible idea
I totally understand the convenience and utility of link shorteners! But I'm arguing that the benefit is generally not worth the cost.

As to using malware scanners, that would be better than nothing, but malware scanners are not anywhere near good enough to be an adequate solution by themselves.

Adding an attribute that tells you the real URL is a bit pointless because that means that the real URL has to be encoded, so you aren't saving any bytes -- you're actually increasing the number of bytes required because you have to include both the shortened link and the unshortened link. Also, it is a weak move because the URL you'd see would not be the one in use. You are effectively having to take the service's word for it, which means that it's a point of failure that when exploited would make everything even more dangerous by giving you a false sense of comfort.

Also, the problem of unnecessary data leakage remains unaffected by those proposed solutions.

The entire point of link shorteners is one of obfuscation: hide the real URL while presenting you with a shorter encoded one. In my opinion, this is an unacceptably risky proposition.

There is one situation where I'm completely comfortable with them, though: if I were the one running the shortening service, then I'd be much more comfortable with using it, because I retain in control of my data and I can do security audits.
[ link to this | view in thread ]
Jeremy Lyman (profile), 19 Apr 2016 @ 9:26am

Re: Re: Re: Link shorteners are a terrible idea
Yup! I'm much more likely to set up a redirect page on one of my domains than use a shortening service. Not in response to any specific flaw, but because I get to decide how long the url is active and I can even assign it a meaningful address. It's obviously not for everyone, but can be handy if you're able.
[ link to this | view in thread ]