Latest Leak Shows NSA Can Collect Nearly Any Internet Activity Worldwide Without Prior Authorization
from the the-NSA-should-really-stop-issuing-denials dept
The newest NSA leak has just been posted at the Guardian and it gives credence to Snowden's earlier claim the he could, "from his desk," wiretap nearly anyone in the world. US officials, including NSA apologist/CISPA architect/Internet hater Mike Rogers, denied Snowden's claim, with Rogers going so far as to call the former NSA contractor a liar. The documents leaked today seem to indicate otherwise.
A top secret National Security Agency program allows analysts to search with no prior authorization through vast databases containing emails, online chats and the browsing histories of millions of individuals, according to documents provided by whistleblower Edward Snowden.Greenwald isn't kidding about the "broad justification." The slides tout the breadth of the search program, which provides results other programs can't. As is stated in the opening slides, XKeyscore allows agents to pull up tons of data (in search of "anomalies") and work backward to refine the results. The justification for these broad searches is available via a pulldown menu, as can (sort of) be seen in this screenshot, which gives agents a variety to choose from. (From the list, it appears that anything ending with "outside the US" is fair game.)
The NSA boasts in training materials that the program, called XKeyscore, is its "widest-reaching" system for developing intelligence from the internet.
[T]raining materials for XKeyscore detail how analysts can use it and other systems to mine enormous agency databases by filling in a simple on-screen form giving only a broad justification for the search. The request is not reviewed by a court or any NSA personnel before it is processed.
XKeyscore utilizes a variety of plugins to allow searches, including email addresses, phone numbers, IP addresses, full logs of every DNI session and machine-specific cookies. This gives agents an advantage other surveillance programs don't.
The purpose of XKeyscore is to allow analysts to search the metadata as well as the content of emails and other internet activity, such as browser history, even when there is no known email account (a "selector" in NSA parlance) associated with the individual being targeted.The slides warn that the data collected will be too large to parse (or even store for a great length of time). It recommends harvesting first and "selecting" second, in order to refine the results (using a "Strong Selector"). Agents are directed to look for "anomalous events," some of which seem a bit troubling.
Analysts can also search by name, telephone number, IP address, keywords, the language in which the internet activity was conducted or the type of browser used.
One document notes that this is because "strong selection [search by email address] itself gives us only a very limited capability" because "a large amount of time spent on the web is performing actions that are anonymous."
These "anomalies" are common enough that plenty of non-terrorists will be getting a second look from agents utilizing this program. And again we see the NSA's instant distrust of anyone using encryption. This is one of the hazards of "collecting it all" and then working backwards. It's easy to make common behavior look suspicious if you start at an end assumption and connect the dots in reverse.
- E.g., Someone whose language is out of place for the region they are in
- Someone who is using encryption
- Someone searching the web for suspicious stuff
Also troubling are some of the suggested applications of the search program shown in the slide deck, including "show me all the VPNs startups in Country X" and "show me all exploitable machines in Country X."
On top of this, there's the sheer breadth of the program.
The quantity of communications accessible through programs such as XKeyscore is staggeringly large. One NSA report from 2007 estimated that there were 850bn "call events" collected and stored in the NSA databases, and close to 150bn internet records. Each day, the document says, 1-2bn records were added.Because of the massive size of the data haul, metadata is retained and stored longer while more specific data is released. This still allows agents to perform broad searches to gather as much data as possible while relying on the stored metadata to put other connections together. Once they have the connections, the shallow search can be better utilized with the "strong selectors."
The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours."
The data harvested isn't solely relegated to foreign communications, no matter what the pulldown menu says. The power of the database pretty much guarantees the inadvertent collection of data on American citizens. This is exacerbated by the fact that some web traffic will be indeterminate in origin or termination. This leads to violations of the few laws that do pertain to NSA data collection, something the NSA documents admit is a problem. Of course, as Snowden pointed out, there's always a solution.
In recent years, the NSA has attempted to segregate exclusively domestic US communications in separate databases. But even NSA documents acknowledge that such efforts are imperfect, as even purely domestic communications can travel on foreign systems, and NSA tools are sometimes unable to identify the national origins of communications.Speaking of "justification," the slides claim that over 300 terrorists have been caught using XKeyscore. And the NSA has responded to the Guardian's leak with the usual claims that everything here is legal and audited, etc., which, again, doesn't make it right or even constitutional. It just makes it what it is: the end result of more than a decade's worth of expansion, secret law interpretations and compliant administrations.
Moreover, all communications between Americans and someone on foreign soil are included in the same databases as foreign-to-foreign communications, making them readily searchable without warrants.
Some searches conducted by NSA analysts are periodically reviewed by their supervisors within the NSA. "It's very rare to be questioned on our searches," Snowden told the Guardian in June, "and even when we are, it's usually along the lines of: 'let's bulk up the justification'."
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: metadata, nsa, nsa surveillance, search, surveillance, xkeystone
Reader Comments
Subscribe: RSS
View by: Time | Thread
[ link to this | view in thread ]
Words Fail Me
[ link to this | view in thread ]
Lets bulk up the justification
Whenever things weren't kosher, they wouldn't say no, they just directed the feds to put some more stuff in there. The FISA court isn't a court, it's a proofreader.
[ link to this | view in thread ]
Re: Words Fail Me
[ link to this | view in thread ]
Re: Re: Words Fail Me
[ link to this | view in thread ]
Lying with facts
The truth is that the wiretaps have *already* happened at that point. Snowden is inarticulately saying he could search the results of those wiretaps.
It's telling that even Snowden appears to have fallen unknowingly for the line that it's not a 'tap' until its searched.
[ link to this | view in thread ]
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Lying with facts
That all works until the details come out. Ooops. No wonder Snowden is a criminal instead of a whistleblower... he's ruining careers and making "good politicians" look bad.
[ link to this | view in thread ]
AGAIN without The Google.
So I repeat again. Emphasis added:
http://abcnews.go.com/blogs/politics/2013/07/glenn-greenwald-low-level-nsa-analysts-have -p owerful-and-invasive-search-tool/
NOTE especially the last bit about CONTINUOUS monitoring with updates. That's NOT just looking into Google's servers, that's ACTIVE participation by Google.
EVEN MORE, how can anyone continue to overlook how closely Google is tied into NSA as MAIN everyday feature, not just a few requests?
[ link to this | view in thread ]
Things that struck me from the power point presentation.
-If you are going to encrypt you're emails, chats, or phone calls, you're drawing attention to yourself. It behooves you to take all possible precautions with the rest of your internet activities. Don't go halfway!
-There are MAC addresses in Excel documents?
-It looks like the NSA analyzes HTTP headers and does browser fingerprinting. This can help to identify your computer even while going through a proxy.
[ link to this | view in thread ]
Re: AGAIN without The Google.
[ link to this | view in thread ]
Re: Re: Words Fail Me
[ link to this | view in thread ]
HTTPS everywhere
This lulls people into a false sense of security.
Let me point no further than the immediately preceding TechDirt article about how the NSA is in bed with American business, and how this hurts American business.
But first, let me digress. Remember sometime back all the controversy and outrage when Mozilla revoked the SSL signing certificates from a company that had issued root certificates to a third party? In that case, the third party was a company that made border routers for large networks. Those devices could then issue you a genuine signed certificate for, oh, let's just say, Amazon.com, and your web browser would believe it really was talking to Amazon.com. In reality, the intermediate router was what your browser was talking to. Then the router talked to Amazon.com on your behalf. This allowed the intermediate router to intercept, monitor, log or do anything else with your private traffic between you and Amazon.com.
At the time, the end result was that a lot of people began to wonder about just how much SSL and that green reassuring logo in your address bar should be trusted. If you want to Amazon.com, and your browser had a green trust logo, and you clicked it to inspect the certificate, and it was signed by, let's just say, Honest Achmed's Trusty SSL Certificates of Tehran Iran, would you believe that Amazon had purchased their SSL certificates from there?
Now back from my digression to the topic at hand.
Do you suppose that the NSA might secretly make secret arrangements with American certificate authorities (CA's) so that their secret private signing keys and or root certificates are secretly sent to the NSA so that the NSA can secretly play MITM (maniacal monster in the middle) games with your supposedly secure SSL traffic?
I would laugh myself silly if a subsequent leak revealed exactly that.
The entire underlying trust model of supposedly secure traffic on the internet would be broken. Who could trust anything over SSL? Who in other countries could trust American businesses ever again?
[ link to this | view in thread ]
Resigned.
All outrage does is eat me from the inside.
[ link to this | view in thread ]
Re: Re: AGAIN without The Google.
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: AGAIN without The Google.
[ link to this | view in thread ]
Re: Re: Re: Words Fail Me
They also have the power to do the greatest evil, and eventually it will happen if it hasn't already.
[ link to this | view in thread ]
Re: HTTPS everywhere
One of the repercussions concerning all this is that other countries might demand that Internet governance (i.e. ICANN and Verisign) no longer be U.S. based.
[ link to this | view in thread ]
You know this is a big story when ...
http://www.popsci.com/technology/article/2013-07/update-nsa-sucks-more-you-last-thought
[ link to this | view in thread ]
Reminds me of the US accusing huawei of hardwiring espionage stuff in their hardware. I'm sure everybody now have a big question mark concerning Cisco, IBM, Microsoft, Intel etc etc etc
[ link to this | view in thread ]
Re:
(Picture of Philip. J. Fry here)
[ link to this | view in thread ]
Locations
I'm not sure what the dots along Antarctica represent. Satellites?
[ link to this | view in thread ]
Page 24
I want to know if they have searched for any exploitable machines in the United States - and how they've used that information.
[ link to this | view in thread ]
Re: Re: HTTPS everywhere
I don't see any reason why this couldn't be done on a large scale. The computational requirements aren't terribly prohibitive (large, yes, but not prohibitively so). The main constraint would be bandwidth, not CPU cycles, and that's easy to mitigate by scattering your servers across the globe.
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Page 24
[ link to this | view in thread ]
Re: Re: HTTPS everywhere
Suppose I had secretly obtained the root signing certificate from a CA? For the following, I will use a fictional CA and call it VeriSlime.
Here is what my MITM device would need to do.
When you connect to Amazon.com, I first check if I have ever created a certificate for Amazon.com. If so, then I just use that fake cert to accomplish my MITM between you and Amazon.com.
But what if there is a cache miss? You connect to your small town bank site. I don't have that cert in my cache. So I make a connection to your small town bank site just to obtain its cert. I create a new cert with all the same properties, and sign it with VeriSlime's root cert key. That does not take very long to accomplish. And it only must be done once for that cert. I am not breaking any crypto -- merely performing some routine operations. Then using the new cert, I complete your original connection to your small town bank, but doing MITM, using the new cert.
Unless you are alert you might not notice that your bank certificates used to be signed by another CA, and now are signed by VeriSlime.
But if I was the NSA, I might have the root signing certs for every American CA. Then I could sign my impostor cert for your small town bank using the root cert from the same CA that your bank uses.
Now suppose that even though now I use the right CA to sign all my fake certs, you still notice the thumbprint has changed and might be suspicious. Or suppose the bank could insert JavaScript code in their page to check the cert and insure it is what they expected to see? Or like Google, the Chrome browser checks Google certificates to be sure that they really are what they should be?
Well, if I were the NSA, I might simply require every CA to give me a duplicate of any signing certificates that they issue to their customers. So (new application here...) when Microsoft buys a certificate (but this time instead of SSL, let's say a code signing certificate) the CA will issue me a copy of the certificate. That way I can sign any binary code I want it it REALLY IS signed by Microsoft. Now I can impersonate Microsoft's update servers, and have you do a Windows Update to my MITM server, and I could install any freakin' code I want onto your computer and it would be trusted!
What is so difficult to do on a large scale here if I had, say, a twenty person team of experts working on it, starting, say, five years ago?
One thing, using the Microsoft example, that Microsoft could do is to NOT use code signing certs issued by a CA. Set up their own internal CA that creates root certs, signs code, and put your trusted certs into your products (Windows, Office, etc) so that they only trust your own root certs and no third party is involved. But oh, wait -- Microsoft was working with the NSA either willingly or unwillingly. And Google. And everyone else.
[ link to this | view in thread ]
Re: Re: Re: HTTPS everywhere
[ link to this | view in thread ]
Re: Re:
No, really. Don't be afraid.
[ link to this | view in thread ]
Re: Page 24
[ link to this | view in thread ]
Re: Page 24
[ link to this | view in thread ]
Re: Re: Re: HTTPS everywhere
[ link to this | view in thread ]
Re: AGAIN without The Google.
[ link to this | view in thread ]
Re: Words Fail Me
Overall, what stands out as characteristics seems to me to be stupidity and an appalling lack of elegance. Typical Bush-era stuff perpetuated by the current weak administration.
[ link to this | view in thread ]
Re: Re: Re: HTTPS everywhere
Signing code is an entirely different matter. I have always wondered if Microsoft wasn't allowing the FBI/CIA/NSA use their update capability to install code on targeted machines. The, recently publicized, fact that Microsoft was selling or providing security vulnerabilities/exploits to the government undermines that suspicion.
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Re: Re: Re: Words Fail Me
No most of them probably THINK they are serving the greater good. Neo-con ideology really started to take hold out of trying to prevent another Pearl Harbor. Prior to that we felt our best policy was to mostly stay out of foreign conflicts apart from simply providing our allies with some requested support. When that sort policy failed twice to keep us safe, the thinking shifted to keeping tabs on and manipulation of foreign affairs as a means of minimizing the ability of situations where we could be attacked. In a nutshell, it's applying the theory of "the best defense is a good offense" to foreign affairs. The initial reasons behind it are still to keep Americans safe. The major downsides to it are doozies: 1. It tends to make a lot of enemies out of people that wouldn't otherwise consider you an enemy. 2. It fosters a us against the world mentality where you have to constantly overcome the collective strength of practically everyone else to make it work and keep it up in the long term. 3. If you can make it work, then those with the power to control the machine that manage it tend to become corrupted by the power that they have undermining all of the nobleness behind the initial ideology, which is where we are today.
[ link to this | view in thread ]
Re: Re: Re: AGAIN without The Google.
[ link to this | view in thread ]
Re: Lying with facts
[ link to this | view in thread ]
We the public have been lied to so often and frequently that trust is no longer possible. It is far and beyond time to end the Patriot Act and other laws put on the books that enabled this sort of massive spying. I'm not sure that just defunding NSA is enough.
What I am sure of is I don't recognize this country as the one I served in the military for. This country as it is being revealed begins to look more and more like Russia or China in it's keeping track of the populace. With absolutely no justifications beyond 'it might' qualifying. Give 'it might' to the paranoid and it becomes a certainly even through it is never proven to actually be so.
Enough is enough.
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: AGAIN without The Google.
http://www.wired.com/threatlevel/2013/07/google-neutrality/
[ link to this | view in thread ]
Pg. 28
Perhaps their claims that "we've stopped terrorist attacks with this surveillance!" might have a little credibility.
Of course, we'd be taking a leap of faith and assuming that all the terrorists the NSA has helped stop are actually terrorists who were planning to, well, cause terror among the general populace (American or otherwise), and not some unlucky bastard who got a nasty case of "mistaken identity" and was dragged in with the real threats because the government couldn't risk letting the guy go because he'd make a big fuss about everything.
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Its automagic!!
They just havent entered your name into their search box yet.
[ link to this | view in thread ]
My favorite part is at the very end...
"An audience member yelled “bullshit” at one point, while another shouted, “You lied to Congress, why do we believe you’re not lying now?” Both remarks received applause from the audience."
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Re: HTTPS everywhere
Yeah, they'd need a data center the size of a football field or two.
Like the one that's being built by the NSA... oh.
[ link to this | view in thread ]
Re: Re: Re: HTTPS everywhere
[ link to this | view in thread ]
Re: Re: Re: Re: HTTPS everywhere
Decryption (if you have the key) is a pretty fast operation. Encryption is a bit slower, but it's not crazy slow.
Mostly so that they don't have to rely on the cooperation of anybody. With a MITM attack, neither endpoint needs to help you, or even know that you're doing it.
[ link to this | view in thread ]
Re: Re: Re: HTTPS everywhere
[ link to this | view in thread ]
Re: Re:
How do you know?
There have been numerous instances of routers from various companies (including Cisco) having backdoors installed without the knowledge of the software devs and most of the hardware engineers.
[ link to this | view in thread ]
Re: Re: Re: Re: Re: HTTPS everywhere
[ link to this | view in thread ]
Re:
To use the logic of the company that created bank routing numbers and the trolls who defended them.
[ link to this | view in thread ]
Re: Pg. 28
As part of their story, I'd like to know where these captured terrorists are (Bagram? Abu Ghraib?) and who did the capturing (US or another nation).
[ link to this | view in thread ]
Re: Re: Re:
Can you point me to a discussion of backdoors for Cisco? Also, what other router companies are you thinking of?
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: HTTPS everywhere
I did not do benchmarks when I installed the crypto, so I can't give exact figures -- but whatever slowdown the crypto is causing was low enough that it was unnoticeable in practical usage.
[ link to this | view in thread ]
Re: Re: Re: Re:
I cannot give you a list of the routers that I know about personally right now, but a quick web search turns up a list of usual suspects, including Cisco.
Unless you're doing packet analysis of the traffic to and from a router while the back door is actually in use or attempt a known exploit and find that it succeeds, it's almost impossible to be sure that the router is not compromised -- even if you can guarantee that the higher-level code isn't.
This is one reason why I don't use commercial routers at all between my network and the internet (although I do use them for internal routing).
[ link to this | view in thread ]
Re:
It is very scary to think about. I rarely use proxies anyway, but fingerprinting is something I would rather avoid giving away to foreign parties, no matter the cause.
[ link to this | view in thread ]
www.wired.com/wiredscience/2013/01/dna-data-storage-2/
[ link to this | view in thread ]
Re: Re: Re: Re:
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Re: HTTPS everywhere
[ link to this | view in thread ]
Re: Re: Re: Re: Re:
I did a lot of testing of throughput including accounting for every single packet received on a port and where it went. These counts occurred in standard industry hardware outside of our proprietary ASICs and FPGA code. I would have noticed a discrepancy. If there was a backdoor in an ASIC it would still have to be triggered or configured by software. Even if there was a secret configurable register, there needed to be software that handled reads or writes to that specific interface. I knew all the low-level software. The only possibility I can see is if the compiler itself had been altered to add secret code to all the builds. I just find that hard to believe the company would go to that degree of trouble and risk screwing up any logic that would be impossible for most of the developers to debug.
[ link to this | view in thread ]
Re: Lying with facts
Artists use lies to tell the truth. Politicians use the truth to tell lies.
[ link to this | view in thread ]
NASA Spying
Well, when the Executive Branch, Congess and the courts provide a reason as to why we should trust them on this issue, then maybe we won't have a problem. Implicit in Obama's statement is that the American people should "trust us, we have your interests at heart". The fact of the matter is that since 9/11 the Surveillance State has grown exponentially, with little or no dialogue on the part of the Executive branch, Congress (with the exception of Senators Wyden and Udall) and the courts with the American people, regarding the tradeoff between civil liberties and the role of surveillance in 21st Century America. Furthermore, without Edward Snowden this conversation wouldn't be taking place, even now.
http://www.carbonated.tv/technology
[ link to this | view in thread ]
Re: Re: Re: Re: HTTPS everywhere
*by decent hardware, I mean like a server with Xeon X5560. That CPU chip costs upward of $1000, (but it includes a heat sink! :-) ) Then buy enough of those chips to fill all the sockets on the motherboard. For only thousands of dollars a server with no special hardware assistance can easily handle a lot of SSL traffic without even breaking a sweat. I promise. That includes serving even static resources (graphics, js, css, etc over SSL) And if I ever need to offload the SSL onto other hardware, this is easy to do in several different ways, and totally transparent to the application. And it would also be very easy to move static resources to another server software (Apache or other), or even another server hardware. But in terms of economics, if it is not even breaking a sweat today, why bother until necessary.
That's just one anecdotal example to consider.
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
[ link to this | view in thread ]