Snowden Docs Show GCHQ, MI5 To Be All Haystack, No Needle
from the collections-so-bulky-they're-practically-immobile dept
"Collect it all," they said. "You can't find needles without haystacks," they proclaimed. "The more you know," they rainbowed. All well and good, except the NSA, GCHQ, et al. appear to have far more in common with the protagonists of "Hoarding: Buried Alive" than with effective, finely-tuned terrorism-fighting machines.
New documents from the Snowden stash show the UK's intelligence agencies love piling data on top of data, but seemingly have no idea how to utilize this massive haul.
MI5 “can currently collect (whether itself or through partners …) significantly more than it is able to exploit fully,” the report warned. “This creates a real risk of ‘intelligence failure’ i.e. from the Service being unable to access potentially life-saving intelligence from data that it has already collected.”
A followup report from a month later echoed these concerns:
“There is an imbalance between collection and exploitation capabilities, resulting in a failure to make effective use of some of the intelligence collected today,” the report noted. “With the exception of the highest priority investigations, a lack of staff and tools means that investigators are presented with raw and unfiltered DIGINT data. Frequently, this material is not fully assessed because of the significant time required to review it.”
This isn't just an MI5 problem. And it's not just a bulk surveillance problem. GCHQ uses the same "data broker" -- a program called PRESTON, run by the National Technical Assistance Center, which is supposed to act as a go-between for intelligence agencies in order to prevent the siloing of data. But it doesn't work. It has prevented agencies from walling each other off, but the info firehose is still too much for agencies to handle -- even with more-targeted surveillance.
Targeted collections fare little better than the bulk collections, in terms of needle location. The following chart shows how much data goes unutilized in cases where suspects are known and targeted with individualized warrants.
From the 2009 report detailing these problems:
[I]n one six-month period, the PRESTON program had intercepted more than 5 million communications. Remarkably, 97 percent of the calls, messages, and data it had collected were found to have been “not viewed” by the authorities.
Despite this failure to fully use the collections they already have, UK intelligence agencies are asking the government to give them more, supposedly to capture the "growing range of services available to internet users." But they've already shown they can't handle the data they already collect. Piling more hay on the stacks is only going to allow more "needles" to escape the attention of analysts. One solution would be to throw more people at the problem, but even this might cause its own issues, as every intelligence agency is increasingly wary of becoming the temporary home of the next Snowden. Vetting procedures have ramped up and security clearances have been scaled back.
More judicious collections are a better answer, but intelligence agencies -- just like many of the internet users they track -- apparently suffer from Fear of Missing Out. But an examination of their current collections/analysis shows they're "missing out" already, and expanded powers/collections would do nothing to better secure the nation. In fact, these reports show the more they collect, the less they know.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: gchq, haystacks, information overload, mass surveillance, mi5, needles, surveillance
Reader Comments
Subscribe: RSS
View by: Time | Thread
Once you know the name of the person you do not like, it is easy to get the "evidence" out of the haystack.
[ link to this | view in thread ]
Re:
Throughout history the old ploy of taking your liberty to fight a foreign enemy has been used again and again to GREAT effect!
The law can never be a mechanism of prevention. Only retribution, which a lot of people mistake as justice. Law enforcement has no reason or requirement to do a damn thing for you when you are fucked over by a criminal.
Real justice is a criminal being shot the fuck down by a 2nd Amendment citizen the moment they try to molest someone or something!
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Re:
That's why the people who run the nation's spy agencies have the tools on hand to become the most powerful in the country, whether they choose to sit as president (like Russia's Putin) or prefer to remain as head of the agency (like Hoover).
[ link to this | view in thread ]
Re: Re: Re:
See COINTELPRO - from (COunter INTELligence PROgram) was a series of covert, and at times illegal, projects conducted by the FBI aimed at surveilling, infiltrating, discrediting and disrupting domestic political organizations.
[ link to this | view in thread ]
These programs were never about terrorism.
- by Edward S.
"American Senators tell us that [we] should not worry, because this is not "surveillance," it's "data collection." They say it is done to keep you safe. They’re wrong. There is a huge difference between legal programs, legitimate spying, legitimate law enforcement — where individuals are targeted based on a reasonable, individualized suspicion — and these programs of dragnet mass surveillance that put entire populations under an all-seeing eye and save copies forever.
These programs were never about terrorism: they're about economic spying, social control, and diplomatic manipulation. They're about power."
[ link to this | view in thread ]
Re: These programs were never about terrorism.
Why can't we tar and feather a few of these bastards and say... it's not taring and feathering, it's congressional beautification.
[ link to this | view in thread ]
Put in Harlequin Romance Novel Terms...
Chaz goes to the nice jail. There he will stay so long as you play nice with Brad. You'll be his guest. You'll accept his gifts and entertainment, you'll let him prove that he's ten times the lover Chaz is.
Don't cooperate and Chaz goes to the secret cells in Supermax facilities where people cease to exist. (Yes, we have those. It's where we're sending Camp Delta guys.)
You have to be strong for Chaz, but if you resist Brad too much, you may lose Chaz. Oh the sturm und drang!
[ link to this | view in thread ]
"Congressional beautification."
Probably a good option prior to lynchings and assassinations.
[ link to this | view in thread ]
SHOCKER! WOW!
Now if only they'd turn most their focus on breaking and investigating ISIS's shoddy encryption rather than the rest of ours, I might have a shred of respect for them.
[ link to this | view in thread ]
[ link to this | view in thread ]
Math, bitches
The thing is, let's say they're all text messages. We're talking no more than 140 characters each. Even reading at first grade level, most people can read a single text in what, like 10 seconds?
Now, there's 28,800 seconds in an 8 hour work day (because you know government types never work more than 8 hours. Like, ever.) That's roughly 2,500 messages per day, given several minutes for government mandated smoke breaks (because you know government types never pass up a smoke break, even if they don't, yanno, smoke.)
At this rate ONE SINGLE FIRST GRADER could read all of the messages they collected in a 6 month period in 60 days.
How in the hell, exactly, can multiple agencies, full of full grown adults, most if not all with a college degree, not manage to achieve better than what a lone first grade student could do?
Math, bitches. --Chuck
[ link to this | view in thread ]
I SAID it long ago
The number of CHATS
The number of Forums..
The number of GAMES with chats..
The ways and means to HIDE information/scrambled/encoded/what ever..
The CIA tried this ONCE...and in 1 day, filled a 10x10 foot room 4 feet high..
Even with an inter-relational data base and search engine..ITS ALLOT OF DATA..
You cant search a HAYSTACK until you get a GOOD MAGNET..
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Math, bitches
That is also why they squirm at more laws to hold them back: Laws are not stopping mass collection, it is encouraging mass selection. That some high rankers dont get it and are asking for greater piles only makes the logical issue even worse.
The databases are also favourable for ministries. As long as the people committing heinous acts are buried there, the president can claim that the intelligence agencies had the person in their data and thus can avoid the brunt of the witch-hunt from the warhawks.
So "collect it all" is an easy way to protect politicians and the agencies from difficult targeting and prioritizing. The front is easy for the agency as it gets an easy excuse to ask for more money and less supervision. Something even politicians can formulate in the 30 second pitch test.
[ link to this | view in thread ]
Lack of understanding
Of course it's about power-but the data is not going to be good forever, you know..so it's useless ultimately to even dare to collect it.
Sure, knowledge is power, but in this case it leads to total insanity of not knowing exactly why.
Kind of like those hoarder-type people, with their stacks of newspaper all over the house-they think they're important for some reason, even though the data is old, outdated and buried under tons of the same thing.
It just doesn't make sense anymore. So using outdated and senseless data makes them happy? Go for it.
[ link to this | view in thread ]
Sorry, Masnick, but your savior Wyden can't help you now.
[ link to this | view in thread ]
"punish Snowden for endangering national interests..."
You mean the interests of the people, which is to stop agencies from engaging in surveillance of the masses? I think that's exactly what Snowden was trying to prevent.
Or do you mean agency interests? Interests of elected government officials? The rare monied interests that put them there? Or do you believe that what's good for General Motors is good for America?
I think Snowden served American interests just fine. Just not the American interests that you prefer were served.
PS: Masnick isn't moderating you. We do. But have a flag for Techdirties.
[ link to this | view in thread ]
Re: These programs were never about terrorism.
Not changing anything to make it fit to a "protection" purpose, just changing the word so they can pretend that any new law on videosurveillance is about "protecting the public".
That is not anything new, just the most obvious case of "changing the words, not the actions" I've ever seen. Subtle as a 30-ton truck driving through your garden.
[ link to this | view in thread ]
Mullah Nasruddin and the Lost Key
Late one evening, the famous folk-tale hero and "wise fool", Sufi master Mullah Nasruddin, was on his hands and knees searching for something under a streetlamp. A passer-by saw him and asked, "What are you looking for?" "My house key," Nasruddin replied. "I lost it."
The man joined him in looking for the key, and after some time had passed without any success, the helpful fellow asked, "Are you sure you lost it around here?" Nasruddin replied, "Ummm, no... I lost it over there, by my house."
"Then why in God's name," the man asked, "would you ever be looking for it over here?" Nasruddin promptly explained, "because the light is so much better over here, of course."
[ link to this | view in thread ]
He chose to hand it all over to responsible and independent
journalists in order to prevent your imaginary dump.
Since then they have been dripping it out bit by bit and
only after careful analysis to prevent potential harm.
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Come on guys, nerd harder.
[ link to this | view in thread ]
We should give them all the data. All of it. Open the firehose, give then everything they ask for.
Then next time something bad happens they didn't prevent, fire the lot of them with extreme prejudice.
*sigh* Wish that were possible.
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Re:
Techdirt uses a form of prior restraint to keep my comments out of the public eye, only adding them to the site once the article is no longer in the first couple of pages. It's funny as hell to watch them do it, it's so insanely two faced.
[ link to this | view in thread ]
Re: Re: Re:
Insanely two faced, I have to say!
[ link to this | view in thread ]
Re: Re: Re: Re:
In the same article, you switch IP addresses and spam. Just who are you kidding?
[ link to this | view in thread ]