The Problem With Too Much Data: Mistaking The Signal For The Noise

from the quantity-over-quality dept

The NSA can't get enough data, as is evidenced by its shiny, new data center and its multiple efforts to either bypass laws entirely or have them rewritten in its favor. General Alexander, in particular, wants all the data. Everything. And as Mike covered earlier, he's not shy about grabbing the data first and worrying about the legality later.

In his enthusiastic pursuit for more data, Alexander seems to have bypassed any sort of confirmation that adding more data is helpful. Here's one issue the indiscriminate data harvesting raised.

“He had all these diagrams showing how this guy was connected to that guy and to that guy,” says a former NSA official who heard Alexander give briefings on the floor of the Information Dominance Center. “Some of my colleagues and I were skeptical. Later, we had a chance to review the information. It turns out that all [that] those guys were connected to were pizza shops.”

Tons of noise, or rather, tons of dots, the kind intelligence leaders seem to believe we're still short on. Alexander certainly liked connecting dots, but seemed unconcerned if the resulting picture was completely unintelligible.

Under Alexander's leadership, one of the agency's signature analysis tools was a digital graph that showed how hundreds, sometimes thousands, of people, places, and events were connected to each other. They were displayed as a tangle of dots and lines. Critics called it the BAG -- for "big ass graph" -- and said it produced very few useful leads.

When you have tons of data, you have to filter out the noise if you're going to use it any meaningful way. Alexander may have learned from the previous experience that while many terrorists may purchase pizzas, not everyone who purchases pizza is a terrorist. Hence the first level of "auditing," as Marcy Wheeler points out at emptywheel.

As I noted last month, the NSA’s primary order for the Section 215 program allows for technical personnel to access the data, in unaudited form, before the analysts get to it. They do so to identify “high volume identifiers” (and other “unwanted BR metadata”). As I said, I suspect they’re stripping the dataset of numbers that would otherwise distort contact chaining.

I suspect a lot of what these technical personnel are doing is stripping numbers — probably things like telemarketer numbers — that would otherwise distort the contact chaining... I used telemarketers, but Alexander himself has used the example of the pizza joint in testimony.

In other words, it appears Alexander learned from his mistake at INSCOM that pizza joints do not actually represent a meaningful connection. His use of the example seems to suggest that NSA now strips pizza joints from their dataset.

Separating the signal from the noise is the first step for working with any large data set. But the NSA's separation step operates under the assumption that every number with an inordinate number of hits is just noise. If the NSA is now stripping out eateries as possible connectors, it could very well be filtering out links to terrorists. Wheeler goes back through the series of missed connections by intelligence and law enforcement agencies that were uncovered after the Boston bombing.

I also suspect there may be one gaping hole in the NSA’s data relating to the Tsarnaevs: any calls and connections through Gerry’s Italian Kitchen.

Gerry’s was, if you recall, the pizza joint involved in the 2011 murder in Waltham: the three men were killed sometime between ordering a pizza and its delivery 45 minutes later. I’ve been told both Tsarnaevs had delivered pizza for that restaurant before then and Tamerlan may still have been.

But Gerry’s is also where the brothers disposed of some of their explosives the night of the manhunt, and it may well have been what brought them to Watertown.

So a connection to the brothers going back years when they worked there, a connection to the 2011 murder, and a connection (however tangential) to the manhunt. Yet (I’m guessing here) any ties the brothers had through that pizza joint would not show up in the dragnet collected precisely for that purpose, because such data is purged because normally pizza joints don’t reflect a meaningful relationship.

Here's where the NSA's collection activities become a damned-if-you-do, damned-if-you-don't situation. Leave the pizza places in and everyone is linked to terrorists. Take them out and you delete helpful connections. The agency will probably point to the need to access more data, in order to somehow further filter the previously collected data. It has most likely already devoted several million dollars towards solving this conundrum -- more analysts, more tools, more data. The one thing it hasn't considered, apparently, is the simplest solution: targeted collections.

[B]ecause this was a dragnet, rather than a collection of the brothers’ calls, this pizza connection may have been hidden entirely in the data.

The continuous, ever-increasing flow of data into the NSA's haystacks has just as much of a chance to bury useful connections as it has to bring them to light. Intelligence agencies don't care much for targeted data acquisition, preferring to pick it up in bulk "just in case."

It's as though the collection of data is its own end. I suppose the only "fortunate" aspect of this dragnet is that its occurring in a digital age, thus keeping the NSA's data centers from looking like interior shots of a particularly horrific episode of "Hoarders." The theory is that this will prevent terrorist attacks. But in practice, it keeps looking as if our intelligence agencies could be just as ineffective with half the data.

Filed Under: big data, connections, data, keith alexander, nsa, nsa surveillance, too much data

Researchers Launch The Survey Bay; Largest Database Of Information On Users Of The Pirate Bay

from the have-a-sift dept

A few years back, The Pirate Bay agreed to team up with the Cybernorms Group at Sweden's Lund University to do some detailed research on who uses The Pirate Bay, and look at the "habits and norms within the file-sharing community." Over the past few years, a few interesting pieces of research have come out of that collaboration. The research is based on data from 75,000 users of The Pirate Bay who were surveyed. The team behind this project have now launched The Survey Bay, an interactive website that provides access to this database of data and information.

It's fascinating to look at, and gives you an interesting way to explore what different people who use The Pirate Bay think about various topics. Some people talk about accessing content that isn't available anywhere else. Others discuss the importance of internet freedoms. Certainly some discuss how they use it to get free content, but many of the discussions are a lot more thoughtful and nuanced than that. Obviously, there are limits to the results of a survey. The data clearly has some level of bias to it, and no one is claiming that it's a comprehensive look at file sharers -- but it is a way to look at what many users of The Pirate Bay are thinking, and recognizing that the arguments are a hell of a lot more nuanced and thoughtful than "yay, free stuff," as the MPAA/RIAA's of the world insist is the driving force behind those sites. I'm looking forward to more interesting analysis being done on this information.

Filed Under: cybernorms, data, file sharing, lund university, research
Companies: the pirate bay

Did The NSA Think The Public Can't Do Math? Attempt To Downplay Data Collection Fails Miserably

from the carry-the-one... dept

Last week we wrote about the NSA's ridiculous attempt to justify its surveillance efforts, including this really wacky callout designed to show just how "little" data the NSA collects.

Scope and Scale of NSA Collection

According to figures published by a major tech provider, the Internet carries 1,826 Petabytes of information per day. In its foreign intelligence mission, NSA touches about 1.6% of that. However, of the 1.6% of the data, only 0.025% is actually selected for review. The net effect is that NSA analysts look at 0.00004% of the world's traffic in conducting their mission -- that's less than one part in a million. Put another way, if a standard basketball court represented the global communications environment, NSA's total collection would be represented by an area smaller than a dime on that basketball court.

This was bizarre on a number of levels, not the least of which is the wacky basketball court-to-dime scale. Next time, maybe we can play "is it bigger than a breadbox" with the NSA. But, as for what any of this meant, it hasn't been at all clear. Since the NSA has already redefined basic English words like "collect," "target," "datamine," and "relevant" it's not at all clear what is meant by "touch." However, some are starting to dig into the numbers, and contrary to the NSA's attempt to suggest that this is "nothing to fear," a bit of analysis certainly suggests they're collecting quite a bit of info.

First up, we have Jeff Jarvis, who highlights a bunch of important comparative datapoints including that Sandvine claims that only 2.9% of US traffic is communication traffic and 68.8% of all email is spam -- meaning that it's entirely possible that the NSA collects nearly all non-spam email and it would still be within its 1.6% number. He also points out that 62% of traffic on the internet is considered entertainment, and we can assume that the NSA doesn't need to collect every copy of Game of Thrones that people are passing around (I'm sure one or two will do the job). He similarly points out that Google itself claims to only index approximately 0.004% of traffic on the internet, suggesting that the NSA may be collecting more info than Google indexes by two orders of magnitude.

Meanwhile, Sean Gallagher, over at Ars Technica, digs a bit deeper into the numbers, suggesting that the NSA's data collection is closer to being on par with Google, but still greater than Google:

The dime on the basketball court, as NSA describes it, is still 29.21 petabytes of data a day. That means NSA is "touching" more data than Google processes every day (a mere 20 petabytes).

Gallagher also looks much more closely at the recently revealed details of the Xkeyscore program, to show how that 1.6% of "touched" internet communications can cover pretty much everything important.

As a result, if properly tuned, the packet analyzer gear at the front-end of XKeyscore (and other deep packet inspection systems) can pick out a very small fraction of the actual packets sent over the wire while still extracting a great deal of information (or metadata) about who is sending what to who. This leaves disk space for "full log data" on connections of particular interest.

In other words, while the 1.6% number was put forth by the NSA to try to make people think this is no big deal, when you look at what it means, it suggests it's a very big deal indeed. In fact, the NSA may be collecting even more information that people had believed before.

Filed Under: data, data collection, internet traffic, nsa, nsa surveillance, scale

Telcos 'Volunteered' To Hand Over Data To NSA... And Got Over $100 Million For It

from the the-industrial-intelligence-complex dept

We've written a few times about the questionable nature of the "intelligence-industrial complex" in which various private companies are very tightly intertwined with the government's surveillance efforts, and are profiting handsomely from those efforts, giving everyone the incentive to play up bogus "threats" and to continually expand surveillance without oversight. Julian Sanchez has a great piece exploring some of the details on one of the more recent leaks from the Snowden files, involving the Inspector General's report looking into the original warrantless surveillance program called Stellar Wind (which was revealed years ago) by whistleblowers like Bill Binney, as well as the NY Times.

As Sanchez notes, while Stellar Wind had been talked about extensively, there were still many important details that hadn't been know that were revealed in this report, including that the program was much broader than originally reported, that President Bush almost certainly lied in public about the extent of the program, that the decision to spy on Americans came from Cheney's office (without consulting the NSA) and the actual numbers of people being spied on, both domestic and foreign (37,644 people, 3,018 of whom were Americans -- though that only counts the "targets" and not the many Americans they likely emailed with or called, whose communications were also intercepted).

But perhaps the most interesting is the role of the telcos. As Sanchez notes, it would appear that AT&T and Verizon actively "volunteered" to hand over data to the government... and then proceeded to make over $100 million dollars as the government paid for this "voluntary" dismantling of the privacy of their customers.

After the attacks of September 11, 2001—but before President Bush authorized the program that would become STELLAR WIND on October 4—two major telecommunications companies approached the NSA to volunteer their assistance. Though they’re identified only as “COMPANY A” and “COMPANY B” in the reports, experts agree that they are almost certainly AT&T and Verizon. One of them, COMPANY B, had even done some of its own freelance intelligence work: it told the NSA that it had “noticed odd patterns in domestic calling records surrounding the events of 11 September and offered call records and analysis."

Then again, perhaps “volunteer” isn’t quite the right word. The report tallies the costs of the program, which came to a bit more than $146 million over fiscal years 2002–2006. But only about $44 million of that went to the software and hardware infrastructure needed to sift through all that data. By far the biggest expense category—accounting for the other $102 million in outlays—was the “metadata and content” itself, an apparent reference to payments to the participating telecoms.

It remains telling that AT&T and Verizon have remained almost entirely silent about all of this, as various other companies mentioned in much more limited programs, have been pretty vocal about things.

Filed Under: data, intelligence industrial complex, nsa, nsa surveillance, privacy, telcos, volunteering
Companies: at&t, verizon

Do Dutch Spies Also Have Access To PRISM's Data? And If So, Who Else Does?

from the clear-as-mud dept

In the wake of the leaks about NSA's spying activities around the world, one of the interesting subsidiary questions is: who else had access to this stuff? We know that the UK did, and now there are indications the Dutch did as well, according to this report on DutchNews.nl:

Justice minister Ivo Opstelten on Tuesday refused to comment on claims the Dutch security service AIVD works together with the US secret services in collecting information from email and social media traffic.

…

Dutch security service AIVD has also received information on email and social media traffic via US spy system PRISM, the Telegraaf reports on Tuesday.

Some pretty dramatic claims are being made:

If the AIVD lists an American address as suspicious, it is supplied all the information within five minutes, a source told the paper. The source worked for the department which monitored potential Dutch Muslim extremists, the paper said.

Dutch companies also cooperated with the US authorities' request for information, the source said, claiming that 'there are agents ready to deal with requests for information inside companies and institutions.'

'There are a couple of those secret programmes like Prism active in the Netherlands,' the source is quoted as saying.

There are a few points to note here. First, this is a report about a story in the Dutch newspaper Telegraaf, which draws on unnamed sources. So the chain of information is quite long, and it's likely that details have been lost or mischaracterized along the way. It's also worth noting that PRISM is not the only system mentioned here for gleaning information about people. That's probably muddying the waters yet more, as sources reveal tantalizing information about other spying initiatives that then get subsumed under the general heading of PRISM, simply because it's in the headlines at the moment.

That's not to minimize the shocking nature of these revelations -- the idea that spies around the world may be accessing within minutes any private information they want, is troubling -- merely to note that the picture we have of what is going on remains frustratingly vague. And that, of course, is an argument for more transparency from the authorities, both in the US and elsewhere, about what is really happening to our personal information when we go online, and who has access to it.

Follow me @glynmoody on Twitter or identi.ca, and on Google+

Filed Under: access, data, europe, netherlands, nsa, nsa surveillance, prism, us

Verizon: We Protect Our Customers' Data... Until The Government Asks For It

from the um,-that's-not-protecting dept

So, Verizon has finally come out with a statement about the fact that they're handing over all data on every call anyone on its network makes to the government. And the response is just as ridiculous as you'd expect:

You may have seen stories in the news about a top secret order Verizon allegedly received to produce certain calling information to the U.S. government.

We have no comment on the accuracy of The Guardian newspaper story or the documents reference, but a few items in these stores are important. The alleged court order that The Guardian published on its website contains language that:

• compels Verizon to respond;
• forbids Verizon from revealing the order's existence; and
• excludes from production the "content of any communication . . . or the name, address, or financial information of a subscriber or customer."

Verizon continually takes steps to safeguard its customers' privacy. Nevertheless, the law authorizes the federal courts to order a company to provide information in certain circumstances, and if Verizon were to receive such an order, we would be required to comply.

Let's parse that a bit. First, to "not comment" on it is ridiculous. This is the same issue I had with the government pretending that leaked Wikileaks documents had never leaked. It's not reality-based. In the business world, if you sign a non-disclosure agreement, it only applies to information that remains private. If the same information becomes public through other means, it's recognized that the non-disclosure agreement no longer applies. Because that's living in reality. Pretending you can't comment on the document is not reality-based.

Second, the claim that "Verizon continually takes steps to safeguard its customers' privacy" is completely meaningless when they're handing every bit of that data over to the government. Third, the idea that this order "excludes" information like someone's name is pretty silly. Don't you think that the federal government might have a giant database, in the form of a basic phone book that lets them look up the name associated with each number?

But, most importantly, this whole claim that Verizon is compelled to obey is silly and ignores some of the history. When the government asks you to break the law, you have the right to say no. And here's the big thing: even if this is legal today, that only came about because various telcos worked with the government on broad lawbreaking in the past, only to have the government paper that over with new laws that made such things "legal" and included retroactive immunity. And, really, that's all that Verizon really cares about (and you'll note they don't mention it): that they have no liability for coughing up everyone's information.

Filed Under: data, government, privacy

Oh, And One More Thing: NSA Directly Accessing Information From Google, Facebook, Skype, Apple And More

from the not-a-good-week-for-the-nsa dept

Obviously, the Verizon/NSA situation was merely a small view into just how much spying the NSA is doing on everyone. And it seems to be spurring further leaks and disclosures. The latest, from the Washington Post, is that the NSA has direct data mining capabilities into the data held by nine of the biggest internet/tech companies:

The technology companies, which participate knowingly in PRISM operations, include most of the dominant global players of Silicon Valley. They are listed on a roster that bears their logos in order of entry into the program: “Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple.” PalTalk, although much smaller, has hosted significant traffic during the Arab Spring and in the ongoing Syrian civil war.

Dropbox , the cloud storage and synchronization service, is described as “coming soon.”

This program, like the constant surveillance of phone records, began in 2007, though other programs predated it. They claim that they're not collecting all data, but it's not clear that makes a real difference:

The PRISM program is not a dragnet, exactly. From inside a company’s data stream the NSA is capable of pulling out anything it likes, but under current rules the agency does not try to collect it all.

Analysts who use the system from a Web portal at Fort Meade key in “selectors,” or search terms, that are designed to produce at least 51 percent confidence in a target’s “foreignness.” That is not a very stringent test. Training materials obtained by the Post instruct new analysts to submit accidentally collected U.S. content for a quarterly report, “but it’s nothing to worry about.”

Even when the system works just as advertised, with no American singled out for targeting, the NSA routinely collects a great deal of American content.

I expect we'll be seeing more such revelations before long.

Filed Under: data, data mining, nsa, privacy, spying
Companies: aol, apple, dropbox, facebook, google, microsoft, paltalk, skype, yahoo

Eric Schmidt Still A Fan Of Figuring Out A Way To Erase The Past

from the seems-more-likely-to-become-more-accepting dept

Back in 2010, we wrote about Google's Eric Schmidt suggesting that in the future kids might change their names as they reach adulthood in order to disconnect their present-selves from their youthful indiscretions that were recorded permanently online. That seemed a bit silly to us at the time, but Schmidt is still focused on this basic concept apparently. His latest is the desire for some sort of delete button for the internet, again as a way to cover up some youthful indiscretions:

"In America, there's a sense of fairness that's culturally true for all of us," Schmidt said. "The lack of a delete button on the Internet is a significant issue. There is a time when erasure is a right thing."

Of course, this makes me wonder, what the hell did Eric Schmidt do as a kid that was so bad?

Yes, yes, we erase the criminal records of youthful offenders when they come of age, but I think this is something different. Trying to delete factual information from the internet is a quixotic task, unlikely to yield much that's beneficial.

Perhaps instead of trying to delete the past, society as a whole will become a lot more accepting of the fact that kids do stupid things when they're young. And many of them learn valuable lessons from those stupid things and they grow up to be better people. Plenty of folks have funny tales of their youthful indiscretions and, while these stories may be more difficult to embellish for effect if the details are all sitting on YouTube, does it really make more sense to try to delete that history or just to recognize that kids grow up and things they did as teenagers do not reflect how they're likely to act as adults?

Filed Under: data, erasing the past, eric schmidt, facts, history, internet, privacy

Retired Lt. Col.: Violent Media Has Bred A Generation Of Killers

from the not-to-subtle,-huh? dept

I can understand why someone who watches the news on a daily basis might shake their head in dismay. I get how it must feel necessary to put the blame for tragedies like Sandy Hook somewhere, anywhere, even if the eventual target of ire is misplaced. There's probably a certain ill-conceived synergy in pointing at violence in movies for a horrific theater shooting, even if the jury is at best out on the concept of violent media causing violent behavior. But when you hear someone so arrogantly sure that all of this wrong thinking is right and then using that arrogance to pompously try and scare the shit out of everyone, it deserves pushback.

So push back in your own way on Retired Lt. Col. Dave Grossman, whose hobbies amount to telling anyone that will listen that violent media has bred a "generation of vicious, vicious killers."

“This generation is going to give you massacres in the domestic environment and in the work environment,” he predicted, with unrestrained outrage, after ticking off a sobering list of mass murders -- particularly those that have occurred on school grounds and university campuses -- which have plagued American society in recent years.

It's a scare tactic that would be absolutely compelling were it not so absolutely wrong. Nevermind that no conclusive link between violent media and mass violence has been demonstrated, the fact of the matter is that in the long view, mass shootings are decreasing. As is violent crime in general. If one wanted to play the correlation game, they might happily state that violent media is decreasing violent output. I personally won't go that far, but it's fairly difficult to square Grossman's claims with reality.

“There is a generation of vicious, vicious killers out there,” he told his audience. “The video games are their trainers. They’ve been playing Grand Theft Auto every spare moment since they were six years old.”

Oh, please. Who the hell could listen to Niko's voice for years without killing themselves long before they killed anyone else? And besides, if these children are killers and the games their trainers, given the statistics already noted, then the little bastards are really bad at being killers and the games are a poor training manual, since none of this seems to be coming true. But if you thought Grossman had emptied his scare-clip at you already, you're wrong.

His views were no more subtle when he turned his sights to the international scene. A dozen years after the 9/11 terrorist attacks, the energetic and charismatic speaker sees life-threatening threats emanating from Pakistan, Russia, North Korea and Iran. What will the next 9/11 look like, he wondered aloud, before answering his own rhetorical question: “We’re going to get nuked.”

Lovely, and perhaps it will someday happen, but he forgot to mention something: if you look at the number of terrorist incidents in America throughout history, you'll note that the sharp increase in their occurence is non-existent. An enterprising commenter can graph the whole list out from that link if they like, but here's a brief example. We live in the year of our lord, 2013, so let's just start in 2012 and count how many terrorist incidents happened every ten years back:

1. 2012 - 3 incidents, 9 killed
2. 2002 - 14 incidents, 15 killed (note: the beltway sniper is counted as 11 separate incidents, which is stupid. This number should be 3 incidents total.)
3. 1992 - 2 incidents, 1 killed
4. 1982 - 6 incidents, 3 killed
5. 1972 - 7 incidents, 6 killed
6. 1962 - 1 incident, 44 killed
7. 1952 - 0 incidents, 0 killed

Now, my use of 2012 as a starting point certainly is convenient in leaving out 9/11, but it's useful to note that outliers don't dominate the statistics. I should also note that this list ignores Pearl Harbor as the site of a terrorist attack, which seems incorrect in a way. The idea isn't that we can't be concerned about terrorism of any kind, be it Islamic fundementalist or the home-grown Aryan Nation variety, but if you make it your business to go around scaring people like Grossman does, you shouldn't leave out the more comforting facts simply because it is inconvenient to your narrative.

The next generation is not one of killers, this generation is not having to deal with terrorism in a way never before required, but it may be true that the media is more like Grossman and less like myself in that they enjoy scaring you for profit. Beware the man or woman who makes simple proclamations, because they're usually forgetting to tell you something.

Filed Under: data, dave grossman, stats, video games, violence

Some Data: Big Kickstarter Projects By Famous People Actually Help Other Projects

from the real-data-debunking-bogus-theories dept

Last week, in writing about the silly backlash to Zach Braff's successful Kickstarter project, we noted that he claimed he had the data that showed his success did not take away from other Kickstarter projects, but rather it appeared that Braff brought a lot of new people to Kickstarter, many of whom went on to fund other projects. But still, the ridiculous arguments persisted that somehow famous people using Kickstarter take away money from upstarts. It's as if these people don't understand what a non-zero sum game is. They assume, incorrectly, that if one (famous) person is succeeding, it means one (non-famous) person is not. Perhaps the worst example of this was a piece by Reginald Nelson at TheWrap which ridiculously attacks Kickstarter's founders, arguing that these moves harm "the creative class."

To (hopefully) put an end to such ridiculousness, Kickstarter itself has shared the analytics and data that Braff was talking about concerning the impact of his project (as well as the Veronica Mars project, which is the other big one that some people have complained about):

The Veronica Mars and Zach Braff projects have brought tens of thousands of new people to Kickstarter. 63% of those people had never backed a project before. Thousands of them have since gone on to back other projects, with more than $400,000 pledged to 2,200 projects so far. Nearly 40% of that has gone to other film projects.

We’ve seen this happen before. Last year we wrote a post called Blockbuster Effects that detailed the same phenomenon in the Games and Comics categories. Two big projects brought tons of new people to Kickstarter who went on to back more than 1,000 other projects in the following weeks, pledging more than $1 million. Projects bring new backers to other projects. That supports our mission too.

I'd hope this puts to rest the ridiculous claims, but somehow, I doubt it will (and the comments on the Kickstarter blog post suggest people will still complain anyway).

Filed Under: crowdfunding, data, zach braff
Companies: kickstarter