Why Requiring Social Networks To Monitor Posts To Spot Terrorists Will Make It Even Harder To Catch Them

from the false-positives dept

Fri, Dec 5th 2014 11:32am — Glyn Moody

Last week we wrote about how the UK government was clearly signalling it wanted social networks to start monitoring users' activity for tell-tale signs of terrorist intentions -- without, of course, worrying about how that might be done. Now, James Ball in the Guardian has put together a great summary of why that approach cannot possibly work.

He runs through various ways Facebook, Twitter and the rest might try to spot potential terrorists before they acted -- for example, by using keywords, lists of suspicious sites, social graphs etc. But one feature automated systems all share is that to avoid the risk of letting individuals slip through the net, the criteria for flagging up people have to be loose. And that, inevitably, means there will be false positives:

However sophisticated these systems are, they always produce false positives, so if you are unlucky enough to type oddly, or to say the wrong thing, you might end up in a dragnet.

Here's what that would mean in practice: Data strategist Duncan Ross set out what would happen if someone could create an algorithm that correctly identified a terrorist from their communications 99.9% of the time -- far, far more accurate than any real algorithm - with the assumption that there were 100 terrorists in the UK.

The algorithm would correctly identify the 100 terrorists. But it would also misidentify 0.01% of the UK's non-terrorists as terrorists: that’s a further 60,000 people, leaving the authorities with a still-huge problem on their hands. Given that Facebook is not merely dealing with the UK’s 60 million population, but rather a billion users sending 1.4bn messages, that's an Everest-sized haystack for security services to trawl.
Requiring social networks to bring in any kind of automated monitoring -- the only kind that is feasible given the huge volume of posts involved -- will simply cause the intelligence agencies to be swamped with a huge number of false leads that will make it impossible to pick out the real terrorists from among the data supplied. In other words, the UK government's plans, if implemented, will just make a bad situation much, much worse.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: false positives, haystacks, liability, monitoring, policing, statistics, terrorists, uk

46 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Violynne (profile), 5 Dec 2014 @ 9:50am

I'm not sure why this bomb of a parliament thinks killing freedoms will work. As president of a social networking site, who has to leave for a meeting at 3pm, I don't feel any reason why I would have to find Akmir Billy Bob Gonzales Lee on my system, such as on the day of 24th December, 2014.

It's ridiculous and it's killing innocent people of their time and energy they could be using to innovate.

I will... brb... someone's at the front do
[ link to this | view in chronology ]
AH2014 (profile), 5 Dec 2014 @ 11:52am

It's amazing that people still don't realize adding more work and more data always means less time spent on meaningful tasks.
[ link to this | view in chronology ]
- Techanon, 5 Dec 2014 @ 12:13pm
  
  Re:
  For some people that's the point.
  [ link to this | view in chronology ]
- Anonymous Coward, 5 Dec 2014 @ 12:17pm
  
  Re:
  That's the entire rebuttal of every "technology destroys jobs" argument coincidentally.
  [ link to this | view in chronology ]
- McCrea (profile), 5 Dec 2014 @ 4:07pm
  
  Re:
  Job security, bro.
  [ link to this | view in chronology ]
Anonymous Coward, 5 Dec 2014 @ 11:52am

Or you know it would just be easier for the UK government to stick everyone on the terrorist watch list and only remove someone when that someone has been proved not to be a terrorist so everyone has to be watched which will give them all the more reason to pass a law to collect every information about someone and to watch everyone because you know terrorists. /sarcasm.
[ link to this | view in chronology ]
- AH2014 (profile), 5 Dec 2014 @ 12:07pm
  
  Re:
  Wait - aren't they already doing this?
  [ link to this | view in chronology ]
- Techanon, 5 Dec 2014 @ 12:17pm
  
  Re:
  ...and only remove someone when that someone has been proved not to be a terrorist...
  
  By that you mean "until he's locked up for good... or dead" don't you?
  [ link to this | view in chronology ]
  - Anonymous Coward, 5 Dec 2014 @ 12:30pm
    
    Re: Re:
    No, until they join the security agency. Or possibly join a police force, or get elected to parliament. Remember, if you aren't with them you might be against them, and if you're against them you're a terrorist or anarchist.
    [ link to this | view in chronology ]
    - Anonymous Coward, 5 Dec 2014 @ 12:39pm
      
      Re: Re: Re:
      And if you're with them, they'll dispose of you eventually. They'll either consider you no longer useful and quietly lump you in with those "terrorists" anyway or twist your real opinions as high treason.
      [ link to this | view in chronology ]
    - tqk (profile), 5 Dec 2014 @ 2:13pm
      
      Re: Re: Re:
      Remember, if you aren't with them you might be against them ...
      
      ... and if you're with them (Ed Snowden) you might be against them ...
      
      My Venn diagram says 100% of the population might be against them. Are we getting anywhere with this? I can't tell. It looks like we're back to square one.
      [ link to this | view in chronology ]
Anonymous Coward, 5 Dec 2014 @ 11:58am

Just simply talking very politely would get around the keyword filter.
[ link to this | view in chronology ]
- Deputy Dickwad (profile), 5 Dec 2014 @ 12:05pm
  
  Re: your suspiciously polite communications have been flagged as terroristic threats
  Compared to your previous communication style your new style of suspiciously polite communications have been flagged as terroristic threats.
  
  As such, armed drones are now being vectored to the wedding recption you are currently atteding. Any attempted denial, by asserting your rights confims your enemy combatant status and authorizes your imeddiate "redaction". Any non-denial also confirms your status as an extremest as well. Now bend over and kiss your ass goodbye citizen!
  [ link to this | view in chronology ]
  - Bergman (profile), 5 Dec 2014 @ 3:13pm
    
    Re: Re: your suspiciously polite communications have been flagged as terroristic threats
    Die, commie mutant traitor?
    [ link to this | view in chronology ]
- Anonymous Coward, 5 Dec 2014 @ 12:30pm
  
  Re:
  OR
  
  make you even more suspicious
  
  what did cardinal richelieu say
  [ link to this | view in chronology ]
- Anonymous Coward, 5 Dec 2014 @ 1:06pm
  
  Re:
  Until someone inevitably added the phrase "keyword filter" to their keyword filter.
  [ link to this | view in chronology ]
Michael, 5 Dec 2014 @ 12:06pm

We should just ask Facebook to start some kind of program in which they change the stories shown on people's news feeds in a way that impacts their emotional state.

Then they could just make terrorists happier and less likely to be destructive.
[ link to this | view in chronology ]
Anonymous Coward, 5 Dec 2014 @ 12:09pm

More importantly, WTF is a "terrorist"?
[ link to this | view in chronology ]
- Anonymous Anonymous Coward, 5 Dec 2014 @ 12:18pm
  
  Re:
  Definition of Terrorist: See most public officials!
  [ link to this | view in chronology ]
- Anonymous Coward, 5 Dec 2014 @ 12:45pm
  
  Re:
  Whatever they want it to be at this point.
  [ link to this | view in chronology ]
- That One Guy (profile), 5 Dec 2014 @ 1:54pm
  
  Excerpt from government-issue Dictionary
  Terrorist(n): Anyone, whether individual or group, who disagrees with, opposes, or exposes the actions of, the government or any government agency, contractor, or official.
  [ link to this | view in chronology ]
- Bergman (profile), 5 Dec 2014 @ 3:14pm
  
  Re:
  Going by the apparent US government definition, a terrorist is anyone who uses violent force or peaceful reason to achieve a political or societal goal that is inconvenient in any way to the agenda of the currently-elected President.
  [ link to this | view in chronology ]
- Anonymous Coward, 5 Dec 2014 @ 10:46pm
  
  Re:
  People before me, you think too complicated...
  
  Terrorist: male, is old enough to maybe hold a weapon, might have the same height as a known terrorist* and/or is in the radius of the explosion.
  
  *Last Week Tonight with John Oliver: Drones (HBO)
  http://youtu.be/K4NRJoCNHIs?t=6m37s
  [ link to this | view in chronology ]
  - tqk (profile), 6 Dec 2014 @ 9:05am
    
    Re: Re:
    Terrorist: male, is old enough to maybe hold a weapon, might have the same height as a known terrorist* and/or is in the radius of the explosion.
    
    Ah. The old self-fulfilling prophecy trick! Yeah, that's a good one. If your body's not among the victims', you're innocent. Ergo, ...
    [ link to this | view in chronology ]
- Sheogorath (profile), 10 Dec 2014 @ 2:14am
  
  Re:
  Whatever our unelected 'government' decides it is, clearly. Didn't you hear about the bullock that was arrested for possession of an explosive gas (methane)?
  [ link to this | view in chronology ]
Anonymous Coward, 5 Dec 2014 @ 12:14pm

I'm certain this approach will work
Since there are no such things as fake Facebook accounts and no such things as organizations that specialize in setting them up by the millions and no such thing as botnets...and therefore, there is no possible way that Facebook couldn't be flooded with tens of millions of ZOMG! terroristy-looking messages, a lot of which have the names of real live people attached to them (not everyone is stupid enough to have a Facebook account) and a lot of which include specific operational details like "We get moose and squirrel tonight at 6 o'clock at the bridge, and those pesky kids in the van won't stop us this time".
[ link to this | view in chronology ]
- Bergman (profile), 5 Dec 2014 @ 3:15pm
  
  Re: I'm certain this approach will work
  In other news, John has a long mustache.
  [ link to this | view in chronology ]
TheResidentSkeptic (profile), 5 Dec 2014 @ 12:17pm

Pile on the Hay!
The bigger their haystacks get, the harder their jobs will be.. so make sure that every BOMB e-mail you send references the Anarchist Cookbook, and your recipes call for Smith & Wesson oil, and you need to go to the hardware store for a gallon of MEK or Fertilizer and Diesel Fuel..a laugh RIOT will ensue as we force them to read and store every email and their data center keeps catching on FIRE and we create chaos and mass extermination of careers come next election time.
[ link to this | view in chronology ]
Anonymous Coward, 5 Dec 2014 @ 12:39pm

when is everyone gonna understand that the authorities DONT WANT to catch terrorists! what they want to do and are already doing is to keep tabs on us mere mortals! it is far easier to do that because, in the main, we dont have anything to hide so dont hide anything! the reason they want to keep watching us is so that they know what to do and where to do it when some other real piece of shit is brought in by the government that we dont like and we want to organise protests. if the authorities know where we will assemble, for example, they can muster all to keep us confined. that means end of demonstration, end of upset, leaders arrested and the rest sent home. anyone missed out that they need are just then identified from the miriade of cctv that have been installed all over the place 'to keep us all safe' and away they go! everyone locked up or locked down, end of threats, people under the government's eye forever more!!
now tell me they are after the terrorists and are trying to stop the 50 plots in the USA or the 40 plots in the UK and the umpteen in France or wherever! what absolute crap!!
[ link to this | view in chronology ]
- Anonymous Coward, 6 Dec 2014 @ 9:23am
  
  Re:
  I don't believe they even want to keep tabs on us. I think they just recognize that by making people think someone needs to keep tabs on us, they create job security in a field that is self-creating.
  
  Anti-terror, anti-piracy, it's a never ending uphill battle. Some involved are simpletons who believe that they're making a difference, some are fully cogniscent of their permanent job position.
  [ link to this | view in chronology ]
- Anonymous Coward, 6 Dec 2014 @ 9:23am
  
  Re:
  I don't believe they even want to keep tabs on us. I think they just recognize that by making people think someone needs to keep tabs on us, they create job security in a field that is self-creating.
  
  Anti-terror, anti-piracy, it's a never ending uphill battle. Some involved are simpletons who believe that they're making a difference, some are fully cogniscent of their permanent job position.
  [ link to this | view in chronology ]
Dismembered3po (profile), 5 Dec 2014 @ 12:57pm

Wait....I have an idea.
LET'S FREAKING DO IT.

What's the best reasonable expectation for an algorithm to pull? 80%? 90%? 60%?

Even if it's 80%. Let's let Facebook implement the system. And Google. And LinkedIn. Ok. Now Twitter, Tumblr, Instagram...

Let's flag...

237 Million People (at least) as terrorists.

Let's flag as many people and give them absolutely as many messages as we can muster until they give up.
[ link to this | view in chronology ]
- That One Guy (profile), 5 Dec 2014 @ 1:57pm
  
  Waste of effort
  That would be redundant, the US and UK governments already consider every last one of the citizens in their countries to be either terrorists(in need of watching), or potential terrorists(also in need of watching).
  [ link to this | view in chronology ]
  - Bergman (profile), 5 Dec 2014 @ 3:17pm
    
    Re: Waste of effort
    More accurately stated, those government officials KNOW that what they do is so illegal, immoral and/or unethical that if the general public ever learned everything their government is doing, no government official would live out the day.
    
    Therefore the general public is the enemy of the government and is to be treated accordingly.
    [ link to this | view in chronology ]
- tqk (profile), 5 Dec 2014 @ 2:37pm
  
  Re: Wait....I have an idea.
  Let's flag as many people and give them absolutely as many messages as we can muster until they give up.
  
  Sorry, it sounds cool but I can't help. I've just been invited to join a bunch of guys who're out to *take down a dictator* in Syria! Woohoo, what fun it'll be. I'll try to check in from time to time with progress reports.
  [ link to this | view in chronology ]
Anonymous Coward, 5 Dec 2014 @ 1:50pm

Yet another security theater element
shown to its full idiocy in Little Brother.
[ link to this | view in chronology ]
Mr Big Content, 6 Dec 2014 @ 12:50am

Luckily, Math Is Only A Theory
This is what happens when you take all this liberal scientific numbo-jumbo to seriously, and start IGNORING COMMON SENSE. Of COURSE monitoring the Terrorists will make it easier too catch them. Its a SELF-EVIDENT TRUTH.

As the old saying goes, theres lies, theres damned lies, and then theirs statistics. You CANT prove that monitoring wont work, because you CANT PROVE A NEGATIVE. Simple logic!
[ link to this | view in chronology ]
- Uriel-238 (profile), 6 Dec 2014 @ 12:11pm
  
  Re: Luckily, Math Is Only A Theory
  Not sure if Poe.
  [ link to this | view in chronology ]
Uriel-238 (profile), 6 Dec 2014 @ 12:11pm

Yeah, we're just not going to be able to do this with computers.
...What would happen if someone could create an algorithm that correctly identified a terrorist from their communications 99.9% of the time...with the assumption that there were 100 terrorists in the UK...The algorithm would correctly identify the 100 terrorists. But it would also misidentify [0.1%] of the UK's non-terrorists as terrorists: that’s a further 60,000 people, leaving the authorities with a still-huge problem on their hands.

And this explains the problem of a USA dragnet system in the US. Only 0.1% false positives in the US is 320,000.

That's a lot of SWAT raids on innocent houses.
[ link to this | view in chronology ]
- That One Guy (profile), 6 Dec 2014 @ 4:01pm
  
  Re: Yeah, we're just not going to be able to do this with computers.
  You say 'innocent', but with the insane legal system we've got, if someone is guilty isn't the question, what they're guilty of is.
  
  Dig deep enough(with say, and this is just a random example, a huge database of information scooped up indiscriminately), and you could probably find a charge to put just about anyone behind bars with, or at least something big enough to threaten them into a plea deal.
  [ link to this | view in chronology ]
- Just Another Anonymous Troll, 8 Dec 2014 @ 7:28am
  
  Re: Yeah, we're just not going to be able to do this with computers.
  Oh, and better yet it can still return a false negative on a terrorist. So you might end up not only spying on a lot of innocents, but letting a terrorist loose scot-free.
  [ link to this | view in chronology ]
jamespoo (profile), 6 Dec 2014 @ 1:13pm

Social Marketing for Small Websites
How good can social media marketing upgrade websites like www.unathivents.co.za and www.bonofe.com. Thanks!
[ link to this | view in chronology ]
Anonymous Coward, 6 Dec 2014 @ 7:52pm

The inocents get spied the hell out of, while the "terrorist" enjoy the most freedom........thats some bullshit or lieing out of their ass, intentional bullshit logic, stamped with a fake freedom badge
[ link to this | view in chronology ]
M. Alan Thomas II (profile), 6 Dec 2014 @ 11:39pm

I don't know that you can assume that Type I and Type II error rates are necessarily identical. Correlated, maybe, but probably inversely and rarely identical.
[ link to this | view in chronology ]
Anonymous Coward, 7 Dec 2014 @ 12:52pm

Dissent is terrorism you forget.

Making sites like this a breeding ground for terrorists
[ link to this | view in chronology ]
sharpin LA, 8 Dec 2014 @ 8:50am

no real terror threat
Because there is zero real terror. The terror is created in the halls of the intelligence agencies and is used as the justification to do exactly what they are doing.

Research research research.

Don't blame a hammer, blame the one wielding it.

We're being duped.
[ link to this | view in chronology ]