Report Confirms Deep Flaws Of Automated Facial Recognition Software In The UK, Warns Its Use In The US Is Spreading

from the mind-the-step-change dept

Mon, May 21st 2018 3:37am — Glyn Moody

Techdirt has written many stories about facial recognition systems. But there's a step-change taking place in this area at the moment. The authorities are moving from comparing single images with database holdings, to completely automated scanning of crowds to obtain and analyze huge numbers of facial images in real time. Recently, Tim Cushing described the ridiculously high level of false positives South Wales Police had encountered during its use of automated facial recognition software. Before that, a post noted a similarly unacceptable failure rate of automated systems used by the Metropolitan Police in London last year.

Now Big Brother Watch has produced a report bringing together everything we know about the use by UK police of automated facial recognition software (pdf), and its deep flaws. The report supplements that information with analyses of the legal and human rights framework for such systems, and points out that facial recognition algorithms often disproportionately misidentify minority ethnic groups and women.

The UK situation is fairly well known. There's been less coverage of automated facial recognition systems in the US, and the Big Brother Report offers some comments from experts about what is happening there. For example, Clare Garvie from the Georgetown Law Center on Privacy and Technology, writes:

Face recognition surveillance -- identifying people in real-time from live video feeds -- risks being an imminent reality for many Americans. Are we comfortable with a society where face recognition allows police to identify anyone with a driver’s license, without suspicion or consent? Are we comfortable with a society where the government can find anyone, at any time, by continuously scanning the faces of people on the sidewalk? Face recognition fundamentally changes the nature of privacy in public spaces. As government agencies themselves have cautioned, face recognition surveillance 'has the potential to make people feel extremely uncomfortable, cause people to alter their behaviour, and lead to self-censorship and inhibition,' chilling the exercise of the rights protected under the First Amendment and calling into question the scope of protections offered by the Fourth Amendment.

Alongside its report, Big Brother Watch has launched the "Face Off" campaign calling for the UK public authorities to stop using automated facial recognition software with surveillance cameras, and to remove the thousands of images of unconvicted individuals from the UK's Police National Database. Given the UK authorities' world-famous love of CCTV and surveillance, it's unlikely they will take much notice.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: face recognition, flaws, uk

53 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

discordian_eris (profile), 21 May 2018 @ 3:52am

When wearing masks* is outlawed, only outlaws will wear masks.

*Or hoodies, large sunglasses, wigs, losing/gaining lots of weight, makeup, plastic surgery, etc. Well hell, there are thousands of non-adversarial things which can and will throw off facial recognition. Just wait 'til someone does a good job of studying what adversarial things can be done to thwart it. Oops, never you mind. That will be outlawed.
[ link to this | view in thread ]
Coyne Tibbets (profile), 21 May 2018 @ 4:10am

Re:
...so only outlaws will know it.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 4:21am

Pastafarians & the law

"facial recognition algorithms often disproportionately misidentify minority ethnic groups"

One reason for that is because minority ethnic groups are allowed to break the rules that are rigidly enforced on non-minorities, such as the wearing of (AFR-busting) head coverings in government photo identification.

https://www.mirror.co.uk/news/uk-news/dad-challenging-dvla-over-right-5473745
[ link to this | view in thread ]
Roger Strong (profile), 21 May 2018 @ 4:49am

Are we comfortable with a society where the government can find anyone, at any time, by continuously scanning the faces of people on the sidewalk?

It won't just be government. There's a growing industry of private companies with networks of license plate readers selling data to insurance companies, law enforcement, corporate investigators, etc.. You can be sure they'll branch into face recognition. Your purchase, social media and cell phone tower history is already for sale.

All those licence plate reader and facial recognition sightings are going into a database. Any face not tied to an real identity can be given a unique ID, tying all sightings of that face together. To be associated with a real identity later on.

So the government won't just know where you are; they'll know where you've been. They can tail you retroactively. Even if you weren't on the radar, once they add your photo to the database, it'll be connected to one of those previously anonymous unique IDs. You license plate will already have a long geolocation history. They'll get a complete dossier on you from the commercial databases.

But here's where it gets really ugly...

You will be associated with other people. When some nutjob does a terrorist attack and police build their instant dossier on him, they'll want to know who he's associated with. So they'll do a simple database search: "Show us anyone seen near this guy multiple times."

If you live in the same neighborhood, that may be you. If you're in the same minority ethnic circles, that's more likely to be you. If you both commute on the same bus every morning, that will be you. Confirmation bias kicks in. Good luck getting on an airplane after that.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 6:56am

As with any groundbreaking new technology, there are two ways to look at this: utopian and dystopian. For instance, in the distant future, such tech might help create a kind of cashless society (whether that's a good or bad thing is another question) such as when purchases at a checkout counter not only photograph the person, but do a retina scan as well (you know that's eventually coming) and there is no longer any need to carry a credit card because the person's body essentially is the credit card.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 6:59am

Re:
Soon, it will be illegal to wear clothing.
.. and then ..
Facial Regression is killing the clothing business!!!
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 7:00am

Re: Pastafarians & the law
Is that why?
Really
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 7:00am

Re:
Ummm - you assume it works?
[ link to this | view in thread ]
JoeCool (profile), 21 May 2018 @ 7:03am

Re:
Until someone pulls a Demolition Man and removes someone's eyeball. Once more with feeling! Biometric data is the USER NAME, not the PASSWORD!!
[ link to this | view in thread ]
Ninja (profile), 21 May 2018 @ 7:04am

Re:
Orwell was a prophet, not a writer.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 7:05am

In the movie "Jason Bourne", they supposedly use this imaginary Facial Recognition coupled with satellites to track in real time thru a congested city. This is complete hollywood bullshit, as usual. These silly hollywood types and their bullshit movies - lol.

The scary part is that some actually believe this shit.
[ link to this | view in thread ]
Ninja (profile), 21 May 2018 @ 7:05am

Re:
What about the utopian look?
[ link to this | view in thread ]
Ninja (profile), 21 May 2018 @ 7:06am

Re:
Can't tell if sarcasm or derp. I'll go with sarcasm.

It's scary that this will be possible in the future.
[ link to this | view in thread ]
Ninja (profile), 21 May 2018 @ 7:08am

Re:
No problem, you can only use State sanctioned clothing, haircut and the likes. You don't want to be subversive, eh?

Kim would be proud.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 8:05am

Re: Re:
It's the age-old question of balancing privacy & civil rights vs. security. A crime-free society (or nearly so) has always been possible for anyone who doesn't mind living in a repressive "big brother" police state. These new surveillance technologies shift that balance even farther toward the police state side, with the benefit of lower crime being the presumptive result.
[ link to this | view in thread ]
Andrew D. Todd, 21 May 2018 @ 8:15am

The Other Way To Do It.
The best way to identify someone is to ask them who they are, and then address the comparatively simple question of whether they are telling the truth. It is comparatively simple to harden ID cards or bank cards against forgery/theft with devices such as Chip&Pin, Biometrics, etc. A typical biometric is a finger print. Assuming that people sort out reliably into a hundred different categories on the basis of their fingerprints, it would be more or less impossible for a pickpocket to determine whose pocket to pick, but it is straightforward to ask people to take off their gloves before using the finger/palm scanner.

The British were attempting to identify football rioters, the ten thousand or so people who go to football games to riot. A more sensible approach would have been to create a secure ticket/ID card, including RFID, which could only be obtained with a reasonable security deposit. You can ask reasonable questions, for example, are there any cards which mysteriously vanished from RFID-view inside the stadium, as if their owners had wrapped them in tinfoil, before starting to skirmish. There are many offices and other workplaces (notably hospitals) where you have to wear your ID card on a cord around your neck. Now, of course there is a cost in this system. If you talk frankly to the average football fan about security, he may decide that he doesn't need to go to football games. He may well decide that it is no fun to be on his best behavior. That is why the British government tried to do stadium security "on the fake."

Artificial Intelligence seems to get tried under circumstances when the people using it was not prepared to do the logical and straightforward thing, and face the consequences. Tesla and Uber talk about Artificial Iintellecnce, because they are not prepared to face the necessity of buying roads for their cars to run over. The new California High-Speed railroad project is building a 3700 foot viaduct in Fresno, to carry the new high-speed railroad line over all manner of urban obstructions. Tesla isn't going to spend that kind of money.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 8:22am

whether it's accurate or not is irrelevant. all the authorities want is to be able to arrest people for no reason, charge them with even less reason and jail them for even less reason. if, during anything, the police can shoot and kill these 'well, we thought they looked similar' people, then there can be all sorts of praise meted out while the officer(s) concerned can have 'well deserved' months off work on full pay, then go back to duty without charge, while waiting for the next sap to come along!!
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 9:02am

Re: Re: Pastafarians & the law
Sometimes yes.

The one thing you can be certain of in society is that no two humans are equal. And the more you try to make it equal, the more unequal you ensure things will be.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 9:11am

Re: Re:
Why do you all complain? Has it not been told to you by even Orwell himself what is to come if you leave your fates to the decisions of government? Despite having been told of the cost of government power, and even discussing that issue here right now, it is certain you will still ensure that Orwell's prophecy will be fulfilled in your very pursuit to combat it!

You laud Orwell while calling those that would help prevent Orwell's vision from being reality fools. I don't think you guys actually understand Orwell at all.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 9:22am

Re: Pastafarians & the law
Where's the evidence this has anything to do with ethnic groups? It looks to be based on religion, and his religion is too much of a minority to break those rules.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 9:24am

Re:

When some nutjob does a terrorist attack and police build their instant dossier on him, they'll want to know who he's associated with. So they'll do a simple database search

Your future-tense description hides the fact that this has already happened.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 9:35am

Re: Re: Pastafarians & the law
It IS ethnic. Not all muslims wear burkas or enforce their wearing. By definition ethnic means..

eth·nic
ˈeTHnik/
adjective
adjective: ethnic

1.
relating to a population subgroup (within a larger or dominant national or cultural group) with a common national or cultural tradition.

As per the definition, the wearing of burkas is most definitely an "ethnic" thing because it is a subgroup activity and also common national and also cultural tradition for some of them.

The internet is your friend if you use it correctly.
[ link to this | view in thread ]
TRX, 21 May 2018 @ 9:39am

The police don't care about the system's accuracy. False positives are a feature; they provide probable cause.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 9:54am

Re: Re: Pastafarians & the law
With the notable exceptions of proselytizers such as Christanity and Islam, many religions and ethnic groups are essentially one and the same. Some religions are forbidden to even be discussed anyone outside of the tribe. Druze is just one such example.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 10:00am

Re: Re: Re: Pastafarians & the law
"many religions and ethnic groups are essentially one and the same."

no, that is not even close to true and is victim to serious over-simplification.

There are many warring factions of Islam, and while there are no know Christian actively warring with each other there are some pretty disparaging remarks made between the denominations.

So no, while it is tempting to make that comparison, history shows that it is quite literally deadly wrong, because some people would actually attempt to kill you for making it.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 10:07am

So this FaceID system fails, thinks you're a really dangerous person, and then the Cops shoot you!!! These days the cops are afraid of their own shadow and will pull their guns out first thing for anything and everything. I've seen it a number of times of guns pulled for a person holding a CAMERA!!!!

You're at least 9 times more likely to be killed by the police than a terrorist.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 10:07am

Re: Re: Re:
Sorry but the logic that a crime-free society requires a repressive big-brother is a classic oxymoron. If there is no crime then what does all this big brother do all the time? Repressive government by it's definition requires that crime be high to self justify its existence. If crime is not high, then make new laws to generate more crime, it is literally that simple.

Conversely, if you want less crime, just remove laws, if you want zero crime, then remove all laws.

The idea of a utopia is one where people automatically "realize" the benefit of living a certain way and opt to do so of their own free will. The literal reason why a utopia is not possible because we already know that people are never going to live towards the total benefit of society over the benefit of the self.

Rules and structure are required so that the humans that kill, take, or imprison people on behalf of those "rules" are justified in doing so by society.
[ link to this | view in thread ]
Ninja (profile), 21 May 2018 @ 11:16am

Re: Re: Re:
Enlighten us with a better system, oh oracle. I'm all ears (or eyes). Except you will not.

We already got you are the master of all answers and omniscient holder of all truth but we are too dumb to understand even if you told us. Now shoo, shoo. Go away.
[ link to this | view in thread ]
Ninja (profile), 21 May 2018 @ 11:17am

Re: Re: Re:
I'm still waiting the utopia. So far it's only dystopia.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 11:43am

Re: The Other Way To Do It.
With an ID card, where the data is stored matters. If it is on the card all that is actually proven is that the card data matches the carrier. If it is stored in a database, a paper card is sufficient, but the system can fail if the database is corrupted. That is any identity system is far from foolproof.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 11:43am

Re: Re: Re: Pastafarians & the law
That's fine and understandable - sort of, but do not attempt to blow smoke up my ass. Lies afford little gain with huge liabilities - so stupid.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 11:45am

Re: Re: Re: Re: Pastafarians &amp; the law
and this means what ... religion causes violence? - well, duh
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 11:48am

Re: Re: Re:
"with the benefit of lower crime being the presumptive result."

You presume quite a bit. I call bullshit.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 11:52am

Re: Re:
Scary, yes. And they want you to be scared, as fearful folk are easily manipulated.

In the future things will be different, hopefully better from a common human point of view, but probably not.

Bad hollywood physics still provides a good laugh.
[ link to this | view in thread ]
ShadowNinja (profile), 21 May 2018 @ 11:53am

Re: Re:
Nah, they'll outlaw smiling first.

Some DMV's have asked people to not smile in their photograph because it messes up the facial recognition software.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 11:54am

Re:
I read that high school students in the US are now more likely to be shot dead than a police officer.
[ link to this | view in thread ]
frank87 (profile), 21 May 2018 @ 1:32pm

it's a taboo
A few years ago I spoke an expert in pattern recognition. He told me what the problem was: people don't want to heat it, but really, there is very little variation in faces. Our faces all look alike (except for a few obvious differences), but our brains tell us differently.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 1:41pm

Re: Pastafarians & the law
Religion and the freedom of it, gives them the same rights Christians have, even if they are a minority with a different ethnicity, It's called adhering to the rules.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 1:46pm

Re: Re:
Everyone has an eye patch.
[ link to this | view in thread ]
Roger Strong (profile), 21 May 2018 @ 2:17pm

Re:

You will be associated with other people. […] they'll do a simple database search: "Show us anyone seen near this guy multiple times."

Speak of the devil. In today's news....

Next year, high schools in Lockport New York will use the "Aegis" CCTV and facial recognition system to track and record the interactions of students suspected of code of conduct violations, keeping a ledger of who speaks to whom, where, and for how long.

The record will be used to assemble evidence against students and identify possible accomplices to ascribe guilt to.

[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 6:12pm

Re: Re: Re:
and this is, sadly, why the joker does not have a license.
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 6:15pm

Re: Re:
"The Authorities" always claim the killer was known to them ... and yet they did not stop it. Why is this?
[ link to this | view in thread ]
Anonymous Coward, 21 May 2018 @ 6:17pm

Re: Re:
I laughed.
Doubt he gets it.
[ link to this | view in thread ]
Andrew D. Todd, 21 May 2018 @ 6:35pm

Re: Re: The Other Way To Do It.
Well, something like Chip&Pin works because the vast majority of card-holders have an investment in the system, and can be relied upon to provide physical security for their particular bis of it. The point is that you have a card which is a functioning cypher machine, and makes it difficult for intermediaries to gobble up useful information. In the case of pick-pockets and biometrics, all that is required is to shift the odds a bit, so that pick-pockets get caught in the act by their intended victims. The system isn't perfect, but it works well enough in practice.

The system has reasonable limits. You cannot buy a house with a bank card, so the stakes for failure aren't too high. On the other side, when you need to do five-cent ttransaactions, you do them with a a purchased vendo-card, not with your back card. Back in the day, circa 1990, I used to have three different photo-copier cards, for two university libraries and Kinkos copying shop, and recharged each, twenty dollars at a time..

Similarly, the vast majority of sports attendees don't wnat to sit next to a thug with a switchblade knife, and, within reason, they will agree to cross-check each-other's tickets, and expose anyone who hasn't got a valid ticket. Of course, as I said, this can backfire. The sports attendees may simply decides that they would prefer to watch their sports in more or less chosen company, at home, or in a sports bar. Sadiums have always built club-houses and press boxes for the choicest segments of their audiences, and a sports bar is just a reasonable development of this.
.
[ link to this | view in thread ]
Rekrul, 21 May 2018 @ 7:01pm

Re:

When wearing masks is outlawed, only outlaws will wear masks.

Actually...

https://en.wikipedia.org/wiki/Anti-mask_laws
[ link to this | view in thread ]
Bergman (profile), 21 May 2018 @ 8:05pm

Re:
Shaving.
[ link to this | view in thread ]
Anonymous Coward, 22 May 2018 @ 1:52am

Re: Re: Re:
Known != watched.
[ link to this | view in thread ]
Anonymous Coward, 22 May 2018 @ 6:12am

Re: Re:
Who will be there to protect us from all those trick or treaters?
[ link to this | view in thread ]
Anonymous Coward, 22 May 2018 @ 6:13am

Re: Re: Re: Re:
When they see something, they don't say anything.
[ link to this | view in thread ]
Wendy Cockcroft, 23 May 2018 @ 5:56am

Re: Re: Re: Re: Re: Pastafarians &amp; the law
Again with this?
[ link to this | view in thread ]
Wendy Cockcroft, 23 May 2018 @ 5:58am

Re: Re: Re: Re:
Once more with feeling: Anarchy doesn't scale!
[ link to this | view in thread ]
Anonymous Coward, 23 May 2018 @ 9:27pm

Amazon sells surveillance to law enforcement
[ link to this | view in thread ]
discordian_eris (profile), 24 May 2018 @ 11:52am

Re: Re: Re: The Other Way To Do It.
Chip and pin only works if the underlying technology and cryptography is sound. Ask Eastern Europe how that worked out for them. Hint, it didn't. Entire countries had to cancel their identity cards for simple fuck ups by the vendors they relied on.
[ link to this | view in thread ]