Facial Recognition Company Says It Won't Sell To Law Enforcement, Knowing It'll Be Abused

(Mis)Uses of Technology

from the taking-a-stand dept

Fri, Jun 29th 2018 1:30pm — Mike Masnick

We just recently wrote about employees at Amazon speaking out inside the company to complain about the company selling its face recognition tools (called: "Rekognition") to law enforcement. That resulted in the CEO of a maker of facial recognition software, Brian Brackeen, to publicly state that his company, Kairos, will not sell to law enforcement.

The full article is worth reading, but he notes that the technology will be abused and misused by law enforcement -- and often in a way that will lead to false arrests and murder:

Having the privilege of a comprehensive understanding of how the software works gives me a unique perspective that has shaped my positions about its uses. As a result, I (and my company) have come to believe that the use of commercial facial recognition in law enforcement or in government surveillance of any kind is wrong — and that it opens the door for gross misconduct by the morally corrupt.

To be truly effective, the algorithms powering facial recognition software require a massive amount of information. The more images of people of color it sees, the more likely it is to properly identify them. The problem is, existing software has not been exposed to enough images of people of color to be confidently relied upon to identify them.

And misidentification could lead to wrongful conviction, or far worse.

As he states later in the piece:

There is no place in America for facial recognition that supports false arrests and murder.

It's good to see this, and whether you support the police or not, we should appreciate this moment -- just as we should appreciate the people at Amazon who stood up and complained about this. Too often lately, the tech industry is getting slammed for not taking into account the impact of their technology in their rush to push forward innovation at any costs. I've always felt that that narrative is a bit exaggerated. I talk to a lot of entrepreneurs who really do think quite a lot about how their technology may impact the world -- both good and bad -- but it's good to see people in the industry speaking out publicly about how that might happen, and why they need to make sure not to oversell the technology in a way where it's likely to cause real harm.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: brian brackeen, face recognition, law enforcement
Companies: kairos

20 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Rocky, 29 Jun 2018 @ 2:02pm

Tragic
It's tragic that we no longer can trust law enforcement with tools that would make everyone safer.
[ link to this | view in thread ]
Anonymous Coward, 29 Jun 2018 @ 2:15pm

But will they use resellers who do?
The only way they can prevent state use is to restrict it explicitly in their licensing. And I seriously doubt they've done that.

More likely this is just a publicity stunt, and they've already filed a second set of incorporation papers, for the company that will be exclusively used for government business. And that company will of course, only hire people who have active secret clearances, so that if anybody talks about the product, they will get federal jail time.

This is how things are done now. Businesss' that do business with the state compartmentalize to allow the state to implement more brutal oversight over their employees. The company extrenalizes its internal security costs to the state, and the state gets to keep its accelerating slide into fascism a secret a little longer.
[ link to this | view in thread ]
Ninja (profile), 29 Jun 2018 @ 2:21pm

This is refreshing. It was despairing to see all the violations with no push back. It's a glimpse of hope in a world dominated by retrocession and dismantling of hard earned rights and improvements regarding bigotry.
[ link to this | view in thread ]
Anonymous Coward, 29 Jun 2018 @ 2:59pm

ROFL!
"Facial Recognition Company Says It Won't Sell To Law Enforcement"

Because they have already sold it to all of them.
[ link to this | view in thread ]
DocGerbil100 (profile), 29 Jun 2018 @ 5:30pm

I tip my hat to Brian Brackeen and Kairos.

Even in the event of a compartmentalised corporate structure that allows abusive trade to be laundered out (as per AC's comment above), the mere acknowledgement of the issues is something that would inevitably come back to bite them later on. Few such secrets ever seem likely to last forever.

Today, this man and his company show every sign of having conducted themselves with honour. A measure of respect is something they've earned. :)
[ link to this | view in thread ]
That Anonymous Coward (profile), 29 Jun 2018 @ 6:12pm

It would be nice if someone could explain to the senior citizens demanding this that it does not work like in the movies & its not going to anytime soon.
[ link to this | view in thread ]
That One Guy (profile), 29 Jun 2018 @ 8:45pm

Re:
Well not with that attitude it won't! Clearly the tech would work perfect, if not better, if those lazy slackers would just nerd harder!
[ link to this | view in thread ]
Coyne Tibbets (profile), 29 Jun 2018 @ 9:49pm

Re: Tragic
I think your statement assumes facts not in evidence.

The evidence that we have seen doesn't really show that facial recognition will make people safer; in fact, there is a fair indication that the contrary would be true.
[ link to this | view in thread ]
Coyne Tibbets (profile), 29 Jun 2018 @ 9:51pm

Re:
No they didn't. They gave it to them. They're getting their money charging by the face.
[ link to this | view in thread ]
Rocky, 30 Jun 2018 @ 3:03am

Re: Re: Tragic
Uhm, that's because the technology is also used by entities that doesn't have the welfare of the people as their first priority, in other words it is entities we can't trust.

And my comment wasn't specific to face recognition.
[ link to this | view in thread ]
Anonymous Cowherd, 30 Jun 2018 @ 3:40am

Then why develop face recognition software at all?
[ link to this | view in thread ]
JoeCool (profile), 30 Jun 2018 @ 6:15am

Re:
Because it has wide applications in fields that do NOT have anything to do with any kind of enforcement. Mostly advertising and sales. Maybe education.
[ link to this | view in thread ]
DOlz (profile), 30 Jun 2018 @ 6:33am

I feel much better
"The full article is worth reading, but he notes that the technology will be abused and misused by law enforcement …”

So I only will be abused by commercial interest then?
[ link to this | view in thread ]
Anonymous Coward, 30 Jun 2018 @ 2:45pm

Re:
I refer the senior citizenry to the film "Minority Report" If that doesn't frighten them or atleast planta seedof critical thinking....Get off my lawn!!!
[ link to this | view in thread ]
JoeCool (profile), 30 Jun 2018 @ 5:08pm

Re: I feel much better
Yeah, but you have at least SOME chance of getting justice when abused by civilians. You have none at all if abused by law enforcement.
[ link to this | view in thread ]
Coyne Tibbets (profile), 1 Jul 2018 @ 7:49am

Re: Re: Re: Tragic
Good point about your comment not being specific to facial recognition, I overlooked that.

But, for the rest, I would counter that some tools cannot be used safely by anyone, even with best intentions.
[ link to this | view in thread ]
Anonymous Coward, 1 Jul 2018 @ 8:14am

Re: Re: Re: Re: Tragic
The problem with tools is that once they have been invented, they cannot be uninvented. Also, when governments try to prevent the use of tools, they always allow themselves to have the tools.

That said, a tool like facial recognition can be useful even with a very high error rate, simply because it can direct a human to take a closer look at a few faces in a sea of thousands of faces. From what I have seen, that is how the UK police use it, as a means of directing a trained officer to take a closer look, and not as an indication that they should go and arrest a face. U.S police practices on the other hand differ and it becomes much more dangerous in their hands.
[ link to this | view in thread ]
Anonymous Coward, 1 Jul 2018 @ 9:36am

Re: Re: Re: Re: Re: Tragic
In the UK, how many crimes have been solved as a result of CCTV?
[ link to this | view in thread ]
Anonymous Coward, 1 Jul 2018 @ 10:30am

Re: Re: Re: Re: Re: Re: Tragic
They have been of assistance in quite a few, often involving the police going round and collecting videos, from private cctv outside shops etc. that might be of use. Then that is not a case of building a huge database, and hoping software will give useful results, but rather police doing a lot of legwork to collect and analyze evidence.
[ link to this | view in thread ]
Anonymous Coward, 2 Jul 2018 @ 6:54am

..we swear that we will not *sell* this to the police.. HOWEVER.. should they ask nicely... with sugar on top.. (or threaten to kick us in the balls with 37 years of guaranteed tax audits....)
[ link to this | view in thread ]