Facial Recognition Software Finally Gets Around To Getting An Innocent Person Arrested
from the great-job-everyone dept
Well, it's happened. The thing people have been warning about for years. A person lost some of their freedom due to a facial recognition mismatch. It may have only been 30 hours, but it should have been zero. And it might have been zero hours if investigators had bothered to read the disclaimers attached to its facial recognition search results.
According to the New York Times report, this is the first time a false positive has led to someone being arrested. Or, at least, the first time the public's been made aware of it. A few years ago, the FBI and a local law enforcement agency used "facial analysis" performed by humans to arrest the wrong man twice for two separate robberies. This time, it was software. And it took 30 hours away from an innocent person.
On a Thursday afternoon in January, Robert Julian-Borchak Williams was in his office at an automotive supply company when he got a call from the Detroit Police Department telling him to come to the station to be arrested. He thought at first that it was a prank.
An hour later, when he pulled into his driveway in a quiet subdivision in Farmington Hills, Mich., a police car pulled up behind, blocking him in. Two officers got out and handcuffed Mr. Williams on his front lawn, in front of his wife and two young daughters, who were distraught. The police wouldn’t say why he was being arrested, only showing him a piece of paper with his photo and the words “felony warrant” and “larceny.”
His wife, Melissa, asked where he was being taken. “Google it,” she recalls an officer replying.
Googling it would not have helped. Williams was taken away by cops and held for 30 hours, accused of shoplifting watches from an upscale store nearly two years earlier. All the police had to work with were a blurry, lo-res screengrab from the store's CCTV camera and facial recognition software provided by DataWorks Plus. DataWorks tests AI algorithms created by contractors by running searches using low-quality images. DataWorks does not provide any measurements of these algorithms' accuracy, however. It apparently packages up its collection of algorithms and sells access to government agencies.
Five months after the crime was committed, Michigan State Police digital image examiner Jennifer Coulson uploaded the image captured by the store's camera. But investigators apparently ignored the big, bold warning attached to the top of the search results.
After Ms. Coulson, of the state police, ran her search of the probe image, the system would have provided a row of results generated by NEC and a row from Rank One, along with confidence scores. Mr. Williams’s driver’s license photo was among the matches. Ms. Coulson sent it to the Detroit police as an “Investigative Lead Report.”
“This document is not a positive identification,” the file says in bold capital letters at the top. “It is an investigative lead only and is not probable cause for arrest.”
The State Police still maintain investigators did nothing wrong. They didn't only rely on the facial recognition search results. They also showed Williams' drivers license photo to the loss prevention person at the store (as part of a "six-pack" of facial photos) and she supposedly identified him.
The contractors that supplied the AI system to the State Police seem skeptical of everything that happened here -- even their own software's participation in the arrest of Williams.
[Brendan] Klare, of Rank One, found fault with [the loss prevention person's] role in the process. “I am not sure if this qualifies them as an eyewitness, or gives their experience any more weight than other persons who may have viewed that same video after the fact,” he said. John Wise, a spokesman for NEC, said: “A match using facial recognition alone is not a means for positive identification.”
One of the officers who questioned Williams said -- after viewing the surveillance video again with Williams in the room -- that he "guessed the computer got it wrong." Even so, Williams was held for another several hours before being released on $1,000 bond. According to his first-person account of this incident, the first 18 hours of his detainment were spent in an overcrowded holding cell without any interaction with investigators or the officers who arrested him.
Two weeks later, prosecutors dropped the charges, but left themselves the option to refile the charges in the future. And the prosecutor's office appears to think that's a possibility. The spokesperson says there's another witness investigators are interviewing and cops might try to run Williams in again for a crime he didn't commit.
And Williams has an alibi. He posted a video to his Instagram account during his commute home from work. This occurred during the time the store was being robbed. But investigators never bothered to check. And this investigation -- which appears to have stemmed almost solely from a "match" generated by a lo-res CCTV screengrab -- resulted in an innocent man having his life severely disrupted.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: arrests, detroit, facial recognition, robert williams, wrong guy
Reader Comments
Subscribe: RSS
View by: Time | Thread
How can a loss prevention officer be a reliable witness for a crime that occurred two years earlier? I can't remember the faces of people I saw last week.
[ link to this | view in chronology ]
Yeah, But,
You're expecting people with an IQ barely above that of prison guards and TSA agents to use reason.
[ link to this | view in chronology ]
Re: Yeah, But,
More expecting a US cop to take any opportunity to hassle a person of colour.
[ link to this | view in chronology ]
Re: Yeah, But,
These are the same people who spell "IQ" as "QI".
[ link to this | view in chronology ]
They are too busy beinf defensive of themselves and their bullshit system to bother to see that justice is served.
Can't even blame the facerec itself for this one.
[ link to this | view in chronology ]
Facial rec software finally ADMITS to an innocent arrested
There, fixed the headline for ya....
[ link to this | view in chronology ]
I'm guessing that officer no longer works for that PD (because in a sane world no PD would want to hire an officer that behaves like that. And we live in a sane world... right?)
[ link to this | view in chronology ]
Re:
Consults Magic 8-Ball
Signs point to "no."
[ link to this | view in chronology ]
The biggest problem here is not the arresting officers, nor the software, but the gormless legal pissant in the DA's office who turned an "investigative lead" into an "arrest warrant" in the face of a rock-solid alibi. The "system" failed, for the same reason it always does--a human being acting inhumanely. (But the failure is not always the same place in the system, because it's humans all the way down.)
[ link to this | view in chronology ]
"but left themselves the option to refile the charges in the future"
Which is 'justice speak' for don't even think about bringing suit against us for screwing your entire life up b/c we fscked up.
Well get QI, you'll get nothing, & we'll make sure you drive very very carefully for the rest of your days.
(Don't believe that last one?? NYPD beat people outside Stonewall on the anniversary... and you claim they don't carry grudges.)
[ link to this | view in chronology ]
You guys need to cut the cops some slack. Because if you’ve never gotten up in the morning to go to work not knowing whether you might have to shoot an innocent person in the face because you got scared, who are you to judge?
[ link to this | view in chronology ]
Training and Experience
Shouldn't the officers' "training and Experience" told them that shoplifters often commit more that one crime? That would've lead them to check on thefts from other retailers to try to establish a pattern. Wouldn't they have known that the thieves frequently sell or trade stolen merchandise? That might have meant that they would have tried to track down the stolen goods to see where they came from.
This just seems like lazy investigative work, and it feels negligent too, since an innocent person payed a price
[ link to this | view in chronology ]
I'm guessing the real thief is a 5foot tall white grandma with only one leg and the "suspect" (not to put too fine a point on it) is black........
because thats basically all the cops need now. An accusation of "being black without a license".
[ link to this | view in chronology ]
Re:
More like being Black without a leash. Or breathing while Black. Seems to be a huge problem with most cops and many people.
[ link to this | view in chronology ]
Anyone ever..
See those crappy Camera's??
And how they get setup?
Mounted to high, not at face level.
Aimed at a Bad position? Sun in the Lens, Not exterior Aimed at the front, NOT focus-able and more then 10-12 feet away..
And it dont matter if you are a 7/11 on the corner or an expensive Jeweler.
most are bad for most recognition, and that dont count if they protect their face, even a small amount.
[ link to this | view in chronology ]
Still thinking it's a good idea to disband the justice system
[ link to this | view in chronology ]
Re: Still thinking it's a good idea to disband the justice syste
Any scheme where bad actors are not inherently limited in the amount of damage they can exact in the pursuit of their own advantage will eventually trigger the "this is why we cannot have nice things" effect.
Disbanding the justice system may be a good idea, but it requires a different humanity. Preventing the justice system from rotting (and keeping to prune/disband those parts that have deteriorated beyond salvaging) is more realistic, but it's a permanent task that will never be completed.
[ link to this | view in chronology ]
Why we can't have nice things
I am pretty sure the US justice system (and the state justice systems underneath them) are too rotten to be adequately addressed by our systems of government to be successfully reformed, though this is not just because of the the level of power attained by the justice system (that can kill or imprison at whim and with impunity) but also because our representatives do not serve the public and our electoral system -- intended to be an overseeing influence on our officials -- doesn't actually do that at all.
But this is not to say I think we should abolish the justice system and and instead have anarchy. We'll still have bad actors.
But many of them can be reduced by providing them viable legal options. (Right now we provide them non-viable legal options.)
Many of them can be reduced by decriminalizing victimless crimes, (e.g. most of our drug crimes) and treating people for addiction and need for illicit medication to cope.
Many of them aren't even addressed by current law enforcement. Most of our violence is, by far, domestic violence, and yet police intervention invariably makes those situations worse for the victims. I don't know where to begin looking for an answer, but I know sending in guys with guns isn't working.
According to BJS, about 65% of major crimes investigated by law enforcement go unsolved. And then there's huge numbers of incidents where the police don't bother.
One category is Elite Deviance (White Collar Crime) which causes more death and more loss of property and more poverty than all the small time bad actors combined, by multiple orders of magnitude. No one went to prison for the Subprime Mortgage Crisis. Those that were convicted in the Enron affair were convicted of defrauding shareholders, the population of California never got justice, but the body counts for both were immense.
No. The law enforcement system is a failure. The police are a separate soldier caste, much like German Freikorps. Prosecutors (mostly white, male and religious) send warm bodies to prison to fuel their political careers. Judges notoriously discriminate against poorer, darker-skinned, people that don't conform to their standards of upright citizen. And this is before we consider capture of the courts by a far-right religious conservative shadow organization.
No. The whole thing needs to go. And it needs to be disbanded now, or we're going to burn it down later.
[ link to this | view in chronology ]
Why I have to edit my rants
[revised] The failure of our justice system is not just because of the the level of power attained by the justice system -- one that can kill or imprison at whim and with impunity -- but also because our representatives don't actually represent the interests of the public...
And that is due to the failure of our election system which can easily be gamed so that we are forced to choose between one corporatist and another or a monster and a slightly-less-monstrous choice.
I'm sorry my grammar is that of a third grader.
At this point, if we were to truly try to reform our current legal system from the beat officer to the US Supreme Court jurists, I think it would take centuries before we'd see something resembling police as they are imagined in fictional media (I like to point at Gotham since the GCPD is supposed to be corrupt.)
And we're going to be long extinct from ecology collapse before that happens.
So abolition it is.
[ link to this | view in chronology ]
Re: Re: Still thinking it's a good idea to disband the justice s
"Any scheme where bad actors are not inherently limited in the amount of damage they can exact in the pursuit of their own advantage will eventually trigger the "this is why we cannot have nice things" effect."
Yeah, but...how come this sort of shit happens almost exclusively in either the US or in known ultra-authoritarian dictatorships and official rogue nations?
"Disbanding the justice system may be a good idea, but it requires a different humanity."
Or it requires a citizenry which rightly realizes that Quis Custodes Ipsos Custodiet has withstood the passage of a good many years because, you know, it isn't rocket science to figure out that law enforcement privileges as they exist in the US draws sadists, thugs, bullies and racists like iron dust to a magnet.
I'm not sure by this time it's even possible to salvage much of the US law enforcement system. There's a centuries-old culture of having the back of even the worst thug in the precinct to contend with. Anyone able to deal with that would find it easy to teach gender equality principles to persian mullahs.
[ link to this | view in chronology ]
Is this how the Detroit Police Department locates criminal suspects? Drugs? Stolen property? Missing persons?
Google it???
[ link to this | view in chronology ]
Re:
Just stick a pin somewhere in Google Maps, and go there in force. Yeah, this pretty much seems to be the case.
[ link to this | view in chronology ]
"Google it"
Translation: I don't have to give you an answer at all so GFY.
[ link to this | view in chronology ]
Lets just blame the computer, not the idiots.
In another article on the subject, it had stated that the poor quality photo was clearly not Williams. What happened to the officer to looked at the computer image and the 'matched.' The rent-a-cop from the store matched the image too.
Blame the GPS when the guy following the GPS drives off of a cliff, ignoring all the "ROAD CLOSED" signs.
[ link to this | view in chronology ]
Blame the computer
When the $2 field drug test yields a false positive, you're a perp.
When the dog signals at you to get a treat, you're a perp.
When wrong address on the warrant is your address, you're a perp.
When you have a large amount of cash (>$60), you're a perp.
When the computer says your face is the face of another guy with a rap sheet, you're a perp.
Once you're a perp, it doesn't matter what you did, just what they can get you for, so long as you'll be in a cage for years.
[ link to this | view in chronology ]
Re: Blame the computer
Before all of this newfangled tech some poor overworked officer actually had to have a hunch that the presumptive per was bad. The dog and the computer saves these "sterling" enforcers of law much intellectual energy they'd rather employ in swinging their nightsticks at whoever the magic 8-ball pointed at.
I guess that's why the police have started taking a knee lately. It's their way of protesting the hazards and stresses of the job.
"Once you're a perp, it doesn't matter what you did, just what they can get you for, so long as you'll be in a cage for years."
Or, not unlikely, dead due to "resisting arrest" for breathing as the relevant arresting officer took the above-mentioned knee.
[ link to this | view in chronology ]
Williams has an alibi.
He posted a video to his Instagram account during his commute home from work. This occurred during the time the store was being robbed. But investigators never bothered to check.
Maybe the police should have Googled it!!
[ link to this | view in chronology ]
Can this guy sue?
Anybody know how this works?
If he were to sue for false arrest, would he have a reasonable chance of winning?
[ link to this | view in chronology ]
Re: Can this guy sue?
Yeah, this seems like it's almost certainly going to wind up with the local residents paying even more money on the settlement he gets when this winds up in court... and the cops will learn nothing as they don't have to pay it. Yay.
[ link to this | view in chronology ]
A lazy way
to make $1000.00. No doubt he did not get that money back.
[ link to this | view in chronology ]
Black person? Check. Color me not surprised. I had that article about AI bias due to biased data being fed to it but I'm lazy to find the link now.
[ link to this | view in chronology ]