Utah Pulls Plug On Surveillance Contractor After CEO's Past As A White Supremacist Surfaces

from the belated-concerns-about-privacy-and-bias-are-better-than-no-concerns-at-all dept

A couple of months ago, a records request revealed a private surveillance contractor had access to nearly every piece of surveillance equipment owned and operated by the state of Utah. Banjo was the company with its pens in all of the state's ink. Banjo's algorithm ran on top of Utah's surveillance gear: CCTV systems, 911 services, location data for government vehicles, and thousands of traffic cameras.

All of this was run through Banjo's servers, which are conveniently located in Utah government buildings. Banjo's offering is of the predictive policing variety. The CEO claims its software can "find crime" without any collateral damage to privacy. This claim is based on the "anonymization" of harvested data -- a term that is essentially meaningless once enough data is collected.

This partnership is now on the rocks, thanks to an investigation by Matt Stroud and OneZero. Banjo's CEO, Damien Patton, apparently spent a lot of his formative years hanging around with white supremacists while committing crimes.

In grand jury testimony that ultimately led to the conviction of two of his associates, Patton revealed that, as a 17-year-old, he was involved with the Dixie Knights of the Ku Klux Klan. On the evening of June 9, 1990 — a month before Patton turned 18 — Patton and a Klan leader took a semi-automatic TEC-9 pistol and drove to a synagogue in a Nashville suburb. With Patton at the wheel, the Ku Klux Klan member fired onto the synagogue, destroying a street-facing window and spraying bullets and shattered glass near the building’s administrative offices, which were next to that of the congregation’s rabbi. No one was struck or killed in the shooting. Afterward, Patton hid on the grounds of a white supremacist paramilitary training camp under construction before fleeing the state with the help of a second Klan member.

If you're wondering where the state of Utah's due diligence is in all of this, there's a partial explanation for this lapse: the feds, who brought Patton in, screwed up on their paperwork.

Because Patton’s name was misspelled in the initial affidavit of probable cause filed in Brown’s case — an FBI agent apparently spelled Damien with an “o” rather than an “e” — any search of a federal criminal court database for “Damien Patton” would not have surfaced the affidavit.

Now that his past has been exposed, the state of Utah has announced it won't be working with Banjo.

The Utah attorney general’s office will suspend use of a massive surveillance system after a news report showed that the founder of the company behind the effort was once an active participant in a white supremacist group and was involved in the shooting of a synagogue.

The AG's office can only shut down so much of Banjo's surveillance software. Other government agencies not directly controlled by the state AG are making their own judgment calls. The University of Utah is suspending its contract with Banjo, but the state's Department of Public Safety has only gone so far as to "launch a review" of its partnership with the company. City agencies and a number of police departments who have contracts with Banjo have yet to state whether they will be terminating theirs.

And the AG's reaction isn't a ban. The office appears to believe it might be able to work through this.

“While we believe Mr. Patton’s remorse is sincere and believe people can change, we feel it’s best to suspend use of Banjo technology by the Utah attorney general’s office while we implement a third-party audit and advisory committee to address issues like data privacy and possible bias,” Piatt said. “We recommend other state agencies do the same.”

It's refreshing to hear a prosecutor state that it's possible for former criminals to turn their lives around and become positive additions to their communities, but one gets the feeling this sort of forgiveness is only extended to ex-cons who have something to offer law enforcement agencies. Everyone else is just their rap sheet for forever, no matter how many years it's been since their last arrest.

The other problem here is the DA's office's tacit admission it did not take data privacy or possible bias into account before granting Banjo access to the state's surveillance equipment, allowing it to set up servers in government buildings, and giving it free rein to dust everything with its unaudited AI pixie dust.

These are all steps that should have taken place before any of this was implemented, even if the state had chosen to do business with a company with a less controversial CEO. This immediate reaction is the right step to take, but a little proactivity now and then would be a welcome change.

Filed Under: cctv, damien patton, precrime, predictive policing, surveillance, utah
Companies: banjo

Harrisburg University Researchers Claim Their 'Unbiased' Facial Recognition Software Can Identify Potential Criminals

from the fresh-hells-delivered-daily dept

Given all we know about facial recognition tech, it is literally jaw-dropping that anyone could make this claim… especially without being vetted independently.

A group of Harrisburg University professors and a PhD student have developed an automated computer facial recognition software capable of predicting whether someone is likely to be a criminal.

The software is able to predict if someone is a criminal with 80% accuracy and with no racial bias. The prediction is calculated solely based on a picture of their face.

There's a whole lot of "what even the fuck" in CBS 21's reprint of a press release, but let's start with the claim about "no racial bias." That's a lot to swallow when the underlying research hasn't been released yet. Let's see what the National Institute of Standards and Technology has to say on the subject. This is the result of the NIST's examination of 189 facial recognition AI programs -- all far more established than whatever it is Harrisburg researchers have cooked up.

Asian and African American people were up to 100 times more likely to be misidentified than white men, depending on the particular algorithm and type of search. Native Americans had the highest false-positive rate of all ethnicities, according to the study, which found that systems varied widely in their accuracy.

The faces of African American women were falsely identified more often in the kinds of searches used by police investigators where an image is compared to thousands or millions of others in hopes of identifying a suspect.

Why is this acceptable? The report inadvertently supplies the answer:

Middle-aged white men generally benefited from the highest accuracy rates.

Yep. And guess who's making laws or running police departments or marketing AI to cops or telling people on Twitter not to break the law or etc. etc. etc.

To craft a terrible pun, the researchers' claim of "no racial bias" is absurd on its face. Per se stupid af to use legal terminology.

Moving on from that, there's the 80% accuracy, which is apparently good enough since it will only threaten the life and liberty of 20% of the people it's inflicted on. I guess if it's the FBI's gold standard, it's good enough for everyone.

Maybe this is just bad reporting. Maybe something got copy-pasted wrong from the spammed press release. Let's go to the source… one that somehow still doesn't include a link to any underlying research documents.

What does any of this mean? Are we ready to embrace a bit of pre-crime eugenics? Or is this just the most hamfisted phrasing Harrisburg researchers could come up with?

A group of Harrisburg University professors and a Ph.D. student have developed automated computer facial recognition software capable of predicting whether someone is likely going to be a criminal.

The most charitable interpretation of this statement is that the wrong-20%-of-the-time AI is going to be applied to the super-sketchy "predictive policing" field. Predictive policing -- a theory that says it's ok to treat people like criminals if they live and work in an area where criminals live -- is its own biased mess, relying on garbage data generated by biased policing to turn racist policing into an AI-blessed "work smarter not harder" LEO equivalent.

The question about "likely" is answered in the next paragraph, somewhat assuring readers the AI won't be applied to ultrasound images.

With 80 percent accuracy and with no racial bias, the software can predict if someone is a criminal based solely on a picture of their face. The software is intended to help law enforcement prevent crime.

There's a big difference between "going to be" and "is," and researchers using actual science should know better than to use both phrases to describe their AI efforts. One means scanning someone's face to determine whether they might eventually engage in criminal acts. The other means matching faces to images of known criminals. They are far from interchangeable terms.

If you think the above quotes are, at best, disjointed, brace yourself for this jargon-fest which clarifies nothing and suggests the AI itself wrote the pullquote:

“We already know machine learning techniques can outperform humans on a variety of tasks related to facial recognition and emotion detection,” Sadeghian said. “This research indicates just how powerful these tools are by showing they can extract minute features in an image that are highly predictive of criminality.”

"Minute features in an image that are highly predictive of criminality." And what, pray tell, are those "minute features?" Skin tone? "I AM A CRIMINAL IN THE MAKING" forehead tattoos? Bullshit on top of bullshit? Come on. This is word salad, but a salad pretending to be a law enforcement tool with actual utility. Nothing about this suggests Harrisburg has come up with anything better than the shitty "tools" already being inflicted on us by law enforcement's early adopters.

I wish we could dig deeper into this but we'll all have to wait until this excitable group of clueless researchers decide to publish their findings. According to this site, the research is being sealed inside a "research book," which means it will take a lot of money to actually prove this isn't any better than anything that's been offered before. This could be the next Clearview, but we won't know if it is until the research is published. If we're lucky, it will be before Harrisburg patents this awful product and starts selling it to all and sundry. Don't hold your breath.

Filed Under: ai, artificial intelligence, bias, facial recognition, precrime
Companies: harrisburg university

IBM Helps Florida Predict Just How Delinquent Your Child's Going To Be

from the your-future-self-appears-to-be-a-trouble-maker dept

We've covered several different instances where the country has been taking baby steps toward the kind of precognitive crime prevention featured in the movie Minority Report -- sans naked gibbering women floating in bathtubs. The most recent effort was courtesy of the Homeland Security Department, who is busily developing a body language analysis prediction system dubbed "Future Attribute Screening Technologies" (FAST) -- which aims to detect "shifty" people who may be getting ready to commit a crime of some sort (or just drank way too much coffee).

More common approaches simply involve software that analyzes a database of offenders and cherry picks out the most likely future offenders (very popular in the UK), or analyzes crime patterns to predict future criminal trends. Along those lines, it looks like the Florida Department of Juvenile Justice has decided to start using IBM predictive analytics software (via Gizmodo) to help them determine which of the 85,000 kids who enter their system each year poses the biggest future threat. IBM has this to say about the new system -- which was an upgrade from Excel:

"Predictive analytics gives government organizations worldwide a highly-sophisticated and intelligent source to create safer communities by identifying, predicting, responding to and preventing criminal activities. It gives the criminal justice system the ability to draw upon the wealth of data available to detect patterns, make reliable projections and then take the appropriate action in real time to combat crime and protect citizens."

Of course many of these patterns simply become evident when people bother to pay attention and use their intellect, and these tools are often just an extension of that. When prediction technology is used, the technology will only be as good as the people using it (in this case to choose rehabilitation paths for kids). But you still have to wonder how accurate these kinds of systems are and how independently verifiable the evidence will be. Can kids who feel they were unfairly, preemptively declared to be bad asses in 2014 see the "reliable" source code?

Filed Under: crime, florida, precrime
Companies: ibm