Chicago PD Oversight Says ShotSpotter Tech Is Mostly Useless When It Comes To Fighting Gun Crime

from the but-that's-the-thing-it-says-it's-good-at-[confused-noises] dept

Gunshot detection tech provider ShotSpotter is fighting a PR battle on multiple fronts after more news surfaced that its analysts may alter detection records to fit police narratives and investigators' theories. Communications and court documents obtained by the Associated Press confirmed ShotSpotter allows law enforcement officers to request modifications to detection records. And the company apparently used to allow police officers to modify the data themselves.

In addition to its questionable handling of evidence, ShotSpotter is also shedding customers. Law enforcement agencies in some cities have decided it's not worth paying for a product that can't reliably detect gunshots. Cities that have dumped ShotSpotter have reported false positive rates as high as 75%.

ShotSpotter has fired back, claiming everyone reporting on its tech is wrong about its tech. It also claims it doesn't alter or allow alteration to reports submitted as evidence in criminal cases. Its assertions ring pretty fucking hollow in the face of all of this reporting, which relies on documents filed in court or obtained through public records requests. ShotSpotter's claims, however, are supported by nothing more than the company's own ineffective anger.

Now, there's even more evidence showing ShotSpotter isn't worth paying for. The Chicago PD's Inspector General has concluded its investigation of the tech the city pays roughly $11 million/year for. And it has found the tech doesn't seem to be worth the money.

The City of Chicago Office of Inspector General’s (OIG) Public Safety section has issued a report on the Chicago Police Department’s (CPD) use of ShotSpotter acoustic gunshot detection technology and CPD’s response to ShotSpotter alert notifications. OIG concluded from its analysis that CPD responses to ShotSpotter alerts can seldom be shown to lead to investigatory stops which might have investigative value and rarely produce evidence of a gun-related crime.

[...]

The CPD data examined by OIG does not support a conclusion that ShotSpotter is an effective tool in developing evidence of gun-related crime.

That's pretty damning. Compare and contrast with ShotSpotter's own irate statements in defense of its product:

[T]he ShotSpotter system is highly accurate at detecting outdoor gunshots and benefits communities battling gun violence.

Well, there's plenty of evidence out there saying the system isn't accurate. And this report [PDF] by the Chicago PD's oversight contradicts the second part of the company's claim. It isn't benefiting "communities battling gun violence." According to this investigation, only the rarest spotted shot leads to anything that might battle gun violence.

And, according to this investigation, the installation of the tech is actually causing more problems for areas of Chicago where gun violence is already an issue. ShotSpotter has given Chicago police officers yet another excuse to engage in suspicionless stops and searches. This is from the report:

In reviewing ISR [investigative stop report] narratives for mentions of ShotSpotter alerts, OIG also identified 10 ISRs (13.9%) in which reporting officers referred to the aggregate results of the ShotSpotter system as informing their decision to initiate a stop or their course of action during the stop, even when they were not responding to a specific ShotSpotter alert. For example, some officers during the reporting period identified the fact of being in an area known to have frequent ShotSpotter alerts as an element of the reasonable suspicion required to justify the stop. Other officers reported conducting “protective pat downs” following a stop because they knew themselves to be in areas where ShotSpotter alerts were frequent.

If there's a silver lining for ShotSpotter in this report, it's one that only benefits the tech provider, rather than Chicago residents. The Inspector General says it's possible there are more investigations linked to ShotSpotter detections, but it can't really tell because the Chicago PD's recordkeeping is a mess. What the IG sees is almost no connection between ShotSpotter reports and gun-crime investigations. But there's a slim chance this may be the PD's fault.

If this result is attributable in part to missing or nonmatched records of investigatory stops that did take place as a direct consequence of a ShotSpotter alert, CPD’s record-keeping practices are obstructing a meaningful analysis of the effectiveness of the technology.

And, if that's part of the problem, then that's on the Chicago PD if the city decides to stop paying $11 million a year for nothing more than additional rights violations by police officers.

Filed Under: chicago, chicago police department, evidence, shotspotter
Companies: shotspotter

Investigation Of ShotSpotter's Practices Is Raising Questions The Company's Angry Statement Really Doesn't Answer

from the bleeding-from-self-inflicted-wounds-always-sucks dept

Earlier this month, another courtroom challenge of evidence exposed another questionable alteration of a gunshot report by law enforcement tech supplier, ShotSpotter. In 2018, a man shot by police officers claimed in his lawsuit that ShotSpotter altered gunshot detection records at the request of law enforcement to back up the officers' narrative -- one that claimed he had shot at them first. No gun was ever recovered and the number of shots originally detected by ShotSpotter matched the number fired by officers, leaving them at least one shot short of their "he shot first" story.

This appears to have happened again. A man, apparently falsely arrested for a murder he didn't commit, was put in jail for eleven months based almost solely on ShotSpotter reports. The problem with the ShotSpotter report is that it kept changing. And again, the alterations made the report align with the presuppositions of law enforcement. The original detection didn't classify the "percussive noise" as a gunshot. This non-determination was manually overridden by a ShotSpotter "analyst" to be classified as a gunshot.

Months later, ShotSpotter relocated the detected noise from where it was originally "heard" to the intersection where the wrongfully-arrested man's car was captured by a nearby surveillance camera, allowing prosecutors to tie together their theory that the person they had already pinned the crime on had actually committed the crime. But, as soon as the wrongfully-arrested man challenged this evidence, prosecutors dropped the case, citing a lack of evidence.

This reporting on ShotSpotter's apparent alteration of reports to better fit law enforcement claims and theories angered ShotSpotter. The company issued an angry statement claiming Motherboard's article on its latest evidentiary… oddities… was bogus and possibly capable of "confusing" readers.

First, ShotSpotter forensic evidence is 100% reliable and based entirely on the facts and science. ShotSpotter has never altered the information in a court-admissible detailed forensic report based on fitting a police narrative.

Whether or not this statement is true is yet to be determined. Its follow-up assertions added more detail but really didn't do much to refute the claims made in the Motherboard article.

VICE’s article falsely alleged that in the Williams case in Chicago, Illinois, prosecutors withdrew ShotSpotter evidence because it would not meet scientific evidentiary standards due to having been altered. This is 100% false. In fact, no ShotSpotter evidence for this case was altered at any time. ShotSpotter forensic analysts evaluated the incident to create a court-admissible forensic report. Based on publicly available data and our understanding of this case, the prosecutor’s theory was that the defendant shot the victim in a car. ShotSpotter detected a gunshot in the area, and we have always publicly stated that ShotSpotter does not guarantee detection of gunshots that are in cars or inside buildings.

This assertion means nothing. The prosecutors relied on the ShotSpotter report -- the one containing a shot the company claims it can't reliably detect (and one that moved several blocks away from its originally-detected location as the case moved forward) -- right up until they couldn't. When the evidence was challenged, prosecutors walked away. That's not a vote of confidence for ShotSpotter's evidentiary value.

The Associated Press has taken a long, detailed look at this case and uncovered plenty of information that casts doubt on the company's assertions about its practices and the value of its sole product: sensors that detect gunshots and provide their locations to law enforcement. In the middle of it all is 65-year-old Michael Williams who spent 11 months in jail, arrested for a homicide he didn't commit. In fact, Williams drove the gunshot victim to the ER after he was shot by someone in an adjacent car.

The AP report says everything Motherboard said (and more) -- the same things ShotSpotter publicly denied in its angry statement.

An Associated Press investigation, based on a review of thousands of internal documents, emails, presentations and confidential contracts, along with interviews with dozens of public defenders in communities where ShotSpotter has been deployed, has identified a number of serious flaws in using ShotSpotter as evidentiary support for prosecutors.

AP’s investigation found the system can miss live gunfire right under its microphones, or misclassify the sounds of fireworks or cars backfiring as gunshots. Forensic reports prepared by ShotSpotter’s employees have been used in court to improperly claim that a defendant shot at police, or provide questionable counts of the number of shots allegedly fired by defendants. Judges in a number of cases have thrown out the evidence.

As for the claim the company never alters reports, especially at the request of law enforcement, the AP found that claim is complete bullshit.

ShotSpotter employees can, and often do, change the source of sounds picked up by its sensors after listening to audio recordings, introducing the possibility of human bias into the gunshot detection algorithm. Employees can and do modify the location or number of shots fired at the request of police, according to court records. And in the past, city dispatchers or police themselves could also make some of these changes.

The company remains unrepentant, even in the face of mounting evidence the tech is faulty and its detections can be set to manual override at the whim of "analysts" or law enforcement officers.

When pressed about potential errors from the company’s algorithm, ShotSpotter CEO Ralph Clark declined to discuss specifics about their use of artificial intelligence, saying it’s “not really relevant.”

Well, it was pretty "relevant" for Michael Williams, jailed for 11 months almost solely on the strength of a twice-altered ShotSpotter report.

As for the "analysts," there's a reason for the scare quotes. ShotSpotter -- in its defensive statement -- makes it appear it retains a host of skilled forensic technicians.

Our expert forensic analysts spend on average eight hours per incident to compile a court-admissible report using specialized tools that differ from those used for alerts. These reports are 100% exact on rounds fired, timing, sequence, and location of shots fired – something they can testify to in court under oath.

But ShotSpotter has made it clear it actually doesn't employ any forensic experts. It just has people it has hired as analysts who do not actually have the technical or scientific background to support the claims they make in court about the reliability of ShotSpotter evidence.

In court filings, ShotSpotter acknowledged: “Our experts are trained using a variety of ‘on the job’ training sessions, and transfer of knowledge from our scientists and other experienced employees. As such no official or formal training materials exist for our forensic experts.”

With experts like these, it's no wonder the court was given contradictory assertions by the company while aiding prosecutors in convicting a man for a crime he didn't commit. Prosecutors claimed Williams shot the victim in his own vehicle. ShotSpotter's own documentation and contract with the Chicago PD made it clear the tech could not reliably detect gunshots fired in vehicles or buildings. But when it came time to back the blue, ShotSpotter was there:

[T]he company declined to say at what point during Williams’ nearly yearlong incarceration it got in touch with prosecutors, or why it prepared a forensic report for a gunshot that allegedly was fired in Williams’ vehicle, given the fact that the system had trouble identifying gunshots in enclosed spaces. The report itself contained contradictory information suggesting the technology did, in fact, work inside cars. Clark, the company’s CEO, declined to comment on the case, but in a follow-up statement, the company equivocated, telling AP that under “certain conditions,” the system can actually pick up gunshots inside vehicles.

The "certain conditions" appear to be "whenever prosecutors need a little help."

ShotSpotter can keep being angry about how it's portrayed in the press. But it can't blame anyone else for its own missteps and failures. Not only does the company seem to overstate the reliability of its sensors, but admissions made in court make it clear the evidentiary value of its reports is, at best, extremely questionable.

Filed Under: evidence, police, shotspotter
Companies: shotspotter

Cities Looking To Dump ShotSpotter Since It's Barely More Useful Than Doing Nothing At All

from the drop-it-like-it's-shot dept

Tech that supposedly detects gunshots has been deployed in multiple cities across the nation with the intent of providing faster response times to possible violence and to give investigators heads up where illegal activity may have occurred. The tech has some pretty serious problems, though.

For one, it cannot reliably detect gunshots.

A 2013 investigation of the effectiveness of ShotSpotter in Newark, New Jersey revealed that from 2010 to 2013, the system’s sensors alerted police 3,632 times, but only led to 17 actual arrests. According to the investigation, 75% of the gunshot alerts were false alarms.

The AI can "hear" the percussive noise and attempt to determine whether or not it's an actual gunshot. It could be a backfire or fireworks or some other noise distinguishably louder than the ambient noise at the detector's location. Far too often (and far more often than ShotSpotter claims), the guesses are wrong.

Out of Fall River’s 51 ShotSpotter activations in 2017, 21 have been false alarms, a 41 percent error rate. The sensors often report loud noises such as car backfires and fireworks as gunshots.

Dupere said there have been another 15 ShotSpotter activations this year that police later determined were unfounded because the responding officers found no evidence of gunfire or any witnesses to corroborate that they had seen or heard gunshots.

More disturbingly, the determinations made by the software can apparently be overridden if investigators truly want a shot to be spotted by tech like that offered by ShotSpotter. ShotSpotter personnel have altered AI judgment calls to fit police narratives. Worse, they've also moved detected shots from one location to another to better fit law enforcement's theory about who was involved in a shooting and where it happened.

ShotSpotter is an investigative tool, but it's a particularly malleable one. Investigators and officers seem pleased ShotSpotter personnel are willing to alter records to better suit their theories and narratives. But that's completely at odds with the ideal of evidence introduced in criminal cases, which is supposed to be free of bias and deliberate manipulation. When (criminal) history can be rewritten on the fly, the facts of the case are no longer factual.

These problems are exaggerated when law enforcement assumes any noise originating from ShotSpotter detectors is a gunshot and sends officers in to harass the locals until something turns up. Like almost every other tech advancement deployed by cops (especially "predictive policing," but there are others), the deployed tech tends to serve pre-existing biases held for decades by the law enforcement community: that minorities are inherently more "dangerous" than the white folks that tend to be overrepresented in law enforcement agencies.

Cities and police departments are loath to disclose the locations of their ShotSpotter sensors, but through public records requests Motherboard also obtained years of data from Kansas City, Missouri; Cleveland, Ohio; and Atlanta, Georgia showing where ShotSpotter sensors generated alerts—a proxy for the general location of the sensors.

In all four cities, the data shows that the sensors are also placed almost exclusively in majority Black and brown neighborhoods, based on population data from the U.S. Census.

So, there's the bigotry angle. This is on top of the evidence-faking issues. And then there are the limitation of the tech, which makes it an unwise investment for cities that actually want to do something about violent crime, rather than just let cops engage in biased policing.

The San Diego City Council was scheduled to vote July 27 on a $1.15 million renewal of its ShotSpotter contract, but city officials withdrew the item from the agenda after dozens of residents submitted public comments opposing the contract.

“ShotSpotter is a perfect example of the patronizing approach to public safety in Black and brown neighborhoods—this idea that we know what’s best for Black and brown community members,” Khalid Alexander, president of Pillars of the Community, told Motherboard. Pillars is an organization that opposes the over-policing of Black and brown residents, and is based in San Diego’s District 4—the only area of the city where ShotSpotter is located.

“I haven’t heard one person in District 4 ... come out and say that the ShotSpotter technology is something needed to keep us safe,” Alexander said.

Unsurprisingly, ShotSpotter says the opposite:

In a statement emailed to Motherboard, ShotSpotter claimed that local communities have told them the opposite, and that it would be "false and misleading" to describe the activists as representing the larger population. "As the communities who suffer most from gun violence, they recognize and appreciate ShotSpotter’s role helping police departments combat gun violence,” the company wrote.

While there's certainly some confirmation bias in play here, it's safe to say it's more than just a land of contrasts out there. But the only party that really has any money at stake is ShotSpotter, and self-preservation is a far stronger motivator than simply feeling underserved by law enforcement agencies that have decided to outsource serving and protecting to passive sensors and artificial intelligence. Taxpayer frustration may touch on fiscal issues, but since it's an aggregate fund, unhappy citizens tend to be less motivated by their bottom lines than companies with multi-million dollar contracts on the line.

And maybe $1.15 million/year seems like a drop in the bucket when it comes to police department budgets. Residents are sick of ShotSpotter and its ability to send officers expecting violence flooding into communities with guns akimbo. In Chicago, the contract now in danger of going unrenewed is $33 million, which is a very tangible hit to the bottom line of a company that early last year recorded record quarterly revenue of a little over $10 million.

Why keep paying for something that isn't doing anything much more than allowing alarmists to claim that violence is on the rise and the only cure is more cop violence? San Diego might not be paying much for its limited rollout of ShotSpotter, but even its small sample set is a complete disappointment.

A San Diego Police Department spokesperson told Voice of San Diego that during the four years ShotSpotter had been in use (as of September 2020) officers had made only two arrests responding to an alert and only one of those was directly linked to the alert.

Meanwhile, 72 of the 584 ShotSpotter alerts during that time period were determined to be unfounded, “a whopping 25 times higher than the 0.5 percent false positive rate put forth by the company,” the Voice of San Diego reported, based on data provided by the city’s police department.

Thirty-six times more false positives than useful investigative leads. $4.60 million (over four years at the going rate) should buy more help than that. It is only two arrests better than having nothing at all, which is so far below rounding error as to be laughable.

Even if the tech improves as time goes on, we still need to ask why we even need it. So far, all it's done is make ShotSpotter more money and justify biased policing efforts by law enforcement agencies willing to make use of any new development that allows them to keep doing the things they've always done. Taxpayers shouldn't be asked to fund programs that set cops up for failure. And they certainly shouldn't be expected to pay for the dubious privilege of being subjected to a system more capable of altering criminal evidence than making neighborhoods safer.

Filed Under: police, predictive policing, shotspotter, technology
Companies: shotspotter

Man Shot By Cops Claims Shotspotter Found Phantom 'Gunshot' To Justify Officer's Deadly Force

from the so-are-cops-still-losing-the-tech-race-or-whatever? dept

A lawsuit originally filed early last year makes some very disturbing allegations about police officers and their relationship with their vendors. New York resident Silvon Simmons was shot three times by Rochester Police Officer Joseph Ferrigno. Simmons was unarmed, but was hit with three of the four bullets fired by Ferrigno as he ran way from the officer.

Shortly before being shot. Simmons had been engaged in "Minding Your Own Business," which can apparently be nearly-fatal. Returning from a trip to a convenience store shortly after 9 pm, Officer Ferrigno cut in front of him, hit Simmons with his spotlight, exited his car with his gun drawn, and opened fire when Simmons began running. According to Simmons' amendment complaint [PDF] filed in August, Ferrigno never stated he was police officer before opening fire. Simmons, blinded by the spotlight, was unsure who was shooting at him. Even if he had known it was cop, he still would have had no idea why he was being stopped, much less shot at.

The number of bullets fired matters, as Tracy Rosenburg of Oakland Privacy reports. Something seriously messed up happened after the shooting. A gun was found in the yard several houses away from where Simmons was stopped. Cops tried to tie this weapon to Simmons to justify Ferrigno's deadly force use, despite the gun being located in the opposite direction of Simmons' flight path.

Not that it would have mattered if it had been found in the same yard where Simmons lay "playing dead" in order to not get shot again by his unseen assailant.

A Ruger revolver was said to have been recovered at the site an hour or so later. The Ruger did not belong to Silvon Simmons. His fingerprints and DNA were not on the recovered gun. The only shell casings recovered at the site were the four bullets from the officer’s Glock. The Ruger had an empty magazine and it was not in the lockback position, indicating it had not been recently fired.

Simmons offered to give the officers even more evidence, but since it was exculpatory, they weren't interested.

Simmons requested his hands and clothing be checked for gunpowder residue. The request was denied. Simmons repeated the request in writing while intubated at the hospital and was told to stop writing questions.

Having reverse-engineered a narrative to support the shooting, cops set about charging Simmons with attempted aggravated murder, attempted aggravated assault on a police officer, and criminal weapon possession. Simmons spent more than a year in jail before being acquitted on all charges. The wounds he sustained are permanent.

The dark and disturbing wildcard in this lawsuit is a Shotspotter gunshot detection device. The sensor reported no shots that night. What it detected during the shooting of Simmons was determined by the device to be "helicopter noise." That alone is concerning, considering the main job of the Shotspotter is to spot gunshots.

But that all changed once a police officer got involved. The Shotspotter forensic report says the incident switched from "helicopter noise" to "multiple gunshots" at an officer's request. This officer requested something extra though: an additional gunshot to support the narrative used to charge Simmons with murder/assault and give Officer Ferrigno post-incident permission to shoot the man fleeing from him.

A Rochester police officer acknowledged at the criminal trial of Silvon Simmons that he left the shooting scene after midnight, returned to the fourth floor at the Central Investigations Division of the Rochester Police Department, logged onto a computer and opened a chat session with Shotspotter, where he provided the location, time, the number of possible shots and the caliber of the weapons allegedly fired. Officer Robert Wetzel also testified that Shotspotter responded to him that they found a fifth gunshot at his request.

Shotspotter's forensic analyst certified the report, testifying that five shots were fired. Shotspotter somehow managed to detect another gunshot hours after the incident, using only the guidance of an officer who desperately needed another shot on the record. The company's forensic expert basically admitted this much while testifying.

This conclusion was based solely upon information provided to Shotspotter by the Rochester Police Department.

There may have been a simple way to prove/disprove the existence of another gunshot -- one that can't be traced to the weapon found or have been observed by anyone but the officer who needed a justification for his deadly force deployment. Spotshotter retains recordings… usually. But somehow this essential recording capturing an officer-involved shooting vanished.

[Shotspotter forensic analyst Paul] Greene testified at the criminal trial that there was “no way to go and look at the original file that was recorded and there is no way to listen to all the audio from that day”. The reason there was no way to do this was that both Shotspotter and the Rochester Police Department had lost the audio recording from the night of April 1, 2016.

Whatever doesn't agree with the narrative had to go, and so Shotspotter's recording went. Shotspotter has customers to please and shareholders to earn profits for. If the customer isn't happy with the product, they're unlikely to keep buying more. If keeping a customer happy means deleting recordings and certifying altered forensic reports, so be it. Whatever it takes to ensure the revenue stream keeps flowing -- even if "whatever" means framing a gunshot victim for a crime he didn't commit.

Filed Under: fake gunshot, joseph ferrigno, rochester police, shotspotter, silvon simmons
Companies: shotspotter