Proprietary Algorithms Are Being Used To Enhance Criminal Sentences And Preventing Defendants From Challenging Them
from the like-the-Coke-formula,-but-for-years-of-someone's-life dept
When law enforcement agencies want to know what people are up to, they no longer have to send officers out to walk a beat. It can all be done in-house, using as many data points as can be collected without a warrant. Multiple companies offer "pre-crime" databases for determining criminal activity "hot spots," which allow officers to make foregone conclusions based on what someone might do, rather than what they've actually done.
Not that's it doing much good. For all the time, money, and effort being put into it, the databases seem to be of little utility.
Many law enforcement agencies use software to predict potential crime hot spots, and the police in Kansas City, Mo., and other places have used data to identify potential criminals and to try to intervene.
[...]
In Chicago, where there has been a sharp rise in violent crime this year, the police have used an algorithm to compile a list of people most likely to shoot or be shot. Over Memorial Day weekend, when 64 people were shot in Chicago, the police said 50 of the victims were on that list.
So much for "intervention." Having a list of people who have a higher risk of being shot doesn't mean much when all it's used for is confirming the database's hunches. However, these same databases are being put to use in a much more functional way: determining sentence lengths for the criminals who have been arrested.
When Eric L. Loomis was sentenced for eluding the police in La Crosse, Wis., the judge told him he presented a “high risk” to the community and handed down a six-year prison term.
The judge said he had arrived at his sentencing decision in part because of Mr. Loomis’s rating on the Compas assessment, a secret algorithm used in the Wisconsin justice system to calculate the likelihood that someone will commit another crime.
We're locking up more people for more years based on criminal activity they'll no longer have the option of possibly performing. This is nothing new. Sentencing enhancement is based on a lot of factors, not all of them confined to proprietary databases. But what is new are the algorithms used to determine these sentence enhancements, most of which belong to private companies who are completely uninterested in sharing this crucial part of the equation with the public.
In Mr. Loomis' case, the software determined he would be likely to engage in further criminal activity in the future. A so-called "Compas score" -- provided by Northpointe Inc. -- resulted in a six-year sentence for eluding an officer and operating a vehicle without the owner's consent. His lawyer is challenging this sentence enhancement and going after Northpointe, which refuses to release any information about how the Compas score is compiled.
What Northpointe has released are statements that confirm the code is proprietary and that the Compas score is "backed by research" -- although it is similarly unwilling to release this research.
The problem here isn't so much the use of algorithms to determine sentence lengths. After all, state and federal guidelines for sentence lengths are used all of the time during sentencing, which includes factors such as the likelihood of future criminal activity. But these guidelines can be viewed by the public and are much more easily challenged in court.
The use of private contractors to provide input on sentencing renders the process opaque. Defendants can't adequately challenge sentence enhancements without knowing the details of the "score" being presented by prosecutors to judges. The algorithms' inner workings should either be made available to defendants upon request, or the "score" should be determined solely by government agencies, where the data and determining factors can be inspected by the public.
We're now in the unfortunate situation where companies are telling judges how long someone should be locked up -- using data which itself might be highly questionable. The feeling seems to be that if enough data is gathered, good things will happen. But as we can see from Chicago's implementation of this technology, the only thing it's done so far is add confirmation bias toetags to the ever-increasing number of bodies in the city's morgues.
The use of locked-down, proprietary code in sentencing is more of the same. It undermines the government's assertion that prison sentences are a form of rehabilitation and replaces it with the promise that criminal defendants will "do the time" so they can't "do the crime" -- all the while preventing those affected from challenging this determination.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: compas assessment, crime, predictions, proprietary algorithms, repeat offenders, sentencing, thought crimes
Companies: compas
Reader Comments
Subscribe: RSS
View by: Time | Thread
Challenge-ability
[ link to this | view in chronology ]
It'll NEVER happen to you or anyone who love because you are "Good People"(tm) and bad people deserve it.
While not directly related there was a report by someone looking into one of the hot spot scoring things cops were using and while he wasn't a risk there was some history at the location where he lived, so that raised his risk assessment even though it happened long before he resided there.
Our Magic Tiger Repelling Rock really works, but we can't explain it... even if allowing its use undermines the basics tenants of justice.
[ link to this | view in chronology ]
Mmmmm, Probability!
Oooh! I started employing the sort of correlational, statistical models (ARIMA) that are useful for this kind of stuff about thirty years ago. Howzabout we apply it to the prediction of municipal and other jurisdictions most likely illegally to abuse the Constitutional rights of citizens up to and including murder?! Cool! Instant, new business model. Oh wait, all of them, never mind.
[ link to this | view in chronology ]
Re: Mmmmm, Probability!
Also, (almost?) zero paying customers sinks this as a business model.
[ link to this | view in chronology ]
Follow the money...
[ link to this | view in chronology ]
Re: Follow the money...
I mean, a realist.
[ link to this | view in chronology ]
Re: Follow the money...
[ link to this | view in chronology ]
Re: Re: Follow the money...
Patent maybe ... certainly trademark disputes, like for Panama Red, Acapulco Gold, Road Apple Red, etc
[ link to this | view in chronology ]
Capitalism
USA! USA! USA! Capitalism eff yeah! Solves all problems even those annoying thinky thingys. Am I right? *high five*
[ link to this | view in chronology ]
Re: Capitalism
[ link to this | view in chronology ]
Re: Re: Capitalism
[ link to this | view in chronology ]
Re: Re: Capitalism
That sounds like it has something to do with capitalism.
[ link to this | view in chronology ]
Re: Re: Re: Capitalism
http://rense.com/general37/fascism.htm
You are all describing Fascism, which is what happens when socialism swings to the far right.
[ link to this | view in chronology ]
Re: Re: Re: Re: Capitalism
[ link to this | view in chronology ]
Not proprietary
[ link to this | view in chronology ]
I don't mind how the algorithms work
[ link to this | view in chronology ]
Re: I don't mind how the algorithms work
[ link to this | view in chronology ]
Re: I don't mind how the algorithms work
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Self-fulfilling prophecy
[ link to this | view in chronology ]
Re: Self-fulfilling prophecy
[ link to this | view in chronology ]
Re: Self-fulfilling prophecy
[ link to this | view in chronology ]
isn't public court a civil right?
[ link to this | view in chronology ]
Police are going to love this.
Whatever happened to "do the crime, serve the time"? Why should criminals from a background not requiring breaking the law semi-regularly to survive get softer sentences?
[ link to this | view in chronology ]
Re: Police are going to love this.
[ link to this | view in chronology ]
Re: Re: Police are going to love this.
[ link to this | view in chronology ]
Re: Re: Re: Police are going to love this.
Minus three strikes, and you're in. Serve the time before the crime.
[ link to this | view in chronology ]
Re: Re: Re: Re: Police are going to love this.
[ link to this | view in chronology ]
Re: Re: Police are going to love this.
Considering how segregated most cities are, it probably doesn't matter much.
[ link to this | view in chronology ]
Northpointe is HEAVILY invested in private prisons and wants as many people as possible to stay as long as possible, since they work them as unpaid slave labor.
[ link to this | view in chronology ]
Re:
That would be unconscionable. It's more like "given current conviction rates and terms, we are likely to fall below 90% prison occupation at the time his usual prison term would end, so we better extend it. Guilty, guilty, guilty, guilty! Do we need more solitary confinement bookings right now?" You can't just make numbers up.
[ link to this | view in chronology ]
They literally have people physically typing stuff in, no background process or database software at all.
Total scam to increase population count at prisons they manage/own.
[ link to this | view in chronology ]
Wait, if they can identify factors that lead to a high risk of reoffending...
Let's assume these companies have developed genuine algorithms, that can assess all of the factors affecting a person, and determine if they will be a criminal or continue to be a criminal.
Let's assume that these algorithms, as a whole, tend towards high degrees of sensitivity and specificity. That they're good tests.
If these algorithms know what combination of factors will make a person commit crimes, then they know what factors can be changed so that person will no longer commit crimes.
To put it another way, the profiles they construct around convicts to determine if they're a risk to the community, can also be used in their rehabilitation.
*checks the article again*
Nope, I don't see any mention of these algorithms being used to recommend education and training, psychological counselling, employment assistance, community service, or any other form of rehabilitation that might ACTUALLY reduce the recidivism rate.
Just...longer sentences. It's like they WANT to keep people in jail and use them for cheap labour for the rest of their lives.
[ link to this | view in chronology ]
Re: Wait, if they can identify factors that lead to a high risk of reoffending...
They served the time for their chance at recidivism, now give them an honest and unbiased opportunity for earning the prison term they have already served in advance.
[ link to this | view in chronology ]
Re: Wait, if they can identify factors that lead to a high risk of reoffending...
When for profit prosons were first mooted in the UK the suggestion was made that they should be paid by results - ie some of the fees would be held back until the prisoner had been released and had not re-offended for a set period.
That would mean that the market mechanisms would be working in our favour and the kind of algoritms you describe would be worth investing in.
Crime, and the prison population, would fall.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Wow - these algorithms can accurately predict criminal activity, awesome - we're gonna be soooo rich!
... oh wait, but not OUR criminal activity - right??????
Right? ... hello?
[ link to this | view in chronology ]
I found the list they're using
[ link to this | view in chronology ]
life imitating fiction... it
[ link to this | view in chronology ]
life imitating fiction... in the worst way
Any other intervention that doesn't result in incarcerarion or death would be a welcome prospect.
[ link to this | view in chronology ]
Wondering
[ link to this | view in chronology ]
Re: Wondering
I mean, the average citizen (and thus jury member) is queasily ok with Guantanamo. Same idea here.
[ link to this | view in chronology ]
So the cops are using technology to improve things...
If the police are able to predict crime hot spots, perhaps they can work to improve policing in the area, or work to change behaviors of the local such that crime drops. That would be true crime prevention.
The deadpool style list is a little bit creepier, and a whole lot harder to turn into action. They can't add protection for every person who might be at risk. But knowing who might be killed might give them a better insight into what is going on in the darker circles of life.
I guess you guys would prefer that the cops spend their time in school studying to be to flight civil rights lawyers. Seems to be a job requirement these days!
[ link to this | view in chronology ]
Re: So the cops are using technology to improve things...
Then why not release the algorithm?
What's the big fucking secret?
[ link to this | view in chronology ]
Re: So the cops are using technology to improve things...
Due process is not an impediment to justice.
[ link to this | view in chronology ]
If you can't do the math in the algorithm, you shouldn't be using it
It is not reasonable to assume that the robed golems have the faintest clue how even one of the underlying subsystems work.
In my experience the aggregation of that much data almost always reveals practical solutions during the development cycle. If you understood the code, you'd be looking at managed approaches to social justice, not punitive approaches.
Selling fear and death is easy. Teaching somebody how to observe entropy through code is a wholely different problem indeed. Those doing the former, outnumber those able to do the latter by at least a thousand to one.
[ link to this | view in chronology ]
By the way...
Must be one of the most evil pieces of Newspeak I ever saw.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Did the defendant just have a court assigned attorney, that was already overworked?
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Perhaps these algorithms should be used in hiring police officers.
Or, y'know, determine who's likely to blow the whistle on police brutality.
[ link to this | view in chronology ]
Re: Perhaps these algorithms should be used in hiring police officers.
That minimizes both the risk to get some pansy unwilling to employ unnecessary force as well as some whistleblower.
[ link to this | view in chronology ]
Throw Book
Else
If Income = Low
Throw Book
Else
Set Free
End If
End If
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
How this works in practice:
Now the prospect is that the perpetrator considers herself mostly above the law, intends breaking it again and again, and aims for a power grab where she'll be able to duck most accountability
Those are rather dire prospects for rehabilitation, so it seems that a doubling of the sentencing is called for and she'll have to serve two terms rather than a single one. In the land of the free and the home of the brave.
[ link to this | view in chronology ]
Re: How this works in practice:
What if in reality she is? Let's say the organization investigating and prosecuting has foreseen two possible future outcomes. The first where they may be held more to account. The second where they have an long (one or two 4-year terms) ongoing "investigation" powerful enough to invert the power structure of the executive branch of the government placing them at the top.
[ link to this | view in chronology ]
full gestapo
secret court computer programs
secret no gun lists
Just wait until the cashless society is enforced: NO MONEY FOR YOU!
Generation Pod won't even blink.
[ link to this | view in chronology ]
Re: full gestapo
[ link to this | view in chronology ]
That's really scary to think about.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Does Northpointe own prisons?
[ link to this | view in chronology ]
[ link to this | view in chronology ]