State Supreme Court Says Secret Software Used In Sentencing Determinations Not A Violation Of Due Process Rights
from the not-as-long-as-it's-used-perfectly-within-an-impossible-set-of-confines dept
An algorithm is deciding certain criminal defendants should spend more time in prison. And that determination can't be fully challenged because the code belongs to a private company which provides the software to the government.
Eric Loomis was determined to be a "high risk" defendant, based on something called a "COMPAS score." COMPAS -- Criminal Offender Management Profiling for Alternative Sanctions -- cranks out Presentence Investigation Reports for use in the courtroom, utilizing a number of factors to generate a score that lets judges know how likely the defendant is to re-offend.
The problems with this system are numerous. For one, the code is proprietary, so defendants aren't allowed to examine the factors that lead to this determination, unlike other sentencing guidelines created by the government, which are open to the public to examine.
Another problem is that the algorithm engages in demographic profiling -- generally considered to be a bad thing when it comes to determining criminal behavior.
Back in May ProPublica published an investigation into the risk-assessment software that found that the algorithms were racially biased. ProPublica looked at the scores given to white people and black people and then whether the predictions were correct (by looking at whether they actually committed or didn’t commit crimes); they found that in Broward County, Florida, which was using software from a company called Northpointe, black people were mislabeled with high scores and that white people were more likely to be mislabeled with low scores.
"Fits the profile" is the new "fits the description" -- something that seems predisposed to putting blacks behind bars more frequently and for longer periods of time. Eric Loomis tried to challenge his COMPAS score but got nowhere with it, as the math behind it is locked up by Northpointe, which claims giving a defendant access to its trade secrets would pose a serious risk to its profitability.
Loomis argued that not giving him access posed a serious risk to his freedom. Allowing Northpointe to keep its algorithm secret was a violation of his due process rights, as it presented an unchallengeable score that could be used to keep him locked up longer than the normal range for the criminal activity he was convicted for.
His case went up the ladder to the Wisconsin Supreme Court, which has found [PDF] that defendants being unable to fully challenge a sentencing determination isn't a Constitutional problem.
Ultimately, we conclude that if used properly, observing the limitations and cautions set forth herein, a circuit court's consideration of a COMPAS risk assessment at sentencing does not violate a defendant's right to due process.
We determine that because the circuit court explained that its consideration of the COMPAS risk scores was supported by other independent factors, its use was not determinative in deciding whether Loomis could be supervised safely and effectively in the community. Therefore, the circuit court did not erroneously exercise its discretion. We further conclude that the circuit court's consideration of the read-in charges was not an erroneous exercise of discretion because it employed recognized legal standards.
Accordingly, we affirm the order of the circuit court denying Loomis's motion for post-conviction relief requesting a resentencing hearing.
The downside of this decision is that Northpointe cannot be forced to hand over its algorithm for examination by criminal defendants. The upside is that the court has issues with using COMPAS scores to determine sentence lengths.
[T]he opinion comes with some interesting caveats about things judges need to keep in mind when using risk scores in sentencing decisions: The two most important factors they’re asked to keep in mind is that software has been found to be racially biased and that the software needs to be constantly monitored and updated with new information. (If you’re relying on data from five or ten years ago, it’s not going to be accurate.)
The court also notes in passing that the software was never intended to be used to determine sentence lengths. It was supposed to used by the Department of Corrections to assess risks posed by parolees or those requesting parole. But it does not go so far as to forbid the use of COMPAS scores in sentencing decisions. Nor does it suggest that opening up the algorithm for examination might bring much-needed transparency to the sentencing process. Instead, the Supreme Court says judges must walk a very fine line when utilizing COMPAS scores.
The queasiness that judges feel about algorithmic risk-assesment is reflected in the concurring opinion filed by Justice Patience Drake Roggensack. “Reliance would violate due process protections,” she writes. “Accordingly, I write to clarify our holding in the majority opinion: consideration of COMPAS is permissible; reliance on COMPAS for the sentence imposed is not permissible.”
Unless a whole lot of judicial explanation accompanies every sentencing decision utilizing a COMPAS score, it's going to be almost impossible for defendants to tell whether a judge has just "considered" Northpointe's presentence investigation reports… or "relied" on them. Any sentence not hitting the upper end of the software's recommendations could be viewed as mere "consideration," even if "reliance" might be a more accurate term.
Without being allowed to closely examine COMPAS scores, defendants still aren't being given a chance to challenge any erroneous information that might be included in these reports. The court's reluctance to fully endorse the use of the software in sentencing decisions is a step forward, but it still allows judges to hand down sentences based on secret formulas that have already shown a predilection for recommending longer sentences to certain demographic groups.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: algorithms, compas, due process, sentencing, wisconsin
Reader Comments
Subscribe: RSS
View by: Time | Thread
I have to wonder about that particular point. Are they really implying that they believe human nature is going to change that much in 5-10 years?
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
I would think the punishment for a crime should not be a variable dependent upon social standing, or any other unrelated data. But we have seen relaxed sentencing simply because the perp is from a rich family, and then said perp skipped home curfew for a vacation in Mexico.
Apparently family income is not a good indicator of recidivism, and yet it is used to predict whether the perp will be a problem in the future. Go figure.
[ link to this | view in chronology ]
Re: Re: Re:
That speaks nothing to the fact that I don't think this software should be used at all by judges. It removes the reason for having human judges in the first place. That said, human judges have been wrong (see the insufficient sentencing of Brock Turner or Ethan Couch) and need better oversight and accountability apparently.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re: Re: Re:
Of course!
[ link to this | view in chronology ]
Re:
If, for example, one of the factors determining your "nature" is your neighborhood's criminal history then, yes, it most certainly can.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
I'd be perfectly happy leaving a lot of decisions in government up to a computer algorithm so long as that algorithm were fairly designed and we could examine it and see as much. It would be much preferable to semi-corrupt persons with their conflicts of interests and biases.
But this situation is disgusting. It's a clear violation of not being able to examine the evidence and testimony against you.
[ link to this | view in chronology ]
Re: Re:
While I agree with the general point of your post that this is a bad ruling and we should be able to inspect these algorithms, this is actually in a legal gray area. There is plenty of testimony during the sentencing phase that a defendant is not allowed to directly respond to, so it's not without precedent (using the term colloquially here, not legally).
[ link to this | view in chronology ]
Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re:
https://en.wikipedia.org/wiki/R_v_Sussex_Justices,_ex_p_McCarthy
Use of a secret computer program is in violation of that principle - and the linked case does actually look surpringly similar to the present one.
A quote from the decision on that case reads "Nothing is to be done which creates even a suspicion that there has been an improper interference with the course of justice. "
[ link to this | view in chronology ]
Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re: Re:
[ link to this | view in chronology ]
Re:
It has nothing whatsoever to do with sentencing. You cannot punish people for things they haven't done yet. Not if you pretend that this has anything to do with justice.
Secret sentencing criteria based on likelihood to reoffend will punish the same deeds harder for people who have a background making it harder to live honestly. So people who more deliberately commit crimes get let off the hook cheaper.
[ link to this | view in chronology ]
Re: Re:
It does seem to come straight out of a George Orwellian novel and is indeed, draconian at best..
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
terminator, here we come!!
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
You're under arrest because the software says you're a 21 year-old black man from a low-income neighborhood.
But I haven't done anything.
Not yet, but the software says you will, so we're locking you up before you get the chance to commit a crime.
[ link to this | view in chronology ]
Due process is supposed to be transparent.
Once again Florida doesn't just shamble but bounds its way down the road to tyranny.
Granted, the US is a shadow of the free nation it once was, but baby steps got us here. Baby steps.
[ link to this | view in chronology ]
Can't have it both ways
Conversely if the software is going to be used it absolutely should be open for examination and challenge by the defense, 'profits' be damned since we're talking about people's freedom here.
[ link to this | view in chronology ]
Predicting whether a person is likely to (re)commit violent crimes runs into a serious problem that pits political correctness vs. cold hard reality. Police statistics show a huge (& highly predictable) correlation between a person's race, sex, and age vs. his tendency to commit violent crimes.
If the goal is to protect individual liberty, then every person must be treated absolutely equally.
But if the goal is to protect society, then some rather Nazi-esque policies would be needed, such as "preemptively" jailing all inner-city black males between the ages of 15 and 30. There's no question that crime would drop enormously, since that tiny demographic is responsible for a statistically huge amount of crime. The question is whether the concepts of equal-rights and due-process that form the cornerstone of modern Western society would allow such a thing, no matter how scientifically justified.
[ link to this | view in chronology ]
Re:
See how much sense the Nazis made? Can we stop reviving their logic again and again and leave it in the graveyard of history?
Our justice system is supposed to punish people for what they did, not for who they are.
[ link to this | view in chronology ]
Police statistics...
Law enforcement has plenty of tools at their disposal by which to circumvent warrants to conduct unreasonable searches. Dog sniffs, for example can give a false positive up to 94% of the time. You just need the right dog.
We have more people in jail (per capita) than any other nation. We also have a solid likelihood that the majority of those people are falsely convicted. Plenty of them have major-crime sentences for petty crimes (e.g. possession).
And then there's the matter that police file reports on what they want to file, and will omit filing on what they don't care about.
So I wouldn't trust police statistics to be an indicator of how how civilians behave, so much as how police behave.
[ link to this | view in chronology ]
Re: Police statistics...
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
[ link to this | view in chronology ]
China approves. I wonder how long till they copy the process and actually manipulate the results.
[ link to this | view in chronology ]
Was Daubert decided in vain?
Since that is the case, shouldn't the defense and courts be able to make an examination, either under Daubert, under Rule 702, or as a matter of common law?
As it stands, I'm with Anonymous Coward. Use of the software without outside review is a clear and direct violation of the Sixth Amendment.
[ link to this | view in chronology ]
Sheesh - could the Court have punted any harder?
http://law2.umkc.edu/faculty/projects/ftrials/zenger/nullification.html
https://www.aclu.org/feature/ fair-sentencing-act
[ link to this | view in chronology ]
Re: Sheesh - could the Court have punted any harder?
In either case, Jury Nullification has no baring on judges using software to decide sentencing lengths. You can not nullify sentencing, you'd have to nullify the conviction that happens before the sentencing, and we are talking about a problem that occurs even for those who should be punished.
[ link to this | view in chronology ]
Jury nullification as a protest to the system.
Because Florida.
[ link to this | view in chronology ]
... so now we have
Sentencing by the Corporations. They do not even have to bribe legislators. wtf
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Something like that could be hugely useful if done correctly. Not so much for sentencing unless except for mitigation. But "correctly" is something we fail to do quite often. (And then hide it, deny it, and make money from it. And further use it inappropriately on top of that.)
[ link to this | view in chronology ]
What we really need
[ link to this | view in chronology ]
Go ask one of your KFC professors of health policy. On second thought, don't.
[ link to this | view in chronology ]
Re:
http://reclaimdemocracy.org/powell_memo_lewis
Powell just wanted to entrench capitalism as the dominant ideology but right-wing activists took the memo and ran with it in directions he never envisaged. Result: activist judges, etc., pushing a fascist line in the courts and neoliberalism presented as a middle-ground position. I never realised how all-pervasive the infiltration was until recently. It's all down to Powell, people.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Disproportionate punishment
Mandatory minimums have done nothing but make for disproportionate punishments.
Of course the rule against cruel punishment should outlaw all the state and federal penal systems in the US. I'm not sure why it doesn't. Not enough abuse?
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Even if you can inspect the algorithm, the one your looking at isn't neccessarily the one used.
Not to mention the fat finger in the source data that seems to always show up after the warden gets a call from the governor, right?
So it isn't just validation of the algorithm, but validation of the input data, which is I can almost guarantee will be badly tainted.
There will be socioeconomic factors that are reflected in the data that are not within the scope of the defendants control. That by itself, means the court is distinguishing formally based on factors that could be regarded as social class.
But I think that isn't the point.
My guess is that COMPAS is mostly about externalizing liability. What they are doing is slopping a bunch of tech around, so that if they get called into court they can say "buh, buh, buh da COMPUTER said so!". Which would be why it is important to insure that nobody knows what the computer is doing.
It is bad science. Reminiscent of eugenics in the early 1930s. They would do themselves a big favor by taking a hard look at this before it snowballs.
[ link to this | view in chronology ]
The SCIENCE of eugenics worked.
But yes, if when they create a black box and give that box authority, it gives the appearance of displacing liability.
The problem is, of course, that people condemned by the black box are not going to accept that it is to blame, so much as the institution --and specific officer-- that pushed it to the black box.
After the black box has incriminated some thousand or so people, I wonder if Florida judges will develop an attrition problem.
[ link to this | view in chronology ]
kangaroo court
[ link to this | view in chronology ]
Court Packing
[ link to this | view in chronology ]
[ link to this | view in chronology ]
The real culprit stand
Somehow, I still blame the bad guy committing multiple crimes knowingly who attempt to finagle the system who make it bad for everyone else when it might be their turn to seek justice.
[ link to this | view in chronology ]
[ link to this | view in chronology ]