DOJ Tells Forensic Experts To Stop Overstating The 'Scientific Certainty' Of Presented Evidence
from the directed-to-use-air-quotes-when-saying-'science'-or-'certainty' dept
The DOJ is finally addressing some long-ignored problems with the forensic evidence its prosecutors rely on. For two decades, FBI forensics experts handed out flawed testimony in hundreds of criminal cases, routinely overstating the certainty of conclusions reached by forensic examination. Of those cases, 28 ended in death penalty verdicts.
An earlier attempt to address issues with flawed science and flawed testimony swiftly ran aground. Federal judge Jed S. Rakoff very publicly resigned from a committee formed to examine these issues after he was informed by the attorney general's office that he wasn't actually supposed to be examining these issues.
Last evening, January 27, 2015, I was telephonically informed that the Deputy Attorney General of the U.S. Department of Justice has decided that the subject of pre-trial forensic discovery -- i.e., the extent to which information regarding forensic science experts and their data, opinions, methodologies, etc., should be disclosed before they testify in court -- is beyond the “scope” of the Commission’s business and therefore cannot properly be the subject of Commission reports or discussions in any respect.
[...]
Because I believe that this unilateral decision is a major mistake that is likely to significantly erode the effectiveness of the Commission -- and because I believe it reflects a determination by the Department of Justice to place strategic advantage over a search for the truth -- I have decided to resign from the Commission, effective immediately. I have never before felt the need to resign from any of the many committees on which I have served over the years; but given what I believe is the unsupportable position now taken by the Department of Justice, I feel I have no choice.
Caleb Mason of Brown, White & Osborn (the "White" is Popehat's Ken White) reports that the DOJ appears to be taking these problems more seriously. It has issued a directive [PDF] forbidding forensic experts from making claims about "scientific certainty" when presenting evidence.
Directive Number 1 provides that agencies must now “ensure that forensic examiners are not using the expressions ‘reasonable scientific certainty’ in their reports or testimony.” Yes, you read that right. The Department of Justice is telling its forensic expert witnesses to stop claiming “scientific certainty.” Why? Because for most forensic disciplines, there never was any, and DOJ is—after decades of resistance—admitting it.
One of the forms of evidence is fingerprints, the thing every law enforcement agency makes sure to obtain when booking suspects because it's supposedly so infallible. But like almost everything else law enforcement forensic experts claim are reasonably certain, scientifically-speaking, examination of prints no more guarantees a match than examining bite marks.
Fingerprint examiners look for “matching points” in prints, but believe it or not, there are no general standards for which points to look at, how many points to look at, or even what counts as a “point.” Not only are there no established standards, there isn’t even general agreement within the forensic analysis community. Some people like eight points, others ten, others twelve. Many examiners insist they can make an identification with just a single point.
Even more amazingly, in stark contrast to DNA matching, no one knows what the statistical likelihood is of two fingerprints sharing particular points, or whether that likelihood is different for different regions or features of the print. This is the crucial question for any identification methodology, because while each person’s fingerprints may be unique, the examiner doesn’t look at every molecule—the examiner looks at whatever five (or eight, or ten) “points” he or she chooses to look at.
Why is this process still so vague even after decades of reliance on it for identifying suspects? Well, it's because the DOJ won't allow anyone other than the government to take a look at the collected records. Researchers who may have been able to make better determinations on how many points are needed for more definitive matches (or how often false positives are returned by the database) have been locked out by the DOJ.
But the big fingerprint databases are controlled by DOJ, and DOJ has steadfastly refused to let researchers use them for the types of analyses geneticists do with DNA. That’s what makes print analysis so frustrating: the data exists, so fingerprint analysis could be a genuine scientific discipline, with publicly-available databases, peer-reviewed research, known error rates, and accepted methodologies. It could be a real body of knowledge about the differential rates of occurrence among populations of particular physical features of our fingerprints. Hopefully one day it will be. But it’s not now, as the DOJ directives finally acknowledge.
The DOJ's not offering to open up its fingerprint database for outside examination. But at least it's admitting it hasn't let anyone without a vested interest in successful prosecutions take a good look at the methods used by its forensic examiners or the collected evidence they're working with.
[As a bonus, here's another fantastic read by Caleb Mason: a Constitutional examination of Jay-Z's hit track, entitled "JAY-Z’S 99 PROBLEMS, VERSE 2: A CLOSE READING WITH FOURTH AMENDMENT GUIDANCE FOR COPS AND PERPS."]
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: doj, evidence, forensics, prosecutors, science
Reader Comments
Subscribe: RSS
View by: Time | Thread
Forensic Science
[ link to this | view in thread ]
So how many have been convicted by testimony that was professed to be of over whelming certainly that an accused was/is involved in a crime by way of scientific evidence? And now the science and testimony is in question.
What troubles me most is the government has people on a committee who deal with these issues but yet they didn't want the committee to look at said issues and ignore it , so they could still have the tactical advantage over an accused and their counsel.
Isn't part of the just systems balance and checks is that their is separation between the various parties involved and that disclosure and following the rules of law and the court not to mention sworn professional oaths and that little swearing in to tell the truth, the whole truth and nothing but the truth still apply these days?
If your an accused the deck seems to be getting higher and higher against you and the amount of collusion in seeking to make sure that prosecutions move along to convictions even if their has been hanky panky or questionable actions on part of the prosecution side of cases that judges seem to be allowing.
It seems as though impartiality is a thing of the past as is the ethics that govern all players in the justice system as it would seem more and more that prosecutors are being allowed a wide berth with disclosure and playing by the rules and that more and more courts are okaying that uneven playing field.
You are no longer innocent until proven guilty more you are now guilty till you can prove your innocence.
It really is a shame how the constitution and peoples rights have taken a back seat to a scorched earth campaign at convictions at any cost no matter what.
[ link to this | view in thread ]
Re: Forensic Science
[ link to this | view in thread ]
Re:
Bias in policing, plea bargaining, lack of public defender funding all benefits the private prison industry - who actively fights any sort of change to their profit stream.
[ link to this | view in thread ]
Re:
Even fingerprints have not passed the Daubert standard yet, though they are taken as gospel.
[ link to this | view in thread ]
Why aren't private entities or even public institutions stepping up to the plate, to do the research on their own?
The methodology and application shouldn't be that hard (especially if comparing to, oh, say particle physics), and I would think that entities like the Innocence Project would have at least some vested interest in the results of such research.
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
DNA
I've seen at least four cases in the news where DNA evidence turned out to be mistaken. Three from lab and evidence collection errors. And one where someone in a factory was packing cotton swabs by hand, his DNA then detected in multiple investigations, leading police in Europe to believe that they were dealing with a serial killer.
It's a good tool, but the real-world certainty isn't as good as the theoretical certainty.
[ link to this | view in thread ]
[ link to this | view in thread ]
doj is moving to foot stomps. you know, like a horse tells how many. even with the whinny at the end for added emphasis.
whereupon the prosecutor will yell, good boy, and throw the stooge an apple.
[ link to this | view in thread ]
[ link to this | view in thread ]
A few months ago I had to be fingerprinted as part of obtaining clearance for work. I went to a place that performed fingerprinting, which they did by pressing my fingers up against the glass surface of some sort of scanning machine, which fed the results into a computer.
They were supposed to run each finger twice to make sure they matched. The guy running it had to do each finger multiple times because he couldn't get matches, and several times he just overrode the requirement to get a second print after multiple failures.
This makes me think maybe I ought to go commit a crime, because my prints don't match themselves! :P
[ link to this | view in thread ]
Re: Re: Forensic Science
Professionals are just as biased, lazy, self serving, and vendetta driven as a shaky witness.
There are many stories where the "forensic" evidence was tampered, flawed, or just out right fucking FAKED!
[ link to this | view in thread ]
Can we get this standard used in climate science?
[ link to this | view in thread ]
Re:
Furthermore, innocent untill proven guilty would also entail a requirement to stay relatively agnostic on evidence for the prosecutor. Such a job is not on the prosecutor as much as the judge, meaning that the prosecutor is free to act in the interest of getting a conviction as long as the judge allows it. Since the prosecutor is in charge of the investigative ressources it sometimes leads to some ridiculously biasing interrogation techniques like those used on Aaron Swartz or worse.
Therefore it is more like guilty unless the evidence says otherwise or the prosecutor deems the political value of a conviction on the case too low.
[ link to this | view in thread ]
Re: Can we get this standard used in climate science?
Inevitably the claim gets thoroughly debunked. The climate model predictions and histories are shown to have been accurate. But by then the next such claim is making the rounds.
[ link to this | view in thread ]
DNA
The greatest source of error comes in collection, labeling, transport and storage of specimens. This may result from innocent error, ignorance, misfeasance or malfeasance. This is an issue in medical systems where people are well trained, and usually dedicated. I can not believe that police are as well trained, as dedicated to truth, or generally as intelligent as the medical world.
You can't trust the justice system because:
1) It has ridiculous laws.
2) It has little concept of science, how to interpret it, and what results mean.
3) Forensic work is subject to massive error and falsification.
4) Evidence gathering is subject to error and malicious behavior.
5) The defense is often not permitted to explain the existence of evidence, or to challenge it in a court of law.
6) First hand eyewitness evidence is well known to be often faulty.
7) Jurors are instructed not to think for themselves as to why a particular piece of evidence is likely to exist, but has not been presented.
8) For those who think we have a wonderful system, tell me why we have 25% of the worlds prisoners, but only 5% of the worlds population.
[ link to this | view in thread ]
Also, how big of a database would someone actually need to do this sort of research? Could they get volunteers?
[ link to this | view in thread ]
Re: Can we get this standard used in climate science?
Not because the climate models are even crudely accurate over a single decade, but because there are so many models that the sheer number is bound to produce a winner or two.
[ link to this | view in thread ]
Fingerprints or Chops
While the average fingerprint is 'quite rare', the quoted myth of 'there is only one per human on earth' is very-much a lie. There are several cases in the past where more than one person has had the same visual print, even in the same relatively-small town. While not all prints are 'so common', the truth is, SOME ARE. If 2-percent of prints are highly common (say, 1 occurrance in 10,000), while 90-percent are rather rare (say, 1 in 1,000,000), and the remaining 9-percent fall somewhere in-between, then in actuality what you have is no better than trying to identify a criminal based on his car's paint-job - heaven help you if you have that red-car or those common fingerprints.
This is the nasty, NASTY little secret that the DOJ doesn't want known about fingerprints, and why they will NEVER allow researchers to investigate their print databases.
[ link to this | view in thread ]
> what I believe is the unsupportable position now
> taken by the Department of Justice, I feel I have
> no choice.
Gee, thanks, Jed.
If you had any stones, you would have made them throw you out first, then issued a press release.
[ link to this | view in thread ]
Re: DNA
News of the Weird occasionally features these stories. Granted, some times it all turns out OK:
[ link to this | view in thread ]
Re: Re: Can we get this standard used in climate science?
Then there is the hole study of the effect of the CO2 on plants and what do you know, the trees, shrubs and more importantly crops will thrive under even greater concentrations of CO2 than we have today. And what does all the green stuff create? More O2.
[ link to this | view in thread ]
Re: Re: Re: Can we get this standard used in climate science?
> More than two dozen reconstructions, using various statistical methods and combinations of proxy records, have supported the broad consensus shown in the original 1998 hockey-stick graph, with variations in how flat the pre-20th century "shaft" appears. The 2007 IPCC Fourth Assessment Report cited 14 reconstructions, 10 of which covered 1,000 years or longer, to support its strengthened conclusion that it was likely that Northern Hemisphere temperatures during the 20th century were the highest in at least the past 1,300 years. Over a dozen subsequent reconstructions, including Mann et al. 2008 and PAGES 2k Consortium 2013, have supported these general conclusions.
Your "long pause in warming" is also thoroughly debunked.
[ link to this | view in thread ]
Re: Can we get this standard used in climate science?
Predictions are left up to weather science and meteorologists, and they always affix a wide margin of error to their predictions.
TL;DR: if you see a prediction without error bounding, it's not scientific.
[ link to this | view in thread ]
Re:
"If they've done nothing wrong, they've got nothing to hide."
/s
Of course, it's tricky when you're dealing with other people's data. Part of the problem with the DOJ database is that the contents aren't even standardized. Some of the samples are of a much higher quality than others, which means you can't even generate a reliable standard of error.
[ link to this | view in thread ]
REALLY
Is this an understatement?? its Just passed the 100 year mark in the USA..and in the world, abit longer..
Always wondered how many dots/dashes there were in a 3/4 inch area..
AND HOW you can match less then the WHOLE thing..
Then to decide the orientation..
its like looking at a CLOCK with no numbers..TURN IT ANY WAY YOU WANT..just line up the Hands for the time..
I love that TV. can take a print from ANY surface..
But its the easiest way to TELL if someone has been in a room..
[ link to this | view in thread ]
Re: Re: Re: Re: Can we get this standard used in climate science?
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Can we get this standard used in climate science?
The selected period starts immediately after 1998's temperature record (making the few years after seem lesser by comparison) and stops before new high temperature records. It's classic cherry picking, and STILL there was overall warming during the period.
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Can we get this standard used in climate science?
Also, you have yet to address the dire consequences we should be facing by now but aren't.
http://www.climate4you.com/images/VostokTemp0-420000%20BP.gif
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Can we get this standard used in climate science?
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Re: Can we get this standard used in climate science?
For "dire consequences" we're not talking about birds bursting into flames in mid-air.
We're talking about sea-level rise, which we know happened before and is happening again now. With a large fraction of the human population now living on the coasts, there's "dire consequences" that weren't there before. We're talking about a large reduction in the world's food supply, especially in places where it'll hurt the most.
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Re: Can we get this standard used in climate science?
And I'm a conservative on the right. I'm just not a Trump/Palin/Cruz follower like yourself.
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Re: DNA
Nah, better by far to just ignore any 'inconvenient' evidence and let the actual guilty person walk free, I mean really, what's a few wrongfully convicted/executed people compared to more paperwork and making someone in a position of authority look bad?
[ link to this | view in thread ]
Re: Re: Re: DNA
- a Texas primary voter commenting on a questionable execution approved by Gov. Rick Perry
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Re: Re: Can we get this standard used in climate science?
[ link to this | view in thread ]
Re: Re: Re: Re: DNA
Of course there is a possibility of cowardice. Being unwilling to go up against the judges, prosecutors and cops who couldn't wait to see this man murdered.
[ link to this | view in thread ]
Re: Forensic Science
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
Re: Re: Re: Can we get this standard used in climate science?
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Can we get this standard used in climate science?
[ link to this | view in thread ]
Re: Re: Re: Re: Re: DNA
[ link to this | view in thread ]
Re: Re: Re:
[ link to this | view in thread ]
The justice system doesn't even use forensics to detect culprits.
They certainly don't re-open the case and look for other suspects.
So why do we regard acquittals as false and evasion, but regard convictions as absolute, enough that we can doom them to a heinously cruel penal system?
We need to completely change how we investigate, and then refine our forensic science accordingly.
Pirates and emperors: Our state policies are more monstrous than the crimes of those trapped in the system.
And we need to change our penal system to include the presumption we probably got the wrong guy and he's there to be reformed and contained, but doesn't necessarily deserve punishment.
[ link to this | view in thread ]
re Uriel
[ link to this | view in thread ]
Re: The justice system doesn't even use forensics to detect culprits.
[ link to this | view in thread ]
Re: Re: The justice system doesn't even use forensics to detect culprits.
One incident that truly surprised me was a bone headed lie told by a judge in an attempt to intimidate a prospective juror during voir dire. I was quite shocked not because of the lie so much, but because it was such a stupid one. Anyone who had read their jury notice, saw the video presented to all jurors, or listened to the welcoming speech by one of the county supreme court judges knew that the judge had lied. Bad enough that he lied, but to make such an ass of himself in front of a 100+ people was hard to believe.
[ link to this | view in thread ]
Re: Re: Can we get this standard used in climate science?
Here's a link:
https://www.ipcc.ch/pdf/assessment-report/ar5/wg1/WG1AR5_Chapter12_FINAL.pdf
[ link to this | view in thread ]
Re: Re: Re: The justice system doesn't even use forensics to detect culprits.
And who exactly is going to call a judge out on a lie, baldfaced or not, in their own courtroom?
When everyone at your 'job' has to suck up to you then I imagine the ego gets just a wee bit swollen, and blatantly lying doesn't seem like much of a problem because even if everyone knows you lied none of them dare say anything about it.
[ link to this | view in thread ]
Re: Re: Re: Re: DNA
Even more worrying is given they were willing to say it how widespread is that murderous mindset in their area...
Yet another reason to stay far, far away from texas I guess.
[ link to this | view in thread ]
Re: Re: Re: Re: Re: DNA
[ link to this | view in thread ]
Re: re Uriel
One presumes they believed that the hanging of pickpockets acted as a deterrent, thereby keeping them safe from pickpockets.
Overly brutal punishments creates a "better to hang for a sheep than a lamb" mentality.
[ link to this | view in thread ]