Scientists Looking To Fix The Many Problems With Forensic Evidence
from the can-you-fix-the-people-performing-the-tests? dept
Everything everyone saw in cop shows as evidence linking people to crimes -- the hair left on someone's clothing, the tire tracks leading out to the road, the shell casings at the scene, etc. -- is all proving to be about as factual as the shows themselves.
While much of it is not exactly junk science, much of it has limited worth. What appears to indicate guilt contains enough of a margin of error that it could very easily prove otherwise. Science Magazine is taking a look at the standbys of forensic science and what's being done to ensure better presentations of evidence in the future.
On a September afternoon in 2000, a man named Richard Green was shot and wounded in his neighborhood south of Boston. About a year later, police found a loaded pistol in the yard of a nearby house. A detective with the Boston Police Department fired the gun multiple times in a lab and compared the minute grooves and scratches that the firing pin and the interior of the gun left on its cartridge casings with those discovered on casings found at the crime scene. They matched, he would later say at a pretrial hearing, “to the exclusion of every other firearm in the world.”Six years ago, the National Academy of Sciences found that these forensic standbys had a much larger margin of error than was portrayed in court by detectives and expert witnesses. It recommended the margin of error be delivered along with the testimony to head off future verdicts based on faulty evidence.
[...]
So how could the detective be sure that the shots hadn’t been fired from another gun?
The short answer, if you ask any statistician, is that he couldn’t. There was some unknown chance that a different gun struck a similar pattern. But for decades, forensic examiners have sometimes claimed in court that close but not identical ballistic markings could conclusively link evidence to a suspect—and judges and juries have trusted their expertise. Examiners have made similar statements for other forms of so-called pattern evidence, such as fingerprints, shoeprints, tire tracks, and bite marks.
To date, not much has changed. While actual junk science like bite marks has largely been discarded by prosecutors, the others remain, even as their reliability has been constantly questioned. The FBI loved hair analysis, right up to the point that it determined its witnesses had overstated test results 90% of the time in the two decades prior to 2000.
Even fingerprints, which have long been considered unassailable because of their supposed uniqueness, aren't much better. Some of it has to do with the presumption that every fingerprint is so unique even a partial print can eliminate suspects. The rest of its issues lie with those matching the prints.
One study of 169 fingerprint examiners found 7.5% false negatives—in which examiners concluded that two prints from the same person came from different people—and 0.1% false positives, where two prints were incorrectly said to be from the same source. When some of the examiners were retested on some of the same prints after 7 months, they repeated only about 90% of their exclusions and 89% of their individualizations.The NIST has given $20 million to the Center for Statistics and Applications in Forensic Evidence (CSAFE) to come up with a better way to present this sort of evidence -- one that clearly accounts for any uncertainties in the results or processes. CSAFE is still trying to figure out how to present this as a number/rating. But that might not be the only problem. The other issue is that juries and judges may not find specifics about forensic reliability to play much of a part in deciding guilt or innocence.
In a 2013 study, for instance, online participants had to rate the likelihood of a defendant’s guilt in a hypothetical robbery based on different kinds of testimony from a fingerprint examiner. It didn’t seem to matter whether they were simply told that a print at the scene “matched” or was “individualized” to the defendant, or whether the examiner offered further justification—the chance of an error is “so remote that it is considered to be a practical impossibility,” for example. In all those cases, jurors rated the likelihood of guilt at about 4.5 on a 7-point scale. “As a lawyer, I would have thought the specific wording would have mattered more than it did,” Garrett says. But if subjects were told that the print could have come from someone else, they seemed to discount the fingerprint evidence altogether.The other part of the problem is the people who perform the tests. Multiple incidents where evidence was falsified or not properly tested have been uncovered. The evidence is only as good as the processes, and if steps are skipped because of sloppiness or laziness, the evidence's credibility becomes highly questionable -- not just for the specific instance where results were faked, but for every test this person has touched.
There's no possible way to eliminate honest errors, much less prevent anyone from falsifying results. In both cases, the problems are caught after the damage has been done. Humans are the most unpredictable part of the chain of evidence but also an irreplaceable part. CSAFE will be working with forensics labs to create best practices, but it can do nothing to prevent the lazy and/or incompetent from completely ignoring the proper steps.
Problems are also present higher up the chain. When bad science or bad practices result in questionable evidence, it's often extremely difficult to have convictions resulting from them overturned.
What’s troubling, [federal judge Nancy] Gertner says, is that when judges accept junk science, an appeals court rarely overrules them. Attaching a numerical probability to evidence, as CSAFE hopes to do, “would certainly be interesting,” she says. But even a standard practice of critically evaluating evidence would be a step forward. “The pattern now is that the judges who care about these issues are enforcing them, and the judges who don’t care about these issues are not.”In this way, the courts are no better than labs where shoddy work is done. Variations in personality undermine the dispassionate nature of science, making it susceptible to human prejudices rather than the strength of the evidence itself.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: evidence, fbi, forensic evidence, forensics
Reader Comments
Subscribe: RSS
View by: Time | Thread
Nothing changes because the system would have to change, and that is not going to happen.
[ link to this | view in thread ]
Goals
The goal is to obtain convictions.
The truth might make that too difficult.
[ link to this | view in thread ]
Science... Good for only 1 direction
We only test a subset of DNA, which means you can get far more false positives than even close to comfortable when you look at the numbers. Additionally with all of the DNA databases now being referenced for criminal searches the police will only see more potential suspects as we move forward. Combine that with the Guilty until proven innocence and ignorant as fuck on science American people and you have a recipe for innocent people going to jail closing the case while the actual criminals walk free!
[ link to this | view in thread ]
Re: Goals
you would not be in court if you were not a dirt bag. America has a fetish for putting innocent people in jail... I mean you have to be guilty for something anyways right?
[ link to this | view in thread ]
Re: Science... Good for only 1 direction
These states likely really need to feed more serfs/poor people into the system to meet the contract terms.
The US Gulag - (War on Every Damn Thing) justice system!
Billions for Pretend investigation's, "Faux forensics" and guaranteed funds to private contractors.
Nowadays mostly the result of of dark money (and future employment promises) fed through the right State/Fed Lobbyists (Definition: purveyor of funds/jobs to corrupt Politicians).
[ link to this | view in thread ]
Hard vs cakewalk
Forensic scientists can debate until they're blue, but nothing is changing until their testimony starts sinking cases. And since forensic scientists are "free" for testimony at $400/hour or so, guess how much testimony they'll be giving for average man.
[ link to this | view in thread ]
Re: Re: Science... Good for only 1 direction
[ link to this | view in thread ]
I have some horror stories...
In lab-based biology classes they had absolutely no qualms "cooking the books" to get a desired data outcome, rather than explain why their experiment didn't work in a scientific manner. Because of that, I refused to work with them on any group projects. And as professionals now, I have little doubt that they are still fudging numbers.
So, think about the issues the FBI has had with this problem and realize the willingness to change data starts early in the careers of these individuals. They somehow manage to not get caught, or professors/instructors don't give a damn.
So forensics has the problem of not only 1.) scientific validity, but 2.) highly unethical scientists willing to manipulate data.
[ link to this | view in thread ]
Re: Re: Goals
Bankers are always innocent even when overwhelming evidence says otherwise.
[ link to this | view in thread ]
Are grammar police unique?*
* http://goo.gl/yMp5eB
Despite my griping, I enjoyed the article. What disturbs me far more than pseudo-scientific bias is the jury selection process itself. Smart, conscientious jurors would help mitigate a lot of these problems. However, in at least one U.S. state jury members are practically guaranteed to be either liars or useless in their role as mediators of justice by one qualifying question along the lines of, "Are you willing to set aside conscience and your own knowledge and decide this case only according to the law and the instructions of the judge?" If laws are always good and judges are always right, why did we need juries in the first place?
[ link to this | view in thread ]
That's our perp right there, plain as day.
These cheap ass video cams with low resolution are no match for the forensic wizards at CSI.
[ link to this | view in thread ]
[ link to this | view in thread ]
[ link to this | view in thread ]
Instead of the lawyers presenting to the judge, they present to the jury (I think it's supposed to be that way anyway - hence, e.g., jury consultants), but let's re-emphasize that point.
Then, during the course of the case, as each witness is done testifying for the lawyers, the jury are given an opportunity to present matters on which they want amplification or further information. Basically, give the jury the opportunity to explore the evidence in their own way, and in their own time.
As to the integrity of the forensic labs, maybe require the prosecution to present the full chain of custody, for examination by the defense, at least.
[ link to this | view in thread ]
Fingerprints
With fingerprints, you have to find so many points of commonality for a match. (I forget the exact numbers but let's say 9 for a partial and 15 for an exact match.) One issue is these numbers are probably too low. Also when these techniques were automated, different programs use different algorithms to find what is common and they produce different results. (This is where you get too many false negatives and a few false positives. Double checking by hand should eliminate false positives, but sometimes this is NOT done as the prosecutor wants evidence he can use. Even so, based on numbers in article, odds are only 1 in 1000 cases would have a false positive, and the defense does have access to the fingerprint to manually double check.)
[ link to this | view in thread ]
Re: Goals
Back to the issue in the post, part of the problem is crime scene evidence is often poor quality (smudged fingerprints, partial prints, etc.) that make accurate identification very difficult. Couple this with very sloppy evidence handling procedures (OJ trial when the detective kept the samples in his coat pocket) and there is no valid evidence.
[ link to this | view in thread ]
Re: Science... Good for only 1 direction
Also, DNA testing has improved tremendously since it was first introduced to the point it is one the best methods to positively identify a person (except for identical twins) available. Many cold cases have been solved because of DNA evidence.
The point of the scientific evidence (all evidence actually) is not that each bit proves absolutely the guilt of the person but that there is enough evidence that beyond a reasonable doubt the person is guilty. If you apply probability, if each bit is 95% certain the more bits will push the overall certainty much higher.
[ link to this | view in thread ]
Re: Fingerprints
[ link to this | view in thread ]
Re: Re: Science... Good for only 1 direction
Tell that to all the people wrongly convicted.
[ link to this | view in thread ]
Re: Re: Fingerprints
Only if the judge allows it.
[ link to this | view in thread ]
Bankers
The law is for those with money.
[ link to this | view in thread ]
No interest in seeking justice.
This has to change before we consider forensic evidence. Otherwise they can just get a company to build a bad-guy detector which sounds an alarm when you point it at a suspect. Currently, like our detector dogs, that is the height of where forensic science can go.
Right now we're in a state where forensic labs are not impartial, and will skew their results to implicate, even to the point of committing open fraud.
In this state, the only purpose a forensic lab can serve is the same as detector dogs, to implicate the innocent and force a conviction.
[ link to this | view in thread ]
Re: Bankers
Funding of the Public Defenders office is a joke.
[ link to this | view in thread ]
Re: Are grammar police unique?*
The earliest meanings of unique when it entered English around the beginning of the 17th century were “single, sole” and “having no equal.” By the mid-19th century unique had developed a wider meaning, “not typical, unusual,” and it is in this wider sense that it is compared: The foliage on the late-blooming plants is more unique than that on the earlier varieties. The comparison of so-called absolutes in senses that are not absolute is standard in all varieties of speech and writing.
[ link to this | view in thread ]
Re: Re: Are grammar police unique?*
yah, it changes everytime i uses it. i aint never wrong, just fresh.
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Re: Bankers
A system of laws that requires lawyers to effectively navigate them is of no service to the people.
Lawyers should be outlawed and Judges should be required to participate in continuing education just like medical professionals!
[ link to this | view in thread ]
Re: Re: Science... Good for only 1 direction
And while the detectives know that limit juries often do not. You will almost never find any jury ever suspect of evidence, far to many of them treat it like evidence is some gospel truth for just exactly the reasons you specified.
It is far to easy to admit a stack questionable evidence that makes someone look damn guilty despite a lot of that evidence being circumstantial and ultimately inconclusive.
A good example is someone not being anywhere near a crime scene but having been accused of lying to the police and going to jail on that alone. Even when they were not lying at all! All it takes is for one officer to lie about you and even worse if they can get a witness to say that they "think" they saw you somewhere else calling your alibi into question and you can go to jail by a just in case you ARE a criminal jury.
We need to seriously stop underestimated the amount of fucks people really do not give. When was the last time you went out and tried to stop someone from being screwed by he government? Now contrast that to all of the specially selected jury's that are lied to that only want to get back to their lives and you can begin to see how the system has been slowly engineered to produce successful criminal prosecutions! Even when evidence is complete bunk.
They just throw a low of evidence at the jury hoping something sticks to overwhelm them and making them think... there has to be no way for this person to be innocent... look at all of that evidence right?
You should read up on the innocence project and you will quickly see how wrong you are.
In all aspects of law, do not discount the corruption factor multiplied by the worthless and ignorance citizen factor, its a huge factor and often the deciding factor... to hell with any meaningful evidence!
[ link to this | view in thread ]
Re: I have some horror stories...
Yep... and this is just an example of an intentionally corrupt tech.
You can bet all sorts of corners are being cut to save costs even by well meaning people.
[ link to this | view in thread ]
Re: Re: Re: Are grammar police unique?*
For example, the word literally.
I smirk every time someone uses "literally" in a sentence where they really mean "figuratively".
That plane was so low that I literally could reach out and touch it.
I would like to see that video.
[ link to this | view in thread ]
Literally offensive
I still cringe at it since literally is the last bastion for non-metaphorisms. So that when I say Game of Thrones (or more properly A Song of Ice And Fire) is a literal epic, that's not saying it's really awesome (though it's pretty durned good) but also fits the properties of the epic genre classification.
[ link to this | view in thread ]