Scientists Looking To Fix The Many Problems With Forensic Evidence

from the can-you-fix-the-people-performing-the-tests? dept

Everything everyone saw in cop shows as evidence linking people to crimes -- the hair left on someone's clothing, the tire tracks leading out to the road, the shell casings at the scene, etc. -- is all proving to be about as factual as the shows themselves.

While much of it is not exactly junk science, much of it has limited worth. What appears to indicate guilt contains enough of a margin of error that it could very easily prove otherwise. Science Magazine is taking a look at the standbys of forensic science and what's being done to ensure better presentations of evidence in the future.

On a September afternoon in 2000, a man named Richard Green was shot and wounded in his neighborhood south of Boston. About a year later, police found a loaded pistol in the yard of a nearby house. A detective with the Boston Police Department fired the gun multiple times in a lab and compared the minute grooves and scratches that the firing pin and the interior of the gun left on its cartridge casings with those discovered on casings found at the crime scene. They matched, he would later say at a pretrial hearing, “to the exclusion of every other firearm in the world.”

[...]

So how could the detective be sure that the shots hadn’t been fired from another gun?

The short answer, if you ask any statistician, is that he couldn’t. There was some unknown chance that a different gun struck a similar pattern. But for decades, forensic examiners have sometimes claimed in court that close but not identical ballistic markings could conclusively link evidence to a suspect—and judges and juries have trusted their expertise. Examiners have made similar statements for other forms of so-called pattern evidence, such as fingerprints, shoeprints, tire tracks, and bite marks.

Six years ago, the National Academy of Sciences found that these forensic standbys had a much larger margin of error than was portrayed in court by detectives and expert witnesses. It recommended the margin of error be delivered along with the testimony to head off future verdicts based on faulty evidence.

To date, not much has changed. While actual junk science like bite marks has largely been discarded by prosecutors, the others remain, even as their reliability has been constantly questioned. The FBI loved hair analysis, right up to the point that it determined its witnesses had overstated test results 90% of the time in the two decades prior to 2000.

Even fingerprints, which have long been considered unassailable because of their supposed uniqueness, aren't much better. Some of it has to do with the presumption that every fingerprint is so unique even a partial print can eliminate suspects. The rest of its issues lie with those matching the prints.

One study of 169 fingerprint examiners found 7.5% false negatives—in which examiners concluded that two prints from the same person came from different people—and 0.1% false positives, where two prints were incorrectly said to be from the same source. When some of the examiners were retested on some of the same prints after 7 months, they repeated only about 90% of their exclusions and 89% of their individualizations.

The NIST has given $20 million to the Center for Statistics and Applications in Forensic Evidence (CSAFE) to come up with a better way to present this sort of evidence -- one that clearly accounts for any uncertainties in the results or processes. CSAFE is still trying to figure out how to present this as a number/rating. But that might not be the only problem. The other issue is that juries and judges may not find specifics about forensic reliability to play much of a part in deciding guilt or innocence.

In a 2013 study, for instance, online participants had to rate the likelihood of a defendant’s guilt in a hypothetical robbery based on different kinds of testimony from a fingerprint examiner. It didn’t seem to matter whether they were simply told that a print at the scene “matched” or was “individualized” to the defendant, or whether the examiner offered further justification—the chance of an error is “so remote that it is considered to be a practical impossibility,” for example. In all those cases, jurors rated the likelihood of guilt at about 4.5 on a 7-point scale. “As a lawyer, I would have thought the specific wording would have mattered more than it did,” Garrett says. But if subjects were told that the print could have come from someone else, they seemed to discount the fingerprint evidence altogether.

The other part of the problem is the people who perform the tests. Multiple incidents where evidence was falsified or not properly tested have been uncovered. The evidence is only as good as the processes, and if steps are skipped because of sloppiness or laziness, the evidence's credibility becomes highly questionable -- not just for the specific instance where results were faked, but for every test this person has touched.

There's no possible way to eliminate honest errors, much less prevent anyone from falsifying results. In both cases, the problems are caught after the damage has been done. Humans are the most unpredictable part of the chain of evidence but also an irreplaceable part. CSAFE will be working with forensics labs to create best practices, but it can do nothing to prevent the lazy and/or incompetent from completely ignoring the proper steps.

Problems are also present higher up the chain. When bad science or bad practices result in questionable evidence, it's often extremely difficult to have convictions resulting from them overturned.

What’s troubling, [federal judge Nancy] Gertner says, is that when judges accept junk science, an appeals court rarely overrules them. Attaching a numerical probability to evidence, as CSAFE hopes to do, “would certainly be interesting,” she says. But even a standard practice of critically evaluating evidence would be a step forward. “The pattern now is that the judges who care about these issues are enforcing them, and the judges who don’t care about these issues are not.”

In this way, the courts are no better than labs where shoddy work is done. Variations in personality undermine the dispassionate nature of science, making it susceptible to human prejudices rather than the strength of the evidence itself.

Filed Under: evidence, fbi, forensic evidence, forensics

Report: 'Nearly Every' FBI Forensics Expert Gave Flawed Testimony In 'Almost All Trials' Over A 20-Year Period

from the when-convictions-are-prized-over-accuracy dept

The FBI seems to be more interested in securing convictions than finding the truth. An investigation into questions about the agency's hair analysis commenced in 1996, but years of foot dragging by the FBI means the full truth has only come to light over the past couple of years. What's detailed in a report compiled by the National Association of Criminal Defense Lawyers and The Innocence Project is an almost surreally callous drive for sucessful prosecutions that potentially put dozens of innocent people behind bars.

The Justice Department and FBI have formally acknowledged that nearly every examiner in an elite FBI forensic unit gave flawed testimony in almost all trials in which they offered evidence against criminal defendants over more than a two-decade period before 2000.

Of 28 examiners with the FBI Laboratory’s microscopic hair comparison unit, 26 overstated forensic matches in ways that favored prosecutors in more than 95 percent of the 268 trials reviewed so far…

This isn't to say that nearly every case reviewed will be overturned. For some, it's too late. Of those reviewed, 28 pertain to prisoners with death sentences -- of which nearly all have already died... or been put to death. For others, their convictions may not have hinged on apparently questionable DNA evidence.

What's uncovered here is just the beginning. There are nearly 1,200 more cases to review. For many of those, it will likely be several years (and several lawsuits) before the truth comes out. Of those 1,200, nearly 700 are being met with stonewalling by local law enforcement and prosecutors, who have refused to provide requested transcripts and other court materials.

Part of this widespread failure is undoubtedly due to the FBI's desire to rack up convictions. (The same is true for those entities it worked with -- local law enforcement agencies and prosecutors.) But a larger portion of this can be chalked up to the FBI's own desire to keep its "slam dunk" forensics analysis from being questioned by anyone inside or outside of its labs. For four decades (1972-2012), the FBI refused to provide any guidelines for the use of, not just hair DNA evidence, but almost any forensic evidence, in court.

The FBI is waiting to complete all reviews to assess causes but has acknowledged that hair examiners until 2012 lacked written standards defining scientifically appropriate and erroneous ways to explain results in court. The bureau expects this year to complete similar standards for testimony and lab reports for 19 forensic disciplines.

Judging from how analysts performed in court, the lack of guidance was apparently construed to mean "put people in jail," rather than unbiased scientific analysis. This news follows on the heels of a highly critical report condemning the agency's faith in "bite mark analysis," a practice that is increasingly being perceived as junk science -- foisted on law enforcement by self-described "experts" with no hard data to back up their findings. The courts, so far, have often indulged bite mark experts, despite a National Academy of Sciences report finding that bite mark analysis provides "no evidence of an existing scientific basis for identifying an individual to the exclusion of all others."

These are people who have the power to effectively end someone's life and they've been instrumental in ensuring that problems tracing back to the early 1970s -- and first examined 25 years later -- remained buried until it could no longer be ignored.

Filed Under: convictions, doj, fbi, forensic evidence, testimony

Judge Resigns From Forensic Science Committee, Calls Out DOJ's 'Trial By Ambush' Tactics

from the you're-free-to-'discover'-this-evidence-AFTER-it's-been-used-to dept

Law enforcement agencies -- including the FBI -- haven't put together the best track record when it comes to forensic evidence. The issues range from fraud to the deployment of junk science. This has led to several wrongful imprisonments. This has also led to the government seeking to address these problems in a very "government" sort of way -- by convening a committee to study the issue, headed by the DOJ.

Issues were found, but the DOJ doesn't want to talk about them. The problems it doesn't want to talk about were a fundamental part of the committee's purpose.

The panel was created to improve the overall reliability of forensic evidence after instances of shoddy scientific analysis by federal, state and local police labs helped convict suspects.

Were. But no longer. This change is what prompted the only federal judge (Judge Jed S. Rakoff) on the committee to resign (after learning of it via an evening phone call from the Deputy Attorney General). From his letter of resignation:

Last evening, January 27, 2015, I was telephonically informed that the Deputy Attorney General of the U.S. Department of Justice has decided that the subject of pre-trial forensic discovery -- i.e., the extent to which information regarding forensic science experts and their data, opinions, methodologies, etc., should be disclosed before they testify in court -- is beyond the “scope” of the Commission’s business and therefore cannot properly be the subject of Commission reports or discussions in any respect.

Which is contrary to the committee's goals. To fix broken forensics, you need to fix the discovery aspects. The government would rather keep both its bad forensics and broken discovery procedures because it gives it an advantage when it comes to prosecution.

Because I believe that this unilateral decision is a major mistake that is likely to significantly erode the effectiveness of the Commission -- and because I believe it reflects a determination by the Department of Justice to place strategic advantage over a search for the truth -- I have decided to resign from the Commission, effective immediately. I have never before felt the need to resign from any of the many committees on which I have served over the years; but given what I believe is the unsupportable position now taken by the Department of Justice, I feel I have no choice.

Rakoff points out elsewhere in his resignation letter that defendants in criminal cases are perpetually disadvantaged during forensic discovery.

As the federal rules of criminal procedure now stand, prosecutors who intend to call forensic experts to testify do not have to supply the same full pre-trial discovery about those experts and the methodological and evidentiary bases for their opinions that parties calling forensic experts in civil cases are required to supply under federal rules of civil procedure.

The DOJ likes its strategic advantage and doesn't want Rakoff screwing with it. You say you want due process? Too bad. As a criminal defendant in today's justice system, you're little more than a bullet catcher for prosecutorial gunslingers.

The notion that pre-trial discovery of information pertaining to forensic expert witnesses is beyond the scope of the Commission seems to me clearly contrary to both the letter and the spirit of the Commission’s Charter… A primary way in which forensic science interacts with the courtroom is through discovery, for if an adversary does not know in advance sufficient information about the forensic expert and the methodological and evidentiary bases for that expert’s opinions, the testimony of the expert is nothing more than trial by ambush.

There's supposed to be equitable sharing and access to anything uncovered during discovery. But there obviously isn't. Rakoff recognized this and made his opinion known to the DOJ. Law enforcement agencies have apparently grown used to dealing from the bottom of the discovery deck and the United States' Deputy DA wants this privilege to remain intact. (See also: Judge Kozinski's dissenting opinion, which called out prosecutors for their "epidemic of Brady [exculpatory evidence] violations." It's not just junk science and fraud. It's also the withholding of evidence that could clear a person facing criminal charges.)

Rakoff's strongly-worded resignation letter (which has been published in full all over the internet) has led to a very minor concession from the DOJ.

However, Acting U.S. Deputy Attorney General Sally Q. Yates invited Rakoff to return, saying she had not been aware the commission had worked openly on its plans for nearly a year.

Yates told the National Commission on Forensic Science that “it seemed only fair” that it “make its determination as to what information should be provided to the Attorney General.”

“This is obviously a critically important issue to the Department,” Yates said. “We take very seriously our obligation to ensure that defendants receive a fair trial."

Except that its original conversation with Judge Rakoff indicates that it would rather do anything but "ensure… defendants receive a fair trial." But its "trial by ambush" intentions were exposed by Rakoff's public resignation and it had no choice but attempt to paper this over with some honorable-sounding words.

While committing to nothing, and mouthing the empty platitudinous “We take very seriously our obligation to ensure that defendants receive a fair trial,” which is a corollary to “we’re from the government, and we’re here to help,” at least one honorable person will sit on the committee to see whether this rises to the level of serious reform.

So, Judge Rakoff is back on the panel. Hopefully, this will lead to a change in the system and a public release of information relating to law enforcement's forensic failings. This nation seems to have no shortage of zealous prosecutors who are willing to secure convictions at the expense of true due process. The balance needs to shift back to the center. Judge Rakoff's resignation has brought this long-running problem back out into public eye -- along with the DOJ's apparent desire to keep its ambush tactics from being disrupted.

Filed Under: attorney general, doj, forensic evidence, forensics, jed rakoff