Scientists Looking To Fix The Many Problems With Forensic Evidence

from the can-you-fix-the-people-performing-the-tests? dept

Everything everyone saw in cop shows as evidence linking people to crimes -- the hair left on someone's clothing, the tire tracks leading out to the road, the shell casings at the scene, etc. -- is all proving to be about as factual as the shows themselves.

While much of it is not exactly junk science, much of it has limited worth. What appears to indicate guilt contains enough of a margin of error that it could very easily prove otherwise. Science Magazine is taking a look at the standbys of forensic science and what's being done to ensure better presentations of evidence in the future.

On a September afternoon in 2000, a man named Richard Green was shot and wounded in his neighborhood south of Boston. About a year later, police found a loaded pistol in the yard of a nearby house. A detective with the Boston Police Department fired the gun multiple times in a lab and compared the minute grooves and scratches that the firing pin and the interior of the gun left on its cartridge casings with those discovered on casings found at the crime scene. They matched, he would later say at a pretrial hearing, “to the exclusion of every other firearm in the world.”

[...]

So how could the detective be sure that the shots hadn’t been fired from another gun?

The short answer, if you ask any statistician, is that he couldn’t. There was some unknown chance that a different gun struck a similar pattern. But for decades, forensic examiners have sometimes claimed in court that close but not identical ballistic markings could conclusively link evidence to a suspect—and judges and juries have trusted their expertise. Examiners have made similar statements for other forms of so-called pattern evidence, such as fingerprints, shoeprints, tire tracks, and bite marks.
Six years ago, the National Academy of Sciences found that these forensic standbys had a much larger margin of error than was portrayed in court by detectives and expert witnesses. It recommended the margin of error be delivered along with the testimony to head off future verdicts based on faulty evidence.

To date, not much has changed. While actual junk science like bite marks has largely been discarded by prosecutors, the others remain, even as their reliability has been constantly questioned. The FBI loved hair analysis, right up to the point that it determined its witnesses had overstated test results 90% of the time in the two decades prior to 2000.

Even fingerprints, which have long been considered unassailable because of their supposed uniqueness, aren't much better. Some of it has to do with the presumption that every fingerprint is so unique even a partial print can eliminate suspects. The rest of its issues lie with those matching the prints.
One study of 169 fingerprint examiners found 7.5% false negatives—in which examiners concluded that two prints from the same person came from different people—and 0.1% false positives, where two prints were incorrectly said to be from the same source. When some of the examiners were retested on some of the same prints after 7 months, they repeated only about 90% of their exclusions and 89% of their individualizations.
The NIST has given $20 million to the Center for Statistics and Applications in Forensic Evidence (CSAFE) to come up with a better way to present this sort of evidence -- one that clearly accounts for any uncertainties in the results or processes. CSAFE is still trying to figure out how to present this as a number/rating. But that might not be the only problem. The other issue is that juries and judges may not find specifics about forensic reliability to play much of a part in deciding guilt or innocence.
In a 2013 study, for instance, online participants had to rate the likelihood of a defendant’s guilt in a hypothetical robbery based on different kinds of testimony from a fingerprint examiner. It didn’t seem to matter whether they were simply told that a print at the scene “matched” or was “individualized” to the defendant, or whether the examiner offered further justification—the chance of an error is “so remote that it is considered to be a practical impossibility,” for example. In all those cases, jurors rated the likelihood of guilt at about 4.5 on a 7-point scale. “As a lawyer, I would have thought the specific wording would have mattered more than it did,” Garrett says. But if subjects were told that the print could have come from someone else, they seemed to discount the fingerprint evidence altogether.
The other part of the problem is the people who perform the tests. Multiple incidents where evidence was falsified or not properly tested have been uncovered. The evidence is only as good as the processes, and if steps are skipped because of sloppiness or laziness, the evidence's credibility becomes highly questionable -- not just for the specific instance where results were faked, but for every test this person has touched.

There's no possible way to eliminate honest errors, much less prevent anyone from falsifying results. In both cases, the problems are caught after the damage has been done. Humans are the most unpredictable part of the chain of evidence but also an irreplaceable part. CSAFE will be working with forensics labs to create best practices, but it can do nothing to prevent the lazy and/or incompetent from completely ignoring the proper steps.

Problems are also present higher up the chain. When bad science or bad practices result in questionable evidence, it's often extremely difficult to have convictions resulting from them overturned.
What’s troubling, [federal judge Nancy] Gertner says, is that when judges accept junk science, an appeals court rarely overrules them. Attaching a numerical probability to evidence, as CSAFE hopes to do, “would certainly be interesting,” she says. But even a standard practice of critically evaluating evidence would be a step forward. “The pattern now is that the judges who care about these issues are enforcing them, and the judges who don’t care about these issues are not.”
In this way, the courts are no better than labs where shoddy work is done. Variations in personality undermine the dispassionate nature of science, making it susceptible to human prejudices rather than the strength of the evidence itself.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: evidence, fbi, forensic evidence, forensics


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • icon
    Groaker (profile), 29 Apr 2016 @ 7:30pm

    This is nothing new. It has been known since at least the late '60s, and I have no doubt before then. But that date is selected because that was when I saw a blind QA study, in a medical journal, on forensic labs. Half the specimens were incorrectly analyzed.

    Nothing changes because the system would have to change, and that is not going to happen.

    link to this | view in chronology ]

  • identicon
    JustShutUpAndObey, 29 Apr 2016 @ 7:41pm

    Goals

    The goal of the court system is not and never has been the truth.
    The goal is to obtain convictions.
    The truth might make that too difficult.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 29 Apr 2016 @ 7:55pm

      Re: Goals

      unfortunately jury's just do not see it that way.

      you would not be in court if you were not a dirt bag. America has a fetish for putting innocent people in jail... I mean you have to be guilty for something anyways right?

      link to this | view in chronology ]

      • identicon
        Anonymous Coward, 30 Apr 2016 @ 4:16am

        Re: Re: Goals

        Only the disadvantaged are viewed in this manner.
        Bankers are always innocent even when overwhelming evidence says otherwise.

        link to this | view in chronology ]

        • icon
          Uriel-238 (profile), 30 Apr 2016 @ 10:48am

          Bankers

          Bankers have money to hire a lawyer good enough to convince them otherwise.

          The law is for those with money.

          link to this | view in chronology ]

          • identicon
            Anonymous Coward, 30 Apr 2016 @ 12:14pm

            Re: Bankers

            Exactly.

            Funding of the Public Defenders office is a joke.

            link to this | view in chronology ]

            • identicon
              Anonymous Coward, 30 Apr 2016 @ 6:54pm

              Re: Re: Bankers

              The real joke is the idea that lawyers were ever necessary.

              A system of laws that requires lawyers to effectively navigate them is of no service to the people.

              Lawyers should be outlawed and Judges should be required to participate in continuing education just like medical professionals!

              link to this | view in chronology ]

    • icon
      madasahatter (profile), 30 Apr 2016 @ 7:57am

      Re: Goals

      The prosecutor's goal is often a conviction but in some jurisdictions the jury is charged with finding the truth.

      Back to the issue in the post, part of the problem is crime scene evidence is often poor quality (smudged fingerprints, partial prints, etc.) that make accurate identification very difficult. Couple this with very sloppy evidence handling procedures (OJ trial when the detective kept the samples in his coat pocket) and there is no valid evidence.

      link to this | view in chronology ]

  • identicon
    Anonymous Coward, 29 Apr 2016 @ 7:53pm

    Science... Good for only 1 direction

    Science is only good for proving that the person IS NOT GUILTY, it is not good enough to PROVE guilt.

    We only test a subset of DNA, which means you can get far more false positives than even close to comfortable when you look at the numbers. Additionally with all of the DNA databases now being referenced for criminal searches the police will only see more potential suspects as we move forward. Combine that with the Guilty until proven innocence and ignorant as fuck on science American people and you have a recipe for innocent people going to jail closing the case while the actual criminals walk free!

    link to this | view in chronology ]

    • identicon
      Possibly moving poor people to jail is the goal - , 29 Apr 2016 @ 9:31pm

      Re: Science... Good for only 1 direction

      Many states have "for profit" private prison systems.
      These states likely really need to feed more serfs/poor people into the system to meet the contract terms.

      The US Gulag - (War on Every Damn Thing) justice system!

      Billions for Pretend investigation's, "Faux forensics" and guaranteed funds to private contractors.
      Nowadays mostly the result of of dark money (and future employment promises) fed through the right State/Fed Lobbyists (Definition: purveyor of funds/jobs to corrupt Politicians).

      link to this | view in chronology ]

      • identicon
        Anonymous Coward, 30 Apr 2016 @ 3:10am

        Re: Re: Science... Good for only 1 direction

        It's criminal how you missed the opportunity to call it Fauxrensics!

        link to this | view in chronology ]

    • icon
      madasahatter (profile), 30 Apr 2016 @ 8:08am

      Re: Science... Good for only 1 direction

      Good detectives know the limits of the evidence. But one key is amount of evidence present. If the murder weapon is found in your possession, your finger prints are present, your DNA is present, you were seen in the area at time of the crime, etc. you now have some explaining to do. One item is likely not enough to get a conviction.

      Also, DNA testing has improved tremendously since it was first introduced to the point it is one the best methods to positively identify a person (except for identical twins) available. Many cold cases have been solved because of DNA evidence.

      The point of the scientific evidence (all evidence actually) is not that each bit proves absolutely the guilt of the person but that there is enough evidence that beyond a reasonable doubt the person is guilty. If you apply probability, if each bit is 95% certain the more bits will push the overall certainty much higher.

      link to this | view in chronology ]

      • identicon
        Anonymous Coward, 30 Apr 2016 @ 9:44am

        Re: Re: Science... Good for only 1 direction

        One item is likely not enough to get a conviction.

        Tell that to all the people wrongly convicted.

        link to this | view in chronology ]

      • identicon
        Anonymous Coward, 30 Apr 2016 @ 7:06pm

        Re: Re: Science... Good for only 1 direction

        I said PROOF not good enough for PROOF!

        And while the detectives know that limit juries often do not. You will almost never find any jury ever suspect of evidence, far to many of them treat it like evidence is some gospel truth for just exactly the reasons you specified.

        It is far to easy to admit a stack questionable evidence that makes someone look damn guilty despite a lot of that evidence being circumstantial and ultimately inconclusive.

        A good example is someone not being anywhere near a crime scene but having been accused of lying to the police and going to jail on that alone. Even when they were not lying at all! All it takes is for one officer to lie about you and even worse if they can get a witness to say that they "think" they saw you somewhere else calling your alibi into question and you can go to jail by a just in case you ARE a criminal jury.

        We need to seriously stop underestimated the amount of fucks people really do not give. When was the last time you went out and tried to stop someone from being screwed by he government? Now contrast that to all of the specially selected jury's that are lied to that only want to get back to their lives and you can begin to see how the system has been slowly engineered to produce successful criminal prosecutions! Even when evidence is complete bunk.

        They just throw a low of evidence at the jury hoping something sticks to overwhelm them and making them think... there has to be no way for this person to be innocent... look at all of that evidence right?

        You should read up on the innocence project and you will quickly see how wrong you are.

        In all aspects of law, do not discount the corruption factor multiplied by the worthless and ignorance citizen factor, its a huge factor and often the deciding factor... to hell with any meaningful evidence!

        link to this | view in chronology ]

  • icon
    Coyne Tibbets (profile), 30 Apr 2016 @ 1:25am

    Hard vs cakewalk

    Let's see...doing it forensic evidence right versus doing a cakewalk to conviction. How do we think real-world CSI's will decide?

    Forensic scientists can debate until they're blue, but nothing is changing until their testimony starts sinking cases. And since forensic scientists are "free" for testimony at $400/hour or so, guess how much testimony they'll be giving for average man.

    link to this | view in chronology ]

  • identicon
    Dr. Glover, 30 Apr 2016 @ 3:53am

    I have some horror stories...

    regarding some forensics science majors I went to college with.

    In lab-based biology classes they had absolutely no qualms "cooking the books" to get a desired data outcome, rather than explain why their experiment didn't work in a scientific manner. Because of that, I refused to work with them on any group projects. And as professionals now, I have little doubt that they are still fudging numbers.

    So, think about the issues the FBI has had with this problem and realize the willingness to change data starts early in the careers of these individuals. They somehow manage to not get caught, or professors/instructors don't give a damn.

    So forensics has the problem of not only 1.) scientific validity, but 2.) highly unethical scientists willing to manipulate data.

    link to this | view in chronology ]

  • identicon
    New Mexico Mark, 30 Apr 2016 @ 4:49am

    Are grammar police unique?*

    It is a good habit to mentally replace the word "unique" with its short definition, "one of a kind". This prevents misuse with modifiers, and allows poor, neglected words like "unusual" to step up to the plate every now and then.

    * http://goo.gl/yMp5eB

    Despite my griping, I enjoyed the article. What disturbs me far more than pseudo-scientific bias is the jury selection process itself. Smart, conscientious jurors would help mitigate a lot of these problems. However, in at least one U.S. state jury members are practically guaranteed to be either liars or useless in their role as mediators of justice by one qualifying question along the lines of, "Are you willing to set aside conscience and your own knowledge and decide this case only according to the law and the instructions of the judge?" If laws are always good and judges are always right, why did we need juries in the first place?

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 30 Apr 2016 @ 12:41pm

      Re: Are grammar police unique?*

      You're not a grammar Nazi. You just choose to ignore the fact that language is fluid and changes over time.

      The earliest meanings of unique when it entered English around the beginning of the 17th century were “single, sole” and “having no equal.” By the mid-19th century unique had developed a wider meaning, “not typical, unusual,” and it is in this wider sense that it is compared: The foliage on the late-blooming plants is more unique than that on the earlier varieties. The comparison of so-called absolutes in senses that are not absolute is standard in all varieties of speech and writing.

      link to this | view in chronology ]

      • identicon
        Anonymous Coward, 30 Apr 2016 @ 1:06pm

        Re: Re: Are grammar police unique?*

        You're not a grammar Nazi. You just choose to ignore the fact that language is fluid and changes over time.

        yah, it changes everytime i uses it. i aint never wrong, just fresh.

        link to this | view in chronology ]

        • identicon
          Anonymous Coward, 1 May 2016 @ 7:07am

          Re: Re: Re: Are grammar police unique?*

          I find it amusing how some words morph onto meaning the exact opposite of what was once the case.

          For example, the word literally.
          I smirk every time someone uses "literally" in a sentence where they really mean "figuratively".

          That plane was so low that I literally could reach out and touch it.

          I would like to see that video.

          link to this | view in chronology ]

          • icon
            Uriel-238 (profile), 1 May 2016 @ 11:41am

            Literally offensive

            A few published and otherwise well-versed writers started using literally for emphasis, much like seriously or really and it caught on.

            I still cringe at it since literally is the last bastion for non-metaphorisms. So that when I say Game of Thrones (or more properly A Song of Ice And Fire) is a literal epic, that's not saying it's really awesome (though it's pretty durned good) but also fits the properties of the epic genre classification.

            link to this | view in chronology ]

  • identicon
    Anonymous Coward, 30 Apr 2016 @ 6:07am

    Zoom in on that reflection .. Enhance .. Rotate
    That's our perp right there, plain as day.
    These cheap ass video cams with low resolution are no match for the forensic wizards at CSI.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 30 Apr 2016 @ 7:24am

    Get rid of the cheap slave labour and the for profit prison complex America has then you will get rid of the incentive to "gift" public officials with large sums of money to put away anyone and everyone in prison on the most dubious of evidence.

    link to this | view in chronology ]

  • icon
    Ninja (profile), 30 Apr 2016 @ 7:26am

    I'm waiting for something that will beat two agents using the same keyboard to outhack a hacker. No, really. I wonder if Clapper and the likes believe it works that way.

    link to this | view in chronology ]

  • identicon
    Quiet Lurcker, 30 Apr 2016 @ 7:29am

    It would take a ground-shift in how things are done in court, but, how about this idea?

    Instead of the lawyers presenting to the judge, they present to the jury (I think it's supposed to be that way anyway - hence, e.g., jury consultants), but let's re-emphasize that point.

    Then, during the course of the case, as each witness is done testifying for the lawyers, the jury are given an opportunity to present matters on which they want amplification or further information. Basically, give the jury the opportunity to explore the evidence in their own way, and in their own time.

    As to the integrity of the forensic labs, maybe require the prosecution to present the full chain of custody, for examination by the defense, at least.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 30 Apr 2016 @ 2:34pm

      Re:

      sadly the prosecution goes out of it's way to weed out any potential juror's that have any basic intelligence about them

      link to this | view in chronology ]

  • identicon
    normanjd, 30 Apr 2016 @ 7:48am

    Fingerprints

    The problem with many of these test is not uniqueness but how we perform the test.

    With fingerprints, you have to find so many points of commonality for a match. (I forget the exact numbers but let's say 9 for a partial and 15 for an exact match.) One issue is these numbers are probably too low. Also when these techniques were automated, different programs use different algorithms to find what is common and they produce different results. (This is where you get too many false negatives and a few false positives. Double checking by hand should eliminate false positives, but sometimes this is NOT done as the prosecutor wants evidence he can use. Even so, based on numbers in article, odds are only 1 in 1000 cases would have a false positive, and the defense does have access to the fingerprint to manually double check.)

    link to this | view in chronology ]

    • icon
      madasahatter (profile), 30 Apr 2016 @ 8:12am

      Re: Fingerprints

      Usually most cases turn on a few key pieces of evidence and both sides are aware of the evidence. A competent defence team will challenge the solidness of these keys.

      link to this | view in chronology ]

      • identicon
        Anonymous Coward, 30 Apr 2016 @ 9:51am

        Re: Re: Fingerprints

        A competent defence team will challenge the solidness of these keys.

        Only if the judge allows it.

        link to this | view in chronology ]

  • icon
    Uriel-238 (profile), 30 Apr 2016 @ 10:58am

    No interest in seeking justice.

    As JustShutUpAndObey, our courts of law are not interested in seeing justice done, but getting the bad-guy. And every suspect not convicted is a bad-guy who escaped, not an innocent who was exonerated.

    This has to change before we consider forensic evidence. Otherwise they can just get a company to build a bad-guy detector which sounds an alarm when you point it at a suspect. Currently, like our detector dogs, that is the height of where forensic science can go.

    Right now we're in a state where forensic labs are not impartial, and will skew their results to implicate, even to the point of committing open fraud.

    In this state, the only purpose a forensic lab can serve is the same as detector dogs, to implicate the innocent and force a conviction.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 30 Apr 2016 @ 3:54pm

    Maybe we need "open-source" evidence analysis. They say many eyes on the code is the best quality control for software. Would the same apply to criminal evidence? Would a forensic professional, knowing that his or her methods and conclusions would be re-examined by hundreds of others, be motivated to work to a higher standard of ethics?

    link to this | view in chronology ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.