Another Law Enforcement Investigation Tool Found To Be A Junk Science Coin Toss

from the Detective-Mesmer-is-on-the-case dept

Ken Armstrong of ProPublica and Christian Sheckler of the South Bend Tribune have uncovered more pseudoscience cops rely on to put people behind bars. It's called SCAN, and it's the creation of Avinoam Sapir.

SCAN stands for Scientific Content Analysis. What's "scientific" about it is undetermined. Sapir believes it is. Sapir's background as a polygraph examiner for Israeli law enforcement apparently turned him into a "human lie detector." Sapir appears to believe his own press, as long as it's positive.

Here's how SCAN works. Or, rather, here's how SCAN operates.

1. Give the subject a pen and paper.

2. Ask the subject to write down his/her version of what happened.

3. Analyze the statement and solve the case.

Those are the steps according to Sapir's site. Law enforcement agencies around the world have spent money purchasing Sapir's SCAN products and training. The scientific breakthrough they're buying has no science backing it. According to Sapir, different words or tense changes in people's written statements can let officers know whether or not suspects are lying.

How does anyone know if SCAN works? Just ask Sapir, apparently.

SCAN works, the site says. “Analysis of statements has been found to be highly accurate and supported by a validation survey conducted in a U.S. governmental agency. In that survey, when SCAN was compared to other methods, the validity of SCAN reached above 95%,” the site says, without identifying the agency or citing or linking to any survey.

SCAN's junk science may be in wide use but it hasn't really made headlines like other forensic pseudoscience -- bite mark analysis, blue jeans analysis, etc. -- but it's no less questionable or potentially dangerous. No one runs SCAN on confessions. They only use the tool to "analyze" statements made by suspects and witnesses. These analyses are rarely introduced as evidence in court.

Good thing, too. It seems unlikely stuff like this would be considered admissible evidence.

After the person writes a statement, the SCAN investigator looks for signs of deception, analyzing, among other things, pronouns used, changes in vocabulary, what’s left out and how much of a statement is devoted to what happened before, during and after an event. Indications of truthfulness include use of the past tense, first-person singular (“I went to the store”); pronouns, such as “my,” which signal commitment; and direct denials, the best being: “I did not do it.” Signs of deception include lack of memory, spontaneous corrections and swapping one word in for another — for example, writing “kids” in one place and “children” in another.

It's not exactly handwriting analysis, which has its own problems. It's its own animal, one that's only available through Sapir. He's offered up some bizarre analysis of his own -- applying his proprietary SCAN to things like Anita Hill's statements about Clarence Thomas, Magic Johnson's disclosure that he has HIV, and James Comey's memoir. His conclusions? Anita Hill had sexual identity issues, Johnson "admitted" to being bisexual, and James Comey is a victim of child abuse.

Comey's statement on Sapir and the SCAN analysis of his memoirs?

“No comment. Never heard of the alleged tool. (And by using the word ‘never’ in conjunction with the word ‘heard,’ I mean only ‘never heard’ and not to suggest childhood trauma. Yikes.) I’m sorry about your five bucks.”

The "five bucks" refers to the amount ProPublica spent to obtain Sapir's analysis of Comey's memoirs. He also sells an analysis of the Mueller Report.

Law enforcement agencies rely on this dubious technique to solve crimes. Their reliance on Sapir's assertions SCAN actually has something to do with science is misplaced. SCAN is apparently used by the FBI (despite Comey's denial), CIA and US Army Intelligence. A government study commissioned to find which, if any, interrogation methods produced usable intelligence had this to say about SCAN:

Studies commonly cited in support of SCAN were scientifically flawed, the review said. “When all 12 SCAN criteria were used in a laboratory study, SCAN did not distinguish truth-tellers from liars above the level of chance,” the review said. The synopsis also specifically challenged two of those 12 criteria, noting: “Both gaps in memory and spontaneous corrections have been shown to be indicators of truth, contrary to what is claimed by SCAN.”

Four more studies -- conducted over a span of twenty years -- arrived at the same conclusion:

Each of those studies (in 1996, 2012, 2014 and 2015) concluded that SCAN failed to help discriminate between truthful and fabricated statements. The 2016 study found the same. Raters trained in SCAN evaluated 234 statements — 117 true, 117 false. Their results in trying to separate fact from fiction were about the same as chance.

This is what scientists think about SCAN, something Sapir says can turn normal cops into "walking polygraphs." If law enforcement experts were upfront about the dubiousness of SCAN conclusions, it wouldn't be an issue. But they're not. Much like every other pseudoscience deployed by investigators, experts testifying in court routinely overstate the certainty of their conclusions. SCAN is a coin toss, but no one presenting this analysis in court is ever going to admit this under oath.

Cops apparently believe it's a coin flip worth investing in. ProPublica's investigation found at least 40 agencies had purchased SCAN training in the last decade. Sapir's website claims agencies in 49 states utilize SCAN. Very few agencies want to talk about their SCAN expenditures and the CIA has refused to confirm or deny the existence of SCAN use records.

The agencies that do believe SCAN works really believe it works. But these are statements of faith. They have no evidence it works, only the word of its creator who says it works. And if you're spending money to train investigators to deploy SCAN analysis, you kind of have to believe it works because the alternative is knowingly throwing money down a hole. Fervently believing something is scientific just isn't good enough when people's freedom is on the line. But not good enough has been good enough for plenty of agencies and the junk science their investigations rely on for years now, and it's not going to change just because another emperor has been discovered wandering naked.

Filed Under: avinoam sapir, content analysis, lie detector, pseudoscience, scan

Europe's Latest Border Security Efforts Combines Junk Science With Lie Detectors

from the strangely-expensive-for-such-a-shit-solution dept

There's border garbage going on in Europe as well. A report by The Intercept shows border officials have cobbled together junk science, tech, and a spin on a notoriously-sketchy piece of equipment into its newest border security offering.

It's called Silent Talker. It subjects travelers to lie detector tests predicated on the fiction that people call tell other people are lying just by looking at them. It's the same pseudo-science that powers the TSA's useless "Behavioral Detection" program. Only this is possibly worse because it considers itself to be a lie detector and it's been known for years lie detectors can't reliably detect lies.

It works like this: travelers upload their passports to the border agency's website and are put face-to-"face" with a blue uniformed avatar. The software takes control of the device's camera to scan the traveler's face and eye movements for "signs of lying."

Here's the thing: it doesn't even work when it's humans doing the face-to-face work. A report on the TSA's Behavioral Detection program found it to be completely lacking in scientific background. The justification for the program was predicated on hearsay, conjecture, and anecdotal evidence. The TSA claimed it was hard science, but actual scientists have said there's no evidence backing the claim that anyone can suss out lies just by looking at people's faces.

These research findings do not imply that people move their faces randomly or that the configurations in Figure 4 have no psychological meaning. Instead, they reveal that the facial configurations in question are not “fingerprints” or diagnostic displays that reliably and specifically signal particular emotional states regardless of context, person, and culture. It is not possible to confidently infer happiness from a smile, anger from a scowl, or sadness from a frown, as much of current technology tries to do when applying what are mistakenly believed to be the scientific facts.

Instead, the available evidence from different populations and research domains—infants and children, adults living in industrialized countries and in remote cultures, and even individuals who are congenitally blind—overwhelmingly points to a different conclusion: When facial movements do express emotional states, they are considerably more variable and dependent on context than the common view allows.

[...]

Efforts to simply “read out” people’s internal states from an analysis of their facial movements alone, without considering various aspects of context, are at best incomplete and at worst entirely lack validity, no matter how sophisticated the computational algorithms.

Given this scientific background, it's no surprise Europe's "Silent Talker" lie detector/behavioral detection program can't actually detect lies or liars.

A person judged to have tried to deceive the system is categorized as “high risk” or “medium risk,” dependent on the number of questions they are found to have falsely answered. Our reporter — the first journalist to test the system before crossing the Serbian-Hungarian border earlier this year — provided honest responses to all questions but was deemed to be a liar by the machine, with four false answers out of 16 and a score of 48. The Hungarian policeman who assessed our reporter’s lie detector results said the system suggested that she should be subject to further checks, though these were not carried out.

The rush to allow tech to do the work of people -- as well as the firm belief that if something is expensive it must be good -- has led to the funneling of time and money into this highly-questionable border security effort.

IBorderCtrl’s lie detection system was developed in England by researchers at Manchester Metropolitan University, who say that the technology can pick up on “micro gestures” a person makes while answering questions on their computer, analyzing their facial expressions, gaze, and posture.

An EU research program has pumped some 4.5 million euros into the project, which is being managed by a consortium of 13 partners, including Greece’s Center for Security Studies, Germany’s Leibniz University Hannover, and technology and security companies like Hungary’s BioSec, Spain’s Everis, and Poland’s JAS.

There's more money on the way and companies will be pushing dubious "solutions" to governments with security money to burn. European nations have been hit with a series of terrorist attacks, which means the wind is now catching most of these governments' caution and junk science purveyors are converting fervent beliefs into long-term government contracts. The European Commission has budgeted nearly 35 billion euros over the next eight years for border control efforts. Being first is better than being correct, and governments all over the world have shown a willingness to allow unproven tech to use their citizens for on-the-job training.

Filed Under: border security, eu, junk science, lie detector

Desperate To Stop Leaks, The Trump Administration Considers Moving From Bad Ideas To Worse Ones

from the do-you-consent-to-the-use-of-sodium-pentothal? dept

The Trump Administration is so frustrated by constant leaks, it's willing to try anything to stop them. Apparently, this may one day involve questionable tech with an extremely-spotty track record. Attorney General Jeff Sessions has an idea -- a bad one -- to address the ongoing White House leakstravaganza.

Sessions' idea is to do a one-time, one-issue, polygraph test of everyone on the NSC staff. Interrogators would sit down with every single NSC staffer (there's more than 100 of them), and ask them, individually, what they know about the leaks of transcripts of the president's phone calls with foreign leaders. Sessions suspects those leaks came from within the NSC, and thinks that a polygraph test — at the very least — would scare them out of leaking again.

Sessions' spitball may never come to fruition but it does indicate the White House is still no closer to uncovering the source(s) of these leaks. This also indicates the AG is willing to alienate NSC staffers with mandatory tests predicated on nothing more than Sessions' belief the leaks came from within the NSC.

On top of that, there's the dubious deterrent effect of deploying machinery whose results should be taken with a shaker of salt. The machines can be beat. Research and "How To" literature bear this out, although the federal government does what it can to prevent the spread of the latter. False positives are a problem as well. Ringing up staffers for leaking just because a machine said yes isn't going to stop the leaks or put investigators any closer to catching the real culprit(s).

But this is the way things go when the federal government is in charge. It relies heavily on polygraph tests when vetting applicants for agencies like the Customs and Border Protection. And it continues to deploy these tests even when applicants have admitted to activities or relationships that would disqualify them from federal employment.

Chances are Sessions' polygraph idea will likely remain just an idea. It could be the multiple discussions Sessions is engaging in are all part of his leak deterrence plan -- to have the threat of mandatory lie detector tests hovering in midair for the rest of tenure. Even so, it's a desperation move that shows the White House can't contain the leaks, much less hope to stop them. Suggestions like the one made by Sessions are just going to turn more staffers against the administration, which means more leaks, rather than less, over the course of this administration's lifetime.

Filed Under: jeff sessions, leaks, lie detector, national security, polygraph

Feds Indict Another Person For Teaching People How To Beat Polygraph Tests

from the it's-blue-it's-blue-the-goddamn-pen-is-blue dept

Polygraph technology is far from infallible and has been for so long that it's practically common knowledge. And yet, the federal government still wants everyone to believe polygraphs tests separate the honest from the liars with incredibly high accuracy. So, it cracks down on those who claim to be able to help others beat the tests.

In 2012, federal agents began investigating Chad Dixon and Doug Williams, two men who sold books, videos and personal instruction sessions on beating polygraph tests. Late last year, Dixon was sentenced to eight months in prison for obstruction and wire fraud charges. The government claimed his actions jeopardized national security, pointing to a client list that included intelligence employees, law enforcement agents and sex offenders.

The government has just handed down an indictment [pdf link] of its second target -- former Oklahoma City police polygraph administrator Doug Williams.

The 69-year-old Norman, Oklahoma, man is the owner of Polygraph.com and charged customers thousands of dollars for instructions on how to beat lie detector tests administered for federal employment suitability assessments, federal security background investigations, and internal federal agency investigations, court documents show.

The government wants to see Williams locked up for fraud, claiming his polygraph-beating business allowed unqualified applicants to "obtain and maintain positions of Federal employment" and the "salary attendant to such positions."

The 23-page indictments details the fed's sting operation, which utilized two undercover agents posing as potential government employees with shady pasts. In both cases, the agents made Williams aware of past wrongdoing (over his initial objections) that should disqualify them from positions in the DHS or Border Patrol. Williams, unfortunately, decided to follow through with personally training both agents, despite his initial hesitance to knowingly assist admitted criminals with obtaining government jobs.

The indictment said that, during a telephone call with an undercover federal agent, Williams said, "I haven't lived this long and fucked the government this long, and done such a controversial thing that I do for this long, and got away with it without any trouble whatsoever, by being a dumb ass."

Well, if the indictment is legit, Williams made some dumbass moves. The unanswered question, though, is if providing instructions on how to beat polygraph tests is illegal or protected speech. The judge handling the conviction of Chad Dixon had a hard time finding a "bright line" between the two.

O’Grady acknowledged “the gray areas” between the constitutional right to discuss the techniques and the crime of teaching someone to lie while undergoing a government polygraph. “There’s nothing unlawful about maybe 95 percent of the business he conducted,” the judge said.

Despite this, Judge O'Grady still gave the government what it wanted: a prison sentence to "deter" activities that were "95 percent" free speech. Obviously, the government is hoping for the same outcome here.

The real weakness here is the government's reliance on polygraph testing, but no one's interested in addressing that. The government farms out employee vetting to private contractors who can't be bothered to do the job properly. Negative results are ignored by government agencies when they stand in the way of hiring the people they want to hire.

A system that can be successfully gamed shouldn't be afforded as much deference (bordering on reverence) as the government gives to polygraph testing. It's a method that's just as likely to reject legitimate candidates as it is to be duped by criminals. Prosecuting those who show others how the test can be beaten is nothing more than punishing the symptom because it's so much easier than dealing with the disease.

Filed Under: doj, doug williams, indictment, lie detector, polygraph
Companies: polygraph.com

FBI Wants To Know If Applicants Have Been Downloading Unauthorized Content

from the limiting-the-supply dept

Earlier this year, FBI Director James Comey suggested that the FBI might consider backing off its policy of refusing to hire anyone who has used marijuana in order to find competent computer folks who can deal with online crimes. After some backlash (and some support) for those statements, Comey quickly backed down, claiming it was all just a joke.

Now, TorrentFreak has revealed that beyond pot smoking disqualifying you, so might your history of downloading some music and movies. This came up at a Sacramento State Career Center information session held by the FBI, where the FBI made it clear that your downloading practices matter to the agency.

“If you’re doing that, stop doing it.” Dupree said.

He explained how the FBI will ask people during interviews how many songs, movies and books they have downloaded because the FBI considers it to be stealing.

During the first two phases of interviews, everything is recorded and then turned into a report. This report is then passed along to a polygraph technician to be used during the applicant's exam, which consists of a 55-page questionnaire. If an applicant is caught lying, they can no longer apply for an FBI agent position.

“If you are accepted to intern at FBI and fail the polygraph you can no longer apply to FBI again.” Dupre said.

From the sound of that, your history might not fully disqualify you, but it may come up again later in the process. TorrentFreak, though, found a post from a few years ago on a job board, where someone says that his downloading past kept him from being hired by the FBI.

It's interesting to note that TorrentFreak also turned up an article saying that downloading unauthorized content does not disqualify you from the CIA. The difference here actually makes some sense -- since the FBI actually does handle some cases involving copyright (though, that still seems ridiculous). Either way, if the FBI is barring people who use pot and who have infringed on copyrights, they're going to find themselves with an increasingly limited supply of computer experts for its computer crimes division.

Filed Under: cia, copyright, downloads, fbi, lie detector

Eight Months In Jail For Teaching People How To Pass A Lie Detector Test

from the for-tests-that-don't-even-work-well dept

Just a few weeks ago, we wrote about how the feds had been directly going after people who claimed to teach others how to pass lie detector tests, claiming that this was part of their war on stopping leakers and security vulnerabilities. What was chilling was that a US official specifically said that they needed to focus their investigative efforts on the people who protested the loudest, which seems like a clear attack on free speech. As we noted at the time, the feds had already filed criminal charges against two people for offering to teach others how to beat lie detector tests, and now one of them has been sentenced to eight months in jail -- for teaching people how to beat a test that many argue has never ever been accurate in the first place.

As we noted last time, TV shows like Mythbusters and Penn & Teller: Bullshit! have both more or less taught people the same thing. While the government sought an even longer sentence, this is still worrisome on a variety of levels. Merely instructing people how to beat an extremely faulty technology should never be a crime. The judge and the DOJ's statements on the case are just bizarre:

O’Grady acknowledged “the gray areas” between the constitutional right to discuss the techniques and the crime of teaching someone to lie while undergoing a government polygraph. “There’s nothing unlawful about maybe 95 percent of the business he conducted,” the judge said.

However, O’Grady added that “a sentence of incarceration is absolutely necessary to deter others.”

Deter others from what? From speaking out about how to trick incredibly unreliable technology that the government probably relies on too much already?

“This crime matters because what he did endangers others,” said Anthony Phillips, a prosecutor with the Justice Department’s division that pursues corrupt public officials.

No it didn't. It doesn't endanger others to show them that a lie detector is faulty and not reliable. What endangers people is the federal government relying on such a dreadful technology that's so easy to "beat."

In many ways, this is similar to discovering a massive security flaw and then blaming the messenger for showing others how that security is flawed. The proper response is to not use the flawed security. But that's not how the government operates, unfortunately.

Filed Under: criminal, lie detector, teaching

Lie Detector Company Threatens Researchers, Draws Much More Attention To Research

from the how-about-a-common-sense-detector dept

Slashdot points us to a story of a lie detector manufacturer, Nemesysco, who apparently was so upset with a report from some Swedish researchers in a technology journal, that they threatened legal action against the journal and the researchers, claiming that they would sue for defamation if the article wasn't taken down. Since the basic point of the journal article was that the lie detecting technology that Nemesysco was betting on simply could not work, you can understand why they might be upset about it. But calling it defamation is highly questionable.

If the information presented in the article was wrong why not just counter it and point out why it's wrong? Threatening defamation lawsuits and trying to shut up the researchers just makes it look like Nemesysco has something to hide. And, indeed, true to the Stresisand Effect, the article reports that the researchers have received a lot more attention for their research since the threats were issued: "It was hardly their intention. But since the article was withdrawn, I have received lots of mail and requests for copies of the article. The article would not have been read to this extent if the company had simply ignored it in silence." Who knows whether or not Nemesysco's lie detector works, but its common sense detector is apparently on the fritz.

Filed Under: lie detector, research, streisand effect
Companies: nemesysco