FBI's Facial Recognition Database Still Huge, Still Inaccurate, And DOJ Shows Zero Interest In Improving It

from the ALL-YOUR-FACE-ARE-BELONG-TO-US dept

The FBI's biometric database continues to grow. Its Next Generation Identification system (NGI) is grabbing everything it can from multiple sources, compiling millions of records containing faces, tattoos, fingerprints, etc. from a blend of criminal and non-criminal databases. It went live in 2014, but without being accompanied by the Privacy Impact Assessment (PIA) it promised to deliver back in 2012.

Lawsuits and pressure from legislators finally forced the FBI to comply with government requirements. That doesn't mean the FBI has fully complied, not even two years past the rollout. And it has no interest in doing so in the future. It's currently fighting to have its massive database exempted from federal privacy laws.

Much of the information we have about the FBI's NGI database has come from outside sources. The EFF and EPIC have forced documentation out of the agency's hands via FOIA lawsuits. And now, the Government Accountability Office (in an investigation prompted by Sen. Al Franken) is turning over more information to the public with its review of the system.

What the GAO found is more bad news. The FBI is all about collecting data. It has little interest in ensuring the data is accurate or even usable.

The FBI’s system searches not just its own database, but also photo databases maintained by seven participating states, the US Department of State – which issues passports – and the US Department of Defense, shared among federal law enforcement agencies and the participating agencies, though access on the state level is obtained through the FBI.

This is only part of the NGI. To amass the 411 million photos it has collected to this point, the FBI dumps in the contents of a national criminal database.

[T]he GAO report found a much larger program, run by the criminal justice information services division of the FBI (CJIS), called Facial Analysis, Comparison and Evaluation, or Face, which “conducts face recognition searches on NGI-IPS and can access external partners’ face recognition systems to support FBI active investigations”.

The multiple inputs -- which allow criminal and non-criminal biometric data to intermingle -- still return an alarmingly high number of false positives. According to data obtained by EPIC, the facial recognition portion shows an error rate of 15-20% in the top 50 results returned from searches. That was the error rate in 2010. We can assume the hit rate has improved since then, but we have no way of knowing what the current error rate is because the FBI is uninterested in policing the accuracy of its database.

From the GAO report [PDF]:

Prior to deploying NGI-IPS, the FBI conducted limited testing to evaluate whether face recognition searches returned matches to persons in the database (the detection rate) within a candidate list of 50, but has not assessed how often errors occur. FBI officials stated that they do not know, and have not tested, the detection rate for candidate list sizes smaller than 50, which users sometimes request from the FBI… Additionally, the FBI has not taken steps to determine whether the face recognition systems used by external partners, such as states and federal agencies, are sufficiently accurate for use by FACE Services to support FBI investigations

The GAO report also points out the FBI has been severely delinquent in its obligations to the public. Reports it was supposed to deliver prior to rollout have only just recently appeared, including one release apparently prompted by the GAO's assessment of the NGI program.

NGI-IPS has been in place since 2011, but DOJ did not publish a System of Records Notice (SORN) that addresses the FBI's use of face recognition capabilities, as required by law, until May 5, 2016, after completion of GAO's review. The timely publishing of a SORN would improve the public's understanding of how NGI uses and protects personal information.

The GAO has made six recommendations to the agency, three of which are being disputed by the DOJ. According to the DOJ, the reason for the mandatory reports being delivered after-the-fact doesn't need to be examined because the FBI "has established practices that protect privacy and civil liberties beyond the requirements of the law." This sounds like the FBI has "nothing to hide," which is at odds with the lack of responsiveness by the agency to demands for updated PIAs and SORNs over the last eight years.

The DOJ also disagrees that it should have to audit the facial recognition database's "hit rate," something that was only 80-85% accurate five years ago. (In fact, the FBI's specifications consider 85% accuracy to be acceptable when returning lists of possible suspects.) The DOJ claims the database can never return a false positive because it apparently has enough manpower and resources to chase down every bogus lead.

In its response, DOJ stated that because searches of NGI-IPS produce a gallery of likely candidates to be used as investigative leads instead of for positive identification, NGI-IPS cannot produce false positives and there is no false positive rate for the system.

The GAO understandably disagrees. Accuracy is important, especially if the FBI is going to put innocent people under investigation… or overlook potentially dangerous suspects.

Without actual assessments of the results from its state and federal partners, the FBI is making decisions to enter into agreements based on assumptions that the search results may provide valuable investigative leads. In addition, we disagree with DOJ’s assertion that manual review of automated search results is sufficient. Even with a manual review process, the FBI could miss investigative leads if a partner does not have a sufficiently accurate system.

The DOJ apparently still feels a 20% chance of putting the wrong person under investigation is still acceptable. And it still believes that it's so far ahead of the privacy curve that it doesn't need to apprise the public of the potential privacy implications of its massive biometric database. The information forced out of its hands by litigants and outside agencies shows the FBI is far more interested in collection than dissemination -- that it should be able to take all it wants from the public without having to hand out anything in return.

Filed Under: accuracy, doj, facial recognition, fbi, next generation identification, ngi, privacy

The FBI Wants Speedy DNA Analysis Added To Its Biometric Dragnet

from the to-preserve-the-integrity-of-your-precious-bodily-fluids dept

The FBI's Next Generation Identification (NGI) database has been discussed here several times, thanks to its "expeditious" blend of criminal and non-criminal data, its postponed-forever Privacy Impact Assessment the agency has been promising since 2008, the limited, four-state rollout of facial recognition software with a 20% error rate, and its peculiar exclusion of DOJ/law enforcement employees from its lifelong criminal database monitoring.

It appears the FBI isn't satisfied with the wealth of biometric information it already has access to. It's grabbed everything external it can possibly get (faces, distinctive marks, fingerprints, civil/criminal records, voice recordings, iris scans [coming soon!]). Now, it's coming for what's inside you.

The FBI is preparing to accelerate the collection of DNA profiles for the government's massive new biometric identification database.

Developers of portable DNA analysis machines have been invited to a Nov. 13 presentation to learn about the bureau's vision for incorporating their technology into the FBI's new database.

So-called rapid DNA systems can draw up a profile in about 90 minutes.

DNA has been an integral part of criminal investigations for a number of years now and there's no question it has played an important role both in securing convictions and exonerating the falsely accused. But what the FBI is proposing is adding input from lab-in-a-box setups that return pass/fail DNA matches in a relative instant.

Rapid DNA analysis can be performed by cops in less than two hours, rather than by technicians at a scientific lab over several days. The benefit for law enforcement is that an officer can run a cheek swab on the spot or while an arrestee is in temporary custody. If there is a database match, they can then move to lock up the suspect immediately.

What used to take days in a secure, sterile lab now can apparently be accomplished in the "field" in a couple of hours. All technological improvements aside, this would appear to be a much less reliable method. Field drug testing kits have been available for years -- which utilize nothing more complex than chemical reactions -- and they've been shown to be far more unreliable than those utilizing them would have you believe. The same can most certainly be said about portable or on-site units wholly divorced from the normal constraints of a lab setting.

The government (so far) realizes this. That's why DNA obtained and analyzed by these units aren't included in the national DNA database. Only results from accredited public-sector laboratories are accepted. The companies manufacturing these devices are obviously interested in seeing this law changed. In the meantime, they've pushed for states to create their own DNA databases.

The FBI would like to see this changed as well, going so far as to issue a statement that is mostly wishful thinking.

FBI officials say their program does not impact any laws currently governing the operation of CODIS. Rapid DNA techniques in booking stations, “will simply expedite the analysis and submission of lawfully obtained samples to the state and national DNA databases,” [Ann] Todd, the FBI spokeswoman, said.

Except that it would impact laws governing CODIS… as they are today.

A legislative tweak is needed to allow DNA processed by a portable machine to be entered into the FBI's systems, bureau officials acknowledge.

Again, the FBI places efficiency above everything else. "Tweaking" the law to include portable devices would "expedite" the filling of the FBI's biometric database. Faster is better, even if the analysis method isn't as reliable as that performed by accredited labs. False positives/negatives are just the acceptable collateral damage of "combating crime and protecting the United States."

There's a huge backlog of untested DNA waiting for CODIS-qualified lab analysis. Offloading some of the work to private labs or portable devices sounds like a great way ease that congestion, but it actually could create more problems. If the government believes that only its chosen labs are capable of producing solid analysis, fixes like those suggested by three California Congressional reps would ask law enforcement (including the FBI) to decide which evidence goes the Gold Standard labs and what gets passed along to the lesser, unproven venues.

When presented with this set of options, law enforcement may prioritize cases badly, routing "time-sensitive" evidence through unproven but quicker analysis while sending out anything that can "wait" to the government's labs. Basically, without an across-the-board certification of all methods (with rigid testing and re-testing to ensure quality) as being equal, there's a good chance collected DNA will be treated just as prejudicially as the suspects themselves. And, if the expansion of CODIS inputs isn't handled with rigorous oversight, the chances of the guilty going free and the innocent being imprisoned increases.

Filed Under: biometrics, dna, doj, fbi, next generation identification, ngi, privacy

FBI's Biometric Program Blends Criminal And Non-Criminal Data For 'Efficiency,' Obsessed With Tracking Care Providers

from the not-discussed?-privacy-impact-or-unintended-consequences dept

As was noted here earlier, the FBI's Next Generation Identification (NGI) system has rolled out, pretty much right on schedule and well ahead of the Privacy Impact Assessment it hasn't updated since the announcement of the "system upgrade" back in 2008.

EPIC has obtained another load of documents from an FOIA request dealing with the "Rap Back" portion of the NGI system, one that provides constant monitoring of certain people -- like suspected criminals, people on parole, employees with security clearances and "trusted positions." Notably, the Rap Back program does not track employees of the criminal justice system, and nothing in what's been obtained even suggests it can be used that way.

The FOIAed documents run 202 pages [pdf link] with tons of duplicate presentation slides and long lists of fully-withheld documents. As is par for the course with sloppy fulfillments like these, the redactions are inconsistent, raising questions about the legitimacy of the exemptions used to justify removal in one place but not another. For instance, two back-to-back copies of the same slide show the FBI has an interest in redacting "DHS" [using exemption b7(e) -- "disclosure of techniques and procedures for law enforcement investigations"], but not a consistent interest.




Elsewhere, the FBI goes into more specifics as to who's included in the term "trusted positions" -- something it offers a lifetime of active monitoring for if that's what the "customer" wishes.

It is imperative that the FBI maintain a complete CHRI database in order to provide customers with information necessary to make informed decisions regarding the backgrounds of individuals whether for criminal justice purposes, noncriminal justice purposes for employment, licensing, and gun permit matters, or for purposes of national and international security. Disposition data is information pertaining to the resolutions of arrest charges or the custody or supervisory status of subjects subsequent to convictions. Disposition data is the core of the criminal history database.

Further details indicate that "trusted positions" are actually just a variety of caretakers, rather than those in positions that contain a healthy mixture of power and access (i.e., law enforcement members).

Availability of complete computerized criminal records is vital for criminal investigations, prosecutorial charging, sentencing decisions, correctional supervision and release, and background checks for licensing, purchasing of handguns, and applying for child-care positions or other responsibilities involving children, the elderly, and the disabled.

While it is definitely preferable to have someone with a clean record providing caretaking services, you have to wonder why the FBI is so focused on monitoring this specific group of employees. There's been a push to include this group in its Rap Back program pretty much since the beginning. Further down in the document, presentation slides suggest making inroads with legislators to turn the FBI's monitoring preference into an integral part of federal law.


The FBI wants officials to target legislators as "champions" and includes documentation on existing and upcoming legislation where wording could be inserted to make participation in Rap Back mandatory.

Bill Name: No Child Left Behind Act of 2007
Designation: S1775
Sponsor: Richard Burr 07/12/07
Cosponsor: 0-D, l-R (as of 08/24/07)

Sections 4222 and 4223 would require the Office of Justice Programs to develop model screening guidelines to include a criminal background check. The Secretary of Education is authorized to award grants to those entities that have conducted background checks on mentors in accordance with these guidelines. The bill does not indicate how these checks would be conducted.

Bill Name: Patient Safety Abuse Prevention Act of 2007
Designation: 1577
Sponsor: Herb Kohl 06/07/07
Cosponsor: (as of 08/24/07)

Section 3 of the bill expands the pilot program, as established under Section 307 of the Medicare Prescription Drug, Improvement, Modernization Act of 2003, to be conducted on a nationwide basis. Section 4 of the bill requires a state and FBI fingerprint criminal history background check on direct patient access applicants and employees of skilled nursing facilities and long-term care facilities. It also requires the state to develop "rap back capability. Section 5 also requires the FBI to develop "rap back" capability by January 1, 2011. The FBI would only be authorized to charge the actual costs of conducting the criminal history background check.

Elsewhere, the documents note that the FBI will be mixing its criminal and noncriminal databases -- for efficiency.

The existing LAFIS criminal and civil repositories are maintained as separate and distinct databases that do not allow for the automated transfer of records among repositories. The proposed IAFIS design change combines the records from the civil and criminal repositories into an interoperable repository. The repository design will facilitate the transition, search, addition, consolidation, modification, expungement, response generation, and file maintenance of criminal and civil information; provide the ability to search the civil records with remote latent fingerprint submissions; and support user required fingerprint search and notification capabilities. With this initiative, the records from the civil and criminal repositories will be maintained in one interoperable repository.

It also notes that in order to keep its fingerprint database up to date, it will need to pull from civil fingerprint records.


And, finally, sitting by itself, unduplicated anywhere else in the 202 duplicate-ridden pages, is a single slide dedicated to mandatory privacy considerations -- neither of which have been fulfilled even though the program is now fully operational.


There's no discussion included as to why the FBI is so focused on including caretakers and childcare providers in its lifelong monitoring program. In terms of its other stated interests (national security/counterterrorism, support for law enforcement), this seems like a much lower-level concern. While these positions would ideally be filled by citizens with clean records, constant updates from the criminal justice system seems a bit much -- especially when good employees may be forced out of work for unrelated infractions, like being arrested for exercising their First Amendment rights (filming public officials, attending protests) or marijuana possession.

It is notable that a legislative attempt to make this sort of monitoring mandatory for government contractors providing security services in war zones was shot down by the executive branch (p. 143). Apparently, it's more important to track caretakers than it is to ensure those operating security teams overseas (with a minimum of oversight) aren't former (or current) felons.

In all of the pages released, nothing deals directly with the program's privacy impact or its deliberate exclusion of criminal justice employees. No concerns are raised and no suggestions are made that law enforcement entities might do well to include their officers in the Rap Back monitoring system. And, even though the documents range from 2008-2011, there are no updated statistics that suggest the facial monitoring software has improved over its 20% error rate. Presumably, it has, but the FBI seems to feel the error rate isn't worth discussing (or releasing), even though that program actually went live in a four-state rollout three years ago.

The mixing of civil and criminal data should be the biggest concern, especially if the program pulls potential mug shots from both when seeking matches for facial recognition. There's a good reason to keep these separate. Blending both for efficiency's sake only makes the existing problem -- law enforcement's disinterest in ensuring that arrestees that were never charged or had records expunged are removed from the system -- even worse. Now, when Rap Back notifications are delivered, they have a higher potential to return data on noncriminals.

Filed Under: biometrics, fbi, next generation identification, ngi, privacy, rap back