Fed watchdog raises questions about FBI facial recognition accuracy, privacy

GAO says the FBI has access to nearly 411 million photos

gao-fed-watchdog-raises-questions-about-fbi-facial-recognition-accuracy-privacy
Credit: Reuters

The FBI needs to get a better handle on accuracy and privacy issues its facial recognition technology has brought to the law enforcement community.

Congressional watchdogs at the Government Accountability Office this week said the current FBI use of face recognition technology “raises potential concerns regarding both the effectiveness of the technology in aiding law enforcement investigations and the protection of privacy and individual civil liberties.”

+ More on Network World: Quick look: Cisco Tetration Analytics | Cisco platform lets IT rein-in disruptive data center operations, security, applications +

The FBI has access to nearly 411 million photos through its internal unit called Facial Analysis, Comparison and Evaluation (FACE) FACE Services that can search or request to search databases owned by the Departments of State and Defense and 16 states, which use their own face recognition systems with drivers license photos chiefly. That number also includes the 30 million inside the FBI’s own face recognition system called Next Generation Identification-Interstate Photo System (NGI-IPS), the GAO stated.

fbi face recognition GAO analysis of FBI documentation

The majority of photos enrolled in NGI-IPS are voluntary submissions from 18,000 federal, state, local, and tribal law enforcement entities. About 70%of the photos in NGI-IPS were criminal mugshots stored in an older FBI system that was not searchable with face recognition technology until the development of NGI, the GAO noted.

An additional problem with NGI-IPS however, is that the “FBI does not conduct operational reviews that would assess the accuracy of face recognition searches on NGI-IPS, it risks spending resources on a system that is not operating as intended and also may miss opportunities for improving the system. Also, by taking steps to assess whether the systems operated by external partners are sufficiently accurate for use by FACE Services the FBI would have better assurance that the systems they use are appropriate for its use, increasing the odds of identifying suspects for active investigations while protecting privacy.”

Further, FBI officials stated that they have not assessed how often NGI-IPS face recognition searches erroneously match a person to the database, the GAO stated.

Some other interesting facts from the GAO report:

  • There are 29 trained biometric images specialists in FACE Services who receive requests from the FBI field offices, investigative divisions, and FBI overseas offices to support active FBI investigations. When an FBI agent submits a probe photo to FACE Services, a biometric images specialist searches NGI-IPS for matches to the probe photo. However, agents may request that the biometric images specialist also search the face recognition systems of FBI’s external partners. The total number of face photos available in all searchable repositories is over 411 million, and the FBI is interested in adding additional federal and state face recognition systems to their search capabilities.
  • From August 2011 through December 2015, FBI agents have requested almost 215,000 searches of external partners’ databases. Of these requests, about 36,000 have included searches on state driver’s license databases.
  • FBI officials have not conducted an operational review of NGI-IPS. As a result, they have not assessed the accuracy of face recognition searches of NGI-IPS in its operational setting—the setting in which enrolled photos, rather than a test database of photos—are used to conduct a search for investigative leads. According to FBI officials, the database of photos used in its tests is representative of the photos in NGI-IPS, and ongoing testing in a simulated environment is adequate. However, according to the National Institute of Standards and Technology, as the size of a photo database increases, the accuracy of face recognition searches performed on that database can decrease due to lookalike faces.
  • A July 2012 Electronic Frontier Foundation hearing statement, false positives can alter the traditional presumption of innocence in criminal cases by placing more of a burden on the defendant to show he is not who the system identifies him to be.
  • Although the FBI has tested the detection rate for a candidate list of 50 photos, NGI-IPS users are able to request smaller candidate lists—specifically between 2 and 50 photos. FBI officials stated that they do not know, and have not tested, the detection rate for other candidate list sizes. According to these officials, a smaller candidate list would likely lower the detection rate because a smaller candidate list may not contain a likely match that would be present in a larger candidate list.
  • A face recognition system with a low detection rate may not sufficiently provide biometric images specialists with matches when they exist in the partner’s system (i.e., the person in the probe photo has a photo in the partner system, but it is not returned as part of the candidate list). As a result, the FBI could miss investigative leads that could have been revealed if the partner system had a better detection rate.

While the FBI/Department of Justice agreed with some of the GAO report it disagreed with the overall tone.

“Despite the voluminous amount hours spent by the FBI staff providing face recognition information to the GAO, the FBI believes GAO staff does not fully appreciate the nature of its face recognition service as being utilized for investigative leads only and not positive identifications. Standards for accuracy can be reasonably different for systems that provide investigative leads as opposed to systems that provide positive identification thus explaining the DoJ’s disagreement with some of the GAO recommendations,” the DoJ wrote in response to the report.

Check out these other hot stories:

Cisco platform lets IT rein-in disruptive data center operations, security, applications

Quick look: Cisco Tetration Analytics

FBI: Business e-mail scam losses top $3 billion, a 1,300% increase in since Jan.

NASA describes out of this world data glitch that almost finished its planet hunter

National Intelligence office wants to perfect the art of security deception

US intelligence unit to advance management of virtual desktop security, systems

Cisco: IP traffic will surpass the zettabyte level in 2016

OpenSwitch finds critical home at Linux Foundation

Feeling jammed? Not like this I bet

FBI: Extortion e-mail, tech support scam-bags turning up the heat

DARPA wants to find the vital limitations of machine learning

To comment on this article and other Network World content, visit our Facebook page or our Twitter stream.
Must read: Hidden Cause of Slow Internet and how to fix it
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.