Face Search at Scale

• NCJ Number: 252436
• Journal: Ieee Transactions on Pattern Analysis and Machine Intelligence 39(6): 1122-1136. Volume: 39 Issue: 6 Dated: 2017 Pages: 122-1136
• Author(s): Dayong Wang; Charles Otto; Anil K. Jain
• Date Published: 2017
• Length: 15 pages
• Grant Number(s): 2011-IJ-CX-K057
• Agencies: NIJ-Sponsored
• Publication Type: Research (Applied/Empirical); Report (Study/Research); Report (Grant Sponsored); Program/Project Description
• Award: Matching Forensic Sketches to Large Face Image Databases

Annotation

Since despite significant progress in face recognition, searching a large collection of unconstrained face images remains a difficult problem, the current project addressed this challenge by proposing a face search system that combines a fast search procedure, coupled with a state-of-the-art commercial off the shelf (COTS) matcher, in a cascaded framework.

Abstract

Given the prevalence of social media websites, one challenge facing computer vision researchers is to devise methods to search for persons of interest among the billions of shared photos on these websites. Given a probe face, the current study first filtered the large gallery of photos to find the top-k most similar faces using features learned by a convolutional neural network. The k retrieved candidates are re-ranked by combining similarities based on deep features and those output by the COTS matcher. The proposed face search system was evaluated on a gallery containing 80 million web-downloaded face images. Experimental results demonstrate that although the deep features perform worse than the COTS matcher on a mugshot dataset (93.7 percent versus 98.6 percent TAR@FAR of 0.01 percent), fusing the deep features with the COTS matcher improves the overall performance (99.5 percent TAR@FAR of 0.01 percent). These findings show that the learned deep features provide complementary information over representations used in state-of-the-art face matchers. On the unconstrained face image benchmarks, the performance of the learned deep features is competitive with reported accuracies. LFW database: 98.20 percent accuracy under the standard protocol and 88.03 percent TAR@FAR of 0.1 percent under the BLUFR protocol; IJB-A benchmark: 51.0 percent TAR@FAR of 0.1 percent (verification), rank 1 retrieval of 82.2 percent (closed-set search), 61.5 percent FNIR@FAR of 1 percent (open-set search). The proposed face search system offers an excellent trade-off between accuracy and scalability on galleries with millions of images. Also, a face search experiment involving photos of the Tsarnaev brothers, who were convicted of the Boston Marathon bombing, the proposed cascade face search system could find the younger brother's (Dzhokhar Tsarnaev) photo at rank 1 in 1 second on a 5 M gallery and at rank 8 in 7 seconds on an 80 M gallery. (Publisher abstract modified)