Deep Multi-Factor Forensic Face Recognition

In forensic science, an automatic face recognition system through computer vision methods can help in narrowing down the suspects list, while the original search space could be millions of mug shot photos from the police department. A case study about Boston bombings has demonstrated that with certain face recognition techniques, the search space of the suspects can be reduced to 1/100. However, until now, there is no working face recognition system that has been accepted within the judicial system. The major issue is the unstable system performance due to internal factors, e.g., aging, and external factors, e.g., image resolution/modality, illumination, pose. Although there are substantial works proposed recently to address these negative uncertain factors, some of which achieved significant progress, especially for the deep learning methods, little has been discussed under the context of forensic science.

In this proposal, we propose a novel deep learning framework for multi-factor forensic face recognition to address the negative factors mentioned above. The technical merits of deep learning are it can well utilize large-scale training data from millions of mug shot photos, and can adapt to complex problems, e.g., “sketch-mug shot” face match, with flexible model structure. This essentially mimics the cognitive process of human being, which processes the visual information layer by layer. Our novel deep learning model is able to normalize the visual features of the face from coarse to fine, to compensate for different poses, modality, low image quality, occlusions, and noisy labels. This project is leveraged by a close collaboration with Henry C. Lee Institute of Forensic Science for practical case study, forensic evaluation, and law enforcement training. This work could affect and benefit broad research and education community for public security. The proposed computer vision based forensic face recognition solution will facilitate narrowing down the suspects list, which not only substantially reduces the human efforts, but also improves the cross-modality face match accuracy. As a result, it will reduce the occurrence and impact of violent crimes, prevent violent crime or to assist victims. It could be broadly applied and deployed in the area where public security is concerned, i.e., transit system in subway, rail station, or airport, road monitoring system at the crossing, and surveillance system at the public facility. The outreach will strengthen interest in science and engineering careers of young scholars and cultivate the desire to perform higher level education and research.