Black Box Evaluation of Hand Detail Image Comparisons

This proposal involves conducting a comprehensive, multiphasic study to characterize the emerging forensic discipline of hand image comparison. This study will provide essential empirical data to laboratory managers, practitioners, standards organizations, and the legal system regarding hand image comparisons, including a detailed understanding of the capabilities, variability, and limits of examiners; the associations between examiners’ abilities and their training and experience; and the extent to which a single examiner’s conclusion is accurate and can be reproduced by a second examiner. The results of this study can be leveraged to support improvements in methodology, standardization, training, and practice, and ultimately inform the scientific foundation of hand image comparison for forensic science.
The proposed study includes five phases:
Phase 1: Hand Comparison Working Group—In Phase 1 the study team will create the Hand Comparison Working Group (HAND-WG), which will be a small volunteer working group of image comparison practitioners and researchers. The HAND-WG will be consulted frequently to assist in determining in detail the approaches used throughout the study.
Phase 2: State of the Discipline Questionnaire (SDQ)—In Phase 2 the study team will develop and conduct a survey designed to characterize the current state of the discipline for hand image comparisons.
Phase 3: Hand Imagery Reference Dataset—In Phase 3 the study team will collect a large scale, ground-truth attributed Hand Imagery Reference Dataset, a reference dataset of hand comparison imagery collected under a variety of controlled conditions. The Hand Imagery Reference Dataset will be publicly released at the conclusion of the study.
Phase 4: Black Box Evaluation of Hand Image Comparison Decisions—In Phase 4 the study team will conduct a Black Box Evaluation of Hand Image Comparisons reported by practicing examiners with varying levels of hand image comparison expertise. The study will evaluate the decisions in terms of accuracy (correctness) and reproducibility (inter-examiner consistency).
Phase 5: Report of Recommendations—In Phase 5 the study team will work with the HAND-WG to evaluate the results of Phases 2-4 and develop a Report of Recommendations with particular focus on lessons learned, implications, and recommendations regarding existing and proposed standards, standard operating procedures, training, and quality assurance.
The proposed work will be conducted by Noblis, a nonprofit research company with a proven history of success in conducting forensic evaluations of examiners, in collaboration with image comparison subject matter experts. CA/NCF