A Black Box Study of the Accuracy and Reproducibility of Tire Evidence Examiners' Conclusions

The accuracy and reproducibility of conclusions by forensic tire evidence examiners have not been systematically and rigorously assessed. The paucity of discipline specific research has resulted in a critical gap in the scientific basis for court testimony in legal proceedings, creating a significant concern for the criminal justice system. The proposed study is an empirical assessment of the accuracy and reproducibility of tire evidence examiners’ conclusions. This study will further assess findings as a function of examiner experience, education, and training, provide essential information to laboratory managers, practitioners and the legal system about the discipline of identifying and assessing tire evidence, and inform an understanding of the capabilities, variability, and limits of tire evidence examiners. Our target participation level is 50 or greater practicing tire track evidence examiners. Each participant will perform a total of 50 tire track evidence comparisons. To approximate operational casework evidence, known and questioned impressions (test samples) will be collected under controlled conditions to ensure a broad distribution of qualitative and quantitative characteristics. To measure accuracy on the definitive conclusions (identification and exclusion), we will report the true positive rate, true negative rate, false positive rate, false negative rate, positive predictive value, and negative predictive value. To address the multilevel conclusion scale, we will also report correct and incorrect associations using definitions developed in our previously conducted black box studies in which multi-level conclusion scales were also employed, including correct positive association rate, correct negative association rate, incorrect positive association rate, and incorrect negative association rate. We define accuracy as the correctness of a determination (variation of a measurement from ground truth). A Nobis custom, web accessible software will be employed as the primary data collection instrument for this study. Noblis and subject matter expert (SME) Jan LeMay will design and execute the proposed study. The study design is modeled after and based on the lessons learned from conducting previous black box studies in the fingerprint, footwear, handwriting, bloodstain, and firearms disciplines. Results will be published in a peer-reviewed journal and presented at forensic conference(s). The resulting dataset of tire impression imagery and associated anonymized examiners’ conclusions will be delivered to NIJ for use in future studies. Note: This project contains a research and/or development component, as defined in applicable law, and complies with Part 200 Uniform Requirements - 2 CFR 200.210(a)(14). CA/NCF