Forensic Footwear Reliability: Part II - Range of Conclusions, Accuracy, and Consensus

• NCJ Number: 255350
• Journal: Journal of Forensic Sciences Dated: September 2020
• Author(s): Nicole Richetelli; Lesley Hammer .; Jacqueline A. Speir
• Date Published: September 2020
• Length: 12 pages
• Grant Number(s): 2016-DN-BX-0152
• Agencies: NIJ-Sponsored
• Publication Type: Research (Applied/Empirical); Report (Study/Research); Report (Grant Sponsored); Program/Project Description
• Award: Understanding the Expert Decision Making Process in Forensic Footwear Examinations: Accuracy, Decision Rules, Predictive Value and the Conditional Probability of an Outcome

Annotation

This article, the second of a three part series, presents the findings of a reliability study that assessed the extent of agreement between forensic footwear examiners in the United States.

Abstract

Between February 2017 and August 2018, West Virginia University conducted a reliability study to determine expert performance among forensic footwear examiners in the United States. Throughout the study's duration, 70 examiners each performed 12 comparisons and reported a total of 840 conclusions. In order to assess the accuracy of conclusions, the similarities and differences between mated and non-mated pairs were evaluated according to three criteria: (i) inherent agreement/disagreement in class, wear, and randomly acquired features; (ii) limitations as a function of questioned impression quality, clarity, and totality; and (iii) adherence to the Scientific Working Group for Shoeprint and Tire Tread Evidence (SWGTREAD) 2013 conclusion standard. Using these criteria, acceptable/expected categorical conclusions were defined. Preliminary results from this study are divided into a series of three summaries. This manuscript (Part II) reports accuracy and reproducibility. For mated pairs, accuracy equals 76.3 percent +/- 13.0 percent (median of 78.6 percent, and a 90 percent confidence interval between 72.2 percent and 80.0 percent). For non-mated pairs, accuracy equals 87.4 percent +/- 9.24 percent (median of 91.4 percent and a 90 percent confidence interval between 84.7 percent and 89.8 percent). In addition, the community assessed agreement (denoted by IQR) of reported results equals the research team's accepted/expected conclusions for 10 out of 12 comparisons. In terms of reproducibility, the 90 percent confidence interval for consensus was computed and found to equal 0.71-0.86 (median of 0.77) for the combined dataset. Although based on a limited sample size, these results provide a baseline.