Application of likelihood ratios and optimal decision thresholds in fire debris analysis based on a partial least squares discriminant analysis (PLS-DA) model

• NCJ Number: 300991
• Journal: Forensic Chemistry Volume: 16 Dated: 2019
• Author(s): A. Allen; et al
• Date Published: 2019
• Grant Number(s): 2015-DN-BX-K051
• Agencies: NIJ-Sponsored
• Award: Substrate Pyrolysis Database Expansion, Evaluation and Use

Annotation

This project used a set of 9,000 computationally generated (in-silico) total ion spectra comprising samples containing ignitable liquids from ASTM E1618-14 classes mixed with pyrolyzed substrates (class IL) and samples containing a mixture of pyrolyzed substrates without ignitable liquid contribution (class SUB) to generate a partial least squares discriminant analysis (PLS-DA) model.

Abstract

The training data set was balanced in IL and SUB class composition and the IL class samples were composed of a uniform distribution of ignitable liquids from the ASTM E1618-14 defined classes. The number of components in the optimized PLS-DA model, 13, was determined by optimizing the area under the receiver operating characteristic curve (ROC AUC) as a function of the number of components. Ten cross validation folds had an average ROC AUC of 0.930 ± 0.006. The PLS-DA model was further challenged with a uniformly distributed and balanced IL/SUB validation set of 1,000 computationally mixed total ion spectra, giving an ROC AUC 0f 0.93. The model which had been validated on computationally generated total ion spectra was tested on a set of “real” fire debris samples from large-scale burn tests that had been independently evaluated by an “informed analyst”. The results from this study demonstrate the application of a chemometric model to the forensic analysis of fire debris data and emphasize the connection between quantified strength of the evidence and categorical decisions based on a defined operational decision point on the ROC curve. The success of the PLS-DA model demonstrates the feasibility of applying chemometric methods to the analysis of fire debris as a means of introducing probabilistic statements of evidentiary value to the court. (publisher abstract modified)