Goodness-Of-Fit Tests and Function Estimators for Receiver Operating Characteristic (ROC) Curves: Inference From Perpendicular Distances

• NCJ Number: 240957
• Author(s): R. Bradley Patterson
• Date Published: 2012
• Length: 284 pages
• Grant Number(s): 2011-CD-BX-0124
• Agencies: NIJ-Sponsored
• Publication Type: Report (Study/Research)
• Award: Advancing The Use Of Receiver Operating Characteristic (ROC) Curves To Measure Performance Of Forensic Methods

Annotation

The uniqueness of this research is its use of differences in a perpendicular direction for goodness-of-fit tests, function estimators, and confidence regions for the receiver operating characteristic (ROC) curve.

Abstract

The ROC curve assesses the performance of classification methods used to identify observations by type. The ROC curve may indicate the performance of a forensic DNA test that classifies genetic samples as to whether or not they match. In order to produce an ROC curve, a sample of observations with known classes must be available. Often, the true ROC function may be a continuous curve that remains unknown. Statistical methods lead to smooth, parametric estimates of the true ROC function from the jagged empirical one. In order to measure the quality of smooth, parametric ROC function estimates, this research project developed a unique statistical goodness-of-fit test. Motivated by that development, the researchers have also created two original estimators of parametric ROC functions. Statistical inference with the ROC curve has traditionally been based on differences between empirical and parametric curves in the vertical direction. What is new and unique in the current work is the use of differences in a perpendicular direction for goodness-of-fit tests, function estimators, and confidence regions for the ROC curve. Working along directions perpendicular to parametric binormal ROC curve, researchers designed a goodness-of-fit test similar to existing statistics based on the empirical distribution function (EDF) for a single random variable. Through large simulations, this new test has exhibited uniformity of p-values under the null hypothesis, and consistency. The original function estimators developed in this work minimize differences in perpendicular directions between empirical and parametric ROC curves. 21 tables 20 figures, and approximately 70 bibliographic listings