Optimizing Juvenile Assessment Performance

OJJDP’s Field Initiated Research and Evaluation Program supports innovative and methodologically sound research and evaluation efforts that inform policy and practice consistent with OJJDP’s mission to advance effective delinquency prevention and juvenile justice system interventions. In nearly every state and in the vast majority of juvenile justice agencies, risk assessments are incorporated into diversion, case management, supervision, and placement practices. Despite two decades of use within the juvenile justice system, little research regarding the methods of risk assessment development is discussed or translated to the field and practitioners. Many of the contemporary tools used today are implemented off-the-shelf, meaning that tools were developed with a specific set of methods, selecting and weighting items used in the prediction of a specified sample of youth. Yet subsequently these instruments are used for prediction on populations and jurisdictions in which the instrument was not designed to serve. What is not known is how the various designs, methods and circumstances of tool development impact the predictive performance when adopted by a jurisdiction. This project will seek to isolate, test, and evaluate the relative impact of seven notable risk assessment development variations, namely: (1) item selection technique, (2) weighting, (3) gender-specificity, (4) race-ethnicity neutrality, (5) outcome specificity, (6) prediction duration, and (7) jurisdiction variation. It is anticipated that each tested variation will provide a small-to-substantial impact on the predictive performance of assessments evaluated. The research team will partner with state agencies providing assessment and recidivism data, including Delaware, Iowa, Oregon, Utah, Washington, and Wisconsin. Based on each site’s preliminary estimates, the researchers expect to gather over 500,000 youth assessments and associated recidivism outcomes as a part of the final unified multi-state dataset. Using a large, multi-state sample of youth assessed using the same assessment, the researchers will construct models that will serve as relevant comparisons for each seven risk assessment development approaches outlined. A robust and varied statistical approach is planned, examining bivariate and multivariate Least Shrinkage and Selection Operator (LASSO) regression modeling techniques. In addition, several sampling approaches will be used to form comparisons between gender and race/ethnic sub-populations. Furthermore, comparisons will be made between the six-site unified sample and models created to capture individual site differences. Anticipated findings will provide OJJDP, the field and practitioners using and implementing risk assessments with an understanding of development variations and methods of tool customization, optimizing predictive performance for their local jurisdiction and youth population. Additionally, each of the six sites will be provided with a product/deliverable, representing a customized tool update, based on their provided data and current findings, with the ability to immediately be implemented and improve risk assessment accuracy. To translate findings to the field, a best practices guide will be developed, allowing the field to make similar adjustments to their own tools in an effort to optimize predictive performance. Note: This project contains a research and/or development component, as defined in applicable law, and complies with Part 200 Uniform Requirements- 2 CFR 200.210(a)(14). CA/NCF