Inter-laboratory variation in interpretation of DNA mixtures

It is generally recognized that there is variation in interpretation and reporting of DNA mixtures. There has not been a large-scale independent study evaluating the extent of this variation with samples selected to be representative of the range of attributes found in actual DNA mixture casework, that includes results from (but is not limited to) current state-of-the-practice probabilistic genotyping software. This project will be a rigorous empirical evaluation of the extent of variation among forensic laboratories on the interpretation of DNA mixtures, using an approach based on the “black box” and “white box” forensic examiner studies conducted by Noblis. The study will be composed of four phases: 1) Policies and Procedures Questionnaire - Online questionnaire to assess laboratory policies and standard operating procedures (SOPs) relevant to DNA mixture interpretation, interpretation software used, and parameter settings. 2) Scenario Questionnaire - Online questionnaire presenting a number of casework-derived scenarios (without DNA data), asking participants to assess how they would conduct analysis for each scenario. 3) Number of Contributors Subtest - Assessment of suitability and number of contributors, given electropherogram data (14 DNA mixtures). 4) Interpretation Subtest - Report of interpretations, given electropherogram data provided with DNA profiles of potential contributors (7 DNA mixtures). The use of subtests provides complementary approaches to collecting data, in order to differentiate among the various potential sources of variability and avoid confounding their effects. A small working group of volunteers from multiple laboratories will be created to provide input on study details. The study will evaluate the current state of the practice in operational interpretation of DNA mixtures, and will not be restricted to specific products or statistical approaches. The project will expand on the results and lessons learned from studies conducted to date and will address concerns raised by the 2016 report from the President’s Council of Advisors on Science and Technology. Participation is open to all forensic laboratories that conduct DNA mixture interpretation as part of their SOPs; non-U.S. laboratories are welcome to participate if they report interpretations in English. Participation in the study requires 10 the participants to agree to use the same diligence in performing these analyses as used in operational casework, and to use their laboratory’s SOPs in performing these analyses. After publication, the study dataset will be provided to NIJ for public release. NCA/NCF