Except for identical twins, no two people have the same DNA. The genetic code that is found in nearly all cells of the human body can be collected from people’s skin, blood, saliva, and bone to create a profile (or “genetic fingerprint”) to identify, or eliminate, potential suspects in a forensic investigation.
DNA technology has come a long way. In the not-so-distant past, we needed a fair amount of either blood or semen to create DNA profiles. Now, with highly sensitive techniques, DNA can be picked up from just a few cells left behind. Objects such as steering wheels, doorknobs, guns, and even clothing often yield “touch DNA” evidence. Current DNA technology is sensitive enough to analyze DNA samples that include contributions from more than one individual, or so called “DNA mixtures.”
DNA mixtures have, however, proven difficult to interpret due to variation in methods, sample quality issues, and interpretation problems (Box 1). The 2016 President’s Council of Advisors on Science and Technology (PCAST) called on researchers to “develop methods with enough standardized and quantifiable detail that they can be performed by either an automated system or human examiners exercising little or no judgement.”[1] And more recently, researchers from the National Institute of Standards and Technology (NIST) have reiterated that these problems continue to plague DNA mixture analysis.[2]
| Issue | Example |
|---|---|
| Method | Lack of consensus regarding standard methods and protocols. |
| Sample Quality | Low concentration of DNA. Allelic dropout (from the DNA amplification process). Stutter (from the amplification of short tandem repeats in DNA). Overlapping alleles between samples (due to multiple contributors). Imbalanced contributor ratios (which can mask some results). |
| Analysis | Use of qualitative versus quantitative methods. Type of statistics applied. Role of software and computational programs. |
NIJ-funded researchers from the Defense Forensic Science Center sought to improve mixed DNA analysis, eager to understand the amount of interpretation variation that exists in the analysis of complex DNA samples, both within and between laboratories. First, they evaluated the current state of the practice in interpretation of DNA mixtures. Then, to objectively assess the variation in forensic DNA interpretation, they created new statistics to quantify interpretation variability.
The researchers created new ways to measure the variability in interpretations of DNA mixtures:
- Between examiners within the same laboratory (intra-laboratory variability), where protocols and training were expected to be similar and variation was expected to be low.
- Among different laboratories (inter-laboratory variability), where protocol and training differences are expected to be greater and variation is therefore predicted to be greater.
Significant Interpretation Variation Found
Traditionally, a DNA profile (or “genotype”) is created for individuals using short tandem repeats (STRs), which are short sequences of DNA that repeat a number of times. The number of times a sequence repeats varies from person to person, and this variation can be used to tell people apart. By evaluating multiple STR regions (also known as “loci”) for an individual, a DNA genotype can be created.
The researchers created mixture samples comprising two DNA sources and three DNA sources, with differing ratios of DNA from the contributors, as shown in Box 2. The researchers generated DNA sample profiles from each mixture and sent the uninterpreted raw data files to the participating forensic laboratories. They included the threshold parameters to be used for analysis and a spreadsheet with specific instructions for entering their interpretations and comments.Fifty-five laboratories with 189 examiners returned completed questionnaires and worksheets. The quantitative analysis focused on laboratories with at least five participating examiners.
| Mixture | Number of Contributors |
Ratio | Reference Sample Provided |
|---|---|---|---|
| 1 | 2 | 3:1 | No |
| 2 | 2 | 2:1 | Yes |
| 3 | 2 | 3.5:1 | No |
| 4 | 2 | 4:1 | No |
| 5 | 3 | 4:1:1 | Yes |
| 6 | 3 | 1:1:1 | No |
Researchers calculated the quality of an examiner’s interpretations by comparing the examiner generated genotypes to the true, known genotypes of each contributor in a mixture. The new accuracy and precision metrics created by the researchers — dubbed the Genotype Interpretation and the Allelic Truth metrics — identified the range of variation in DNA mixture interpretation within the forensic DNA community, as represented by the participating labs.
To capture patterns of variability, they calculated the Genotype Interpretation and Allelic Truth metrics:
- At each locus of a mixture.
- For an individual contributor in a mixture.
- By overall mixture (including all contributor genotypes of a mixture).
- By laboratory.
- By grouping laboratories by a particular jurisdiction (local, state, federal, and international/other).
Results indicated that:
- Significant intra- and inter-laboratory interpretation variation exists.
- Inclusion of a known reference DNA profile has a marked positive effect on interpretability.
- Two-person DNA mixtures are generally interpretable.
- Three-person mixtures are generally beyond the scope of protocol limits for most participating examiners.
Reference Sample and Peak Height Highly Impactful
The results indicated that although most labs could interpret two-sample mixtures (if the sample concentrations were above the lower limit at which DNA could be distinguished from noise), the majority had difficulty in interpreting three-sample mixtures. The inclusion of a reference sample and the ability to examine samples that were well above the threshold of detection had a large impact on the ability of the examiners to interpret a DNA mixture. Some laboratories were able to interpret the difficult three-person mixtures and resolve genotypes for each contributor with nearly equivalent contributor ratios, even under challenging conditions (like having no reference sample).
These results from a handful of labs indicate that accurate and precise interpretations were possible, with accuracy and precision being highly correlated, and emphasize the need for ongoing training and benchmarking within laboratories, as well as the necessity for dissemination of best practices among laboratories.
A Benchmark for Interpretation Performance?
With the amount of weight that is given to DNA evidence in court cases, knowing the amount of variability that exists in DNA interpretation analysis is critical, to give as much information as possible to lab technicians, criminal investigators, juries, attorneys, judges, and anyone who might be invested in the use of DNA in forensics.
The metrics developed by the Defense Forensic Science Center were instrumental in evaluating the variation in interpretations of mixed samples and should serve to spur improvements in the current state of forensic DNA training and quality control. The quantitative data and new measures can be used to benchmark an examiner’s performance, to determine mixture interpretation limitations within a laboratory, and to assess whether a new method implemented in a laboratory yields improved precision and accuracy over previous methodologies, since the metrics can be applied in a variety of ways (e.g., at each locus of a mixture, for an individual contributor in a mixture, by overall mixture, by laboratory, and by grouping laboratories defined by a particular jurisdiction). Ultimately, this should increase our confidence in the reliability of DNA mixture interpretation and the quality of data used in criminal investigations.
About This Article
The work described in this article was supported by NIJ award number 2013-DNR-5042 awarded to Defense Forensic Science Center.
This article is based on the grantee report “DNA Mixture Study: Novel Metrics to Quantify Intra- and Inter-laboratory Variability in Forensic DNA Mixture Interpretation” (pdf, 30 pages), by Emily Rogers, Georgia Institute of Technology; Roman Aranda IV, Defense Forensic Science Center, Forensic Exploitation Directorate, Philippa M. Spencer, Defence Science and Technology Laboratory; and Denise R. Myers, Defense Forensic Science Center, Forensic Exploitation Directorate.