Blind Collaborative Justice: Testing the Impact of Expert Blinding and Consensus Building on the Validity of Forensic Testimony

The National Academy of Sciences (NAS) conducted a review of the field of forensic science and the use of forensic expert testimony and came up with two potential policies to limit expert bias: blinding and consensus building. Blinding consists of forensic experts presenting testimony without knowledge of who is soliciting the testimony. Consensus building refers to a process in which multiple forensic experts will arrive at a consensus regarding the probative nature of the forensic evidence. In light of this, the RAND Corporation will conduct a test of the ability of these two innovations to produce accurate and unbiased testimony.
RAND will conduct an experiment that randomly assigns the blinding representation and the receipt of expert consensus feedback in two separate treatment conditions. To accomplish this, RAND will first develop hypothetical case summaries adapted from actual cases where forensic evidence was presented. A questionnaire will ask respondents questions on the quality and probative nature of the forensic evidence.
RAND will then draw on its approximately 1,200 full and adjunct members to develop an inter-disciplinary expert panel to review the cases. Using RANDs Delphi method for consensus building, an online facilitator will pose questions to direct the expert panel towards consensus on responses to the questionnaire.
Next, RAND will recruit participants from forensic practitioner associations to review the case and answer an online questionnaire. The participants will be randomized into three conditions: 1) Representing the Prosecution; 2) Representing the Defense; 3) Not informed which side they are representing (blinded condition). The forensic experts will provide their initial answers to the questionnaire. The forensic experts will then be randomized into an additional treatment group that is presented with feedback from the expert panel consensus regarding the answers to the questionnaire and a control group that does not receive this feedback. The consensus and non-consensus groups will be matched based on forensic experience and propensity to provide testimony for the prosecution or the defense.
An Analysis of Variance (ANOVA) will be used to test if the blinding condition and/or receiving the consensus feedback produced significantly different answers compared to representing the prosecution, defense, or whether the respondent received the consensus feedback. The order the cases were reviewed and the years of experience of the respondents will be controlled. RAND estimated they would need at least 300 successful respondents to identify treatment effects.
ca/ncf