Content Advisory
Please be aware that this content contains descriptions of violence that may be distressing to some viewers.For example, treatment dilution and treatment migration are common forms of randomization implementation failure in field experiments, and a review of the criminological literature on experiments reveals a lack of consensus on how these problems should be handled when evaluating treatment effects. In addition, issues related to statistical power and desired sample size remain unresolved. Given the relatively long history of dealing with these analytical issues in medicine, literature from that field is reviewed to provide additional insights on the dilemmas created by various design failures in randomized field experiments. The impact of design failures on experimental evaluations is considered to be potentially significant, and the author suggests that the way in which social policy experiments are conducted needs to be critically examined. 34 references