This study evaluated novel multimodal techniques for automated analysis of police body-worn camera (BWC) recordings to assess officers' adherence to procedural justice principles during community interactions. Funded by the National Institute of Justice (Award No. 2020-R2-CX-0010), this collaborative research involved the National Policing Institute, Polis Solutions, the Caruth Policing Institute at the University of North Texas at Dallas, and the Dallas Police Department.
The project pursued two primary objectives: first, to develop TrustStat, an artificial intelligence-based software platform capable of automatically evaluating BWC footage for procedural justice metrics; and second, to validate the automated ratings by comparing them with evaluations conducted by human coders representing three distinct stakeholder groups. The research sought to answer critical questions about whether perceptions of procedural justice differ among community members, university faculty and graduate students, and police supervisors, and whether automated video analytics align with human evaluations across these groups.
The methodology employed a three-phase approach. Initially, the Polis Solutions team developed TrustStat by reviewing existing coding instruments from structured social observation studies and applying natural language processing, computer vision, and multimodal analysis techniques. The system analyzes verbal and non-verbal cues from BWC footage, identifying behaviors aligned with four dimensions of procedural justice: neutrality, participation, trust, and respect. Subsequently, researchers developed a coding instrument for human evaluators structured around these same dimensions, utilizing a seven-point Likert scale to capture the dynamic nature of police-community interactions. Finally, the team conducted a systematic comparison of TrustStat evaluations with ratings from 66 human coders (30 community members, 13 police supervisors, and 23 faculty members and graduate students) who evaluated ten purposively selected BWC videos from the Dallas Police Department.
Statistical analyses using the Kruskal-Wallis test revealed two significant findings. First, despite diverse demographic backgrounds, the three groups of human coders demonstrated remarkable consistency in their procedural justice ratings across videos, with median scores ranging from 90.5 to 93.0 and no statistically significant differences among groups. This finding challenges assumptions that different stakeholder groups would evaluate police performance fundamentally differently and suggests that procedural justice concepts can be reliably quantified by diverse evaluators with minimal training. Second, TrustStat's evaluations aligned closely with human coder assessments, producing a median rating of 89.49 that did not differ significantly from any human coder group. Pairwise comparisons using Dunn's test confirmed no statistically significant differences between TrustStat and individual coder groups.
The implications for law enforcement are substantial. TrustStat and similar automated BWC analysis tools offer agencies the capacity to systematically assess officer performance based on observable criteria related to community interactions, moving beyond reliance on subjective supervisory reports or low-frequency events such as citizen complaints. By enabling comprehensive, data-driven evaluation of the estimated 200,000 monthly BWC videos recorded by agencies like the Dallas Police Department, such technology has the potential to transform performance management, inform training programs, support early warning systems, and ultimately enhance police-community relations and legitimacy.
The study acknowledges limitations including geographic concentration in Dallas, convenience sampling of coders, and the absence of direct feedback from community members featured in the videos. Future research should expand to diverse geographic locations, implement random coder selection, and incorporate mechanisms to capture the lived experiences of individuals directly involved in police interactions captured on BWC footage.
(Author abstract provided.)
Downloads
Similar Publications
- Development and Validation of a Retrospective Self-Report Measure of Childhood Neglect
- Between and within-person relations between psychological wellbeing and distress in adolescence: A random intercept cross-lagged panel examination
- Minimizing inhibition of PCR-STR typing using digital agarose droplet microfluidics