The IMPACTT of a Patrol Officer: Evaluating Productivity Metrics

Patrol officers perform a large number of diverse community services that both police researchers and police agencies have struggled to validly quantify, implement, and evaluate.[1] Although numerous studies have admirably described the duties and responsibilities of patrol officers,[2] far fewer studies have attempted to quantify and evaluate these activities using metrics beyond raw outputs, such as arrests or citations.

There are many reasons for the lack of sophisticated metrics of patrol officer productivity. Lack of data has traditionally stymied researchers, but so has the increasing complexity of a patrol officer’s job — as well as differences in communities’ geography and public safety priorities. Yet despite these methodological challenges and differences across jurisdictions, citizens expect patrol officers to use tax dollars and resources efficiently and productively, and they rightfully expect law enforcement agencies to evaluate the performance of patrol officers.

This is not merely an academic question but a sobering quandary for police agencies and communities. A perceived misuse of time and resources may negatively impact public perceptions of law enforcement, and studies suggest that if citizens perceive police officers as incompetent or unproductive, their trust in and willingness to cooperate with officers may suffer considerably.[3] A lack of valid, standardized productivity metrics may also cause expectations to vary among line-level officers and police supervisors, potentially leading to unexpected criticism and denied pay increases during annual evaluations.

This article advances the discussion on patrol officer productivity by discussing traditional methods for evaluating productivity, identifying recurrent issues concerning productivity metrics, examining innovative methods for evaluating patrol officers, and introducing new guidelines for those who create, use, and analyze patrol officer productivity metrics.

Traditional Patrol Officer Productivity Metrics

Productivity scholars have operationalized productivity in several ways, but the basic concept usually includes two dimensions: efficiency and effectiveness.[4] Efficiency denotes how a person or organization generates an output using the least possible resources, whereas effectiveness measures the quality of a person’s or organization’s outputs. Historically, research analyzing patrol officer productivity has focused on efficiency measures, mainly because agencies generally track and measure activity through raw outputs, such the number of arrests or citations.[5]

However, there is considerable variation in the police literature in how police agencies and researchers measure officer efficiency, and even more broadly, what constitutes productivity. Psychological studies have analyzed police productivity using supervisor evaluations[6] as well as the number of reprimands and citizen complaints received.[7] Other police researchers have approached patrol officer productivity by examining arrest rates,[8] investigative inquiries and quarterly performance evaluations,[9] clearance rates and crime reduction,[10] and traffic citation issuing rates.[11] Although studies have identified and correlated many different predictors of productivity, there is much less focus on whether these measures validly capture the diverse range of patrol officer activities.

There are two main reasons for the substantial variation in how law enforcement agencies and researchers have analyzed patrol officer productivity. First, in-depth law enforcement data about specific officers (beyond their arrests and other readily retrieved outputs) are often not widely available.[12] Second, if these data are available, they may contain raw outputs for a limited number of police activities, such as arrests or citations, rather than detailed information about a wide array of activities, such as directed patrols, community presentations or meetings, and assisting other officers on calls for service. In short, researchers have historically been limited by the lack of availability and depth of data on patrol officer activity as well the subjectivity and difficulty in measuring and analyzing these data.[13]

However, as agencies have improved their methods for capturing a wider variety of patrol officer activity and as fruitful partnerships between researchers and law enforcement agencies have flourished, there has been a renewed interest in developing and improving productivity metrics for patrol officers. Yet despite advances in technology, methodology, and collaborative research between scholars and police agencies, several threats routinely undermine the validity of any measure of patrol officer productivity.

Recurrent Threats to the Construct Validity of Productivity Measures

One of the most significant threats to productivity metrics involves Goodhart’s Law, often phrased as: “When a measure becomes a target, it ceases to be a good measure.”[14] Goodhart’s Law may be especially problematic for the law enforcement profession because productivity is frequently measured in raw outputs, such as citations or arrests.[15] For example, if officers believe their productivity is largely measured in arrests, their patrol activity might solely consist of making arrests, potentially even in situations where arrest is not supported by probable cause, policy, or proper use of discretion. The declining quality of police service is one possible consequence of defining productivity with a small number of measures that can easily become targets.

Another threat involves divergence between the priorities of line-level officers and command staff. If the priorities of command staff do not mirror those of line-level officers and are not clearly communicated to them, then patrol officers may engage in activities unrelated to productivity metrics. For instance, if traffic enforcement is a high priority for command staff but line-level officers view narcotics activity or robberies as more pressing issues, then officers may engage in focused deterrence and other strategies instead of writing traffic tickets. In this situation, patrol officers may not be rated as productive, even though their activities may nonetheless positively impact the community far beyond their performance rating.

A third threat involves the correlation of outputs to community outcomes. A patrol officer’s activities should be correlated with positive community outcomes, such as reductions in violent and property crime and increases in overall quality of life. Productivity metrics that have little or no relationship with crime, disorder, or quality of life likely have little or no validity for agencies and communities, even if officers score exceptionally high on such metrics.

However, the biggest threat to validity may be tracking and measuring tasks that do not fall within the traditional realm of law enforcement activities. Police officers are asked to do more and display a greater array of skills than in prior decades. They must effectively engage with a wide variety of community members and groups, use data and science to advance department and community goals, and connect citizens with resources for any number of issues (e.g., homelessness, mental health, and domestic violence). Police researchers must find ways to validly and reliably measure these types of activities, especially if police departments prioritize them.

Improved Measures of Patrol Officer Productivity

Researchers have attempted to improve traditional measures of patrol officer productivity in several ways. One way involves using a larger number of diverse productivity indicators. Including multidimensional indicators reduces the chances of one measure becoming a target (i.e., it defies Goodhart’s Law), necessitates discussion between line-level and command staff members about what outputs should be included, and allows researchers and agencies to correlate a wide variety of outputs with community outcomes. For instance, instead of only measuring arrests, some researchers have measured police performance using self-initiated stops, warnings, citations, administrative complaints, sick hours, and on-duty injuries as well as z-score summaries to more easily interpret an officer’s performance.[16]

Van Meter’s zero-based approach is another innovative attempt to quantify and evaluate patrol officer performance.[17] His system views police officers as productive before evaluation and assigns each officer a zero, the best possible score. The system analyzes nonscheduled absenteeism, cost of preventable error, and productive use of time to evaluate police officers, and the zero indicates that a police officer has no uncorrected performance issues. However, some have criticized Van Meter’s system for preventing police officers from prioritizing their daily activities,[18] suggesting the potential for a disconnect between command staff and line-level officer priorities.

Borrowing from a baseball statistic called Value Over Replacement Player, researchers have constructed Value Over Replacement Cop (VORC), a metric that accounts for the diverse activities of patrol officers, weights different outputs, evaluates officers in terms of productive time and prosecution rates, and offers police agencies the flexibility to prioritize and weight patrol officer activities (see Exhibit 1).[19] VORC’s major strength is also its weakness — namely, that it allows agencies to prioritize and weight outputs, which leads to criticisms about the appropriate prioritization and weighting of outputs.

A close relative of VORC is Z-Score per Productive Time (Z-PRO), a more sophisticated measure that estimates a patrol officer’s performance in terms of productive time using a combination of z-scores for directed patrols, traffic warnings, traffic citations, DUIs, misdemeanor arrests, felony arrests, and warrant arrests.[20] Exhibit 2 displays Z-PRO’s wide variety of outputs as well as other important measures, such as the number and types of completed reports, minutes spent on follow-up investigations, calls for service minutes, and minutes spent assisting other officers — a major advantage over traditional, more simplistic measures.

One key weakness of these metrics, as well as other innovative metrics, is that their relationship to community outcomes is unknown. Although researchers have examined the correlation between potential components of a productivity metric and community outcomes (e.g., traffic citations and motor vehicle collisions), much less is known about the correlation between broader productivity indices and community outcomes (e.g., how Z-PRO correlates with crime and disorder). However, although I recognize the importance of such outcome measures, instead of narrowing the point of focus to outcomes only, I urge researchers and police agencies to consider the following guidelines when developing, implementing, and analyzing patrol officer productivity metrics.

IMPACTT Guidelines

I designed the IMPACTT guidelines to help police researchers and practitioners evaluate the validity of patrol officer productivity metrics. IMPACTT is an acronym for the following recommendations: The outputs of any productivity metric must be Identified and prioritized, be Measured both quantitatively and qualitatively, be evaluated in terms of Productive time, account for a diverse Array of duties, be Correlated with community outcomes, and be Tracked and Tested over a prolonged period.

First, agencies must Identify and prioritize the outputs of a metric and communicate these priorities to line-level officers. Patrol officers should have a clear understanding of which activities are valued most by their department and community. I also recommend that agencies either weight outputs or use a z-score index to more easily distinguish between low- and high-performing officers.

Second, the outputs of productivity metrics should be Measured both quantitatively and qualitatively. Before implementing a metric, agencies must have the technology to record and measure the targeted outputs, as well as the ability to create and maintain searchable databases so the outputs can be analyzed and evaluated. In terms of qualitatively evaluating outcomes, the rate of prosecution for citations and arrests may be one quality control measure. If an officer makes a large number of arrests that fail to result in charges because of shoddy investigations or follow-up, then the officer’s performance metric should reflect this deficiency.

Third, performance metrics must evaluate patrol officers in terms of Productive time. Too often, researchers and agencies analyze totals for arrests, patrols, or citations without accounting for how many calls for service an officer handled or how many minutes an officer spent assisting other officers, writing reports, or conducting follow-up investigations. The validity of any productivity metric is vastly improved when it accounts for an officer’s available minutes for self-initiated activities, e.g., traffic or warrant enforcement.

Fourth, productivity metrics must include an Array of activities, especially in communities where police agencies are generalist departments. Generalist patrol officers not only respond to calls for service and make arrests but also may conduct traffic enforcement, warrant searches, follow-up investigations, community presentations, directed patrols in high-crime areas, and many other activities. Productivity metrics must be multidimensional to quantitatively capture the diverse array of a patrol officer’s activities.

Fifth, productivity metrics should be Correlated with community outcomes. Agencies should be able to demonstrate that patrol officer outputs (e.g., citations, arrests, performance evaluations) are related to property or violent crime rates, quality of life outcomes, public perceptions of and trust in the police, or public perceptions of crime and disorder. Moreover, agencies should be able to demonstrate that a productivity index — and not just its individual components — is also correlated with community outcomes.

Finally, patrol officer productivity measures should be Tracked and Tested over prolonged periods. This practice allows researchers and agencies to correlate outputs with community outcomes, reprioritize outputs if necessary, and guard against a limited number of measures becoming targets.

Discussion

For many decades, research on methodologies for measuring patrol officer productivity has failed to advance due to a lack of data, insufficient technology to track patrol activities, and a narrow focus on a few types of outputs. Although more sophisticated metrics have been developed, researchers and law enforcement agencies must remain cautious of threats to the validity of these metrics, including the potential for outputs to become targets, a disconnect between the priorities of command staff and line-level officers, and low correlation between metrics and community outcomes. To improve the validity of productivity metrics and guard against recurrent threats, I put forth a series of suggestions called the IMPACTT guidelines. These guidelines recommend that the outputs of productivity metrics should be identified and prioritized, be measured both quantitatively and qualitatively, be evaluated in terms of productive time, account for a diverse array of duties, be correlated with community outcomes, and be tracked and tested over a prolonged period. I believe that researchers and law enforcement agencies can use these guidelines to develop, refine, and assess new methods for evaluating patrol officer productivity.

Disclaimer: Opinions or points of view expressed in this document are those of the authors and do not reflect the official position of the U.S. Department of Justice. Findings and conclusions of the research reported here are those of the authors and do not reflect the official position or policies of the U.S. Department of Justice.