As criminologists, we are typically well trained in evaluation methodology but less so in the issues and nuances associated with technology deployment. When we evaluate criminal justice technology, we must understand both the capacity of the technology and how people use it. Focusing on one at the expense of the other can render the entire evaluation effort futile.
We learned that lesson the hard way during an NIJ-funded evaluation of the use of radio frequency identification device (RFID) technology to reduce sexual assaults and infractions by those incarcerated in a women's prison.[1] Although our evaluation revealed much about the promise of RFID technology for monitoring, tracking, and investigating individuals, it was seen largely as a failure, because the implementation of the technology was fraught with problems. (Read more about the evaluation.)
Social science publications often favor studies that yield statistically significant findings in the expected direction. But we can arguably learn much more from failure than from success, and these lessons can help improve both technology deployment and evaluation methodologies.
This article offers some lessons learned from our evaluation about the challenges and opportunities associated with deploying criminal justice technologies in the manner most likely to yield their intended impact. Doing so requires:
- Developing a clear understanding of how you envision that the technology will work — the logic behind its implementation and use.
- Educating users about implementation and training requirements to ensure cultural buy-in and full deployment — the fidelity piece.
- Engaging in early and ongoing assessment to identify and correct implementation problems and challenges and learn how they relate to intended impact — the feedback loop.
Although the contextual example for this article is implementing RFID technology in a women's prison, the implications for policy, practice, and evaluation can apply to a wide array of criminal justice technologies in a variety of field settings.
RFID in Correctional Settings
RFIDs have a tag or "chip" that uses wireless data communication to transmit information to electronic sensors, allowing users to locate and track tagged objects with a unique identifier.[2] RFID technology was first used during World War II to identify whether approaching planes were friend or foe. Today, commercial groups commonly use it to track merchandise from warehouse to distribution to point of sale.[3] RFID technology is also found in access control and payment systems, such as building access keycards, transit payment cards, and highway electronic toll collection systems.
Most RFIDs are "passive," meaning that the chips can be read only when in close proximity to a sensor. For example, community corrections agencies use passive RFIDs to electronically monitor those on home detention.
"Active" RFIDs have a battery-operated device integrated with the chip, enabling the tagged item to both receive signals and transmit data back to the sensor in near real time. Active RFIDs have three components:
- A chip embedded in a bracelet, which is often worn on the ankle.
- A series of data extension units, which read and transmit data from the chips.
- Computer software that can document the location and identity of those wearing the bracelets.[4]
Institutional correctional settings have been using active RFIDs for at least a decade[5] for things as basic as perimeter control, as well as more sophisticated applications, such as issuing alerts when two rival gang members are in close proximity. RFIDs can aid housing unit "counts," identify whether incarcerated persons are in the proper locations (e.g., classroom, yard, housing unit) according to their schedules, and even monitor their eating habits.[6] The technology can also serve as a valuable investigative tool, generating historical data on an incarcerated person's location and the time of his or her movements.
The Northeast Pre-Release Center (NEPRC), a women's prison in Cleveland, Ohio, that was the setting of our NIJ-funded evaluation from 2005 through 2007, intended to use RFIDs in all of these ways. Unfortunately, the implementation of the technology was filled with problems. Service was interrupted for several months, staff received minimal training, bracelets were used inconsistently (at one point, 25 percent did not have bracelets), and NEPRC restricted software use to perimeter control. The prison never employed the technology's most powerful feature: tailored, exclusionary zones for incarcerated women.[7]
Understanding the Logic
One early flaw in NEPRC's deployment of RFID technology was the absence of a well-articulated logic model. Quite simply, a logic model asks the question, "How is the technology supposed to work?" Answering that question thoroughly will increase the odds that users deploy the technology as intended. In the case of NEPRC, this would have entailed conversations with all staff members about the nature, frequency, and context of their most challenging threats to safety and security. These conversations should have taken place well before NEPRC deployed the RFIDs and should have included detailed discussions of the technology's capabilities and training and operational requirements.
Table 1 presents one example of a simplistic RFID logic model, which can help identify the main uses of the RFIDs. It also helps underscore the critical role of human interaction with the technology. For example, even if a prison sets appropriate exclusionary zones and equips 100 percent of its incarcerated population with RFID bracelets, the technology is unlikely to have an impact if officers do not respond to system alerts that are generated when incarcerated persons are out of place or in close proximity to prohibited people.
| Inputs | Outputs | Outcomes | Impacts |
|---|---|---|---|
| Software: Program perimeter areas and exclusionary zones | Alerts issued when individuals are out of place | Greater compliance with rules | Reduced escapes, enhanced safety |
| Hardware: Sensor installment and ankle bracelets | Ankle bracelets | Increased perception that infractions and violations of facility policy will be detected | Reduced misconduct |
| Training: Staff responses to alerts | Swift response to alerts | Increased identification of misconduct | Reduced misconduct |
| Training: Use of RFID data for investigations | New source of evidence to investigate infractions and misconduct | Increased apprehension, faster case resolution | Reduced misconduct, reduced incidence of false allegations |
Monitoring Fidelity
Engaging users in this thinking early on not only engenders buy-in but also helps guide the development of performance measures. Performance measures can help document whether the technology was implemented as intended. Indeed, implementation fidelity is the cornerstone of effective technology deployment. Institutions must train their officers on how to properly use the technology and must routinely monitor and hold them accountable.
From a researcher's perspective, accurate documentation of the technology's use is a necessity. In the case of the NEPRC RFID evaluation, the prison — unbeknownst to us, the researchers — had never fully implemented the technology's exclusionary zone features, which, theoretically, would yield the greatest deterrent effect. Moreover, halfway into the deployment period, we learned that the system was inoperable and that, as a result, officers had stopped equipping new arrivals with bracelets.
That these incidents occurred and took several months for us to discover represents a failure on the part of all involved parties. The prison should have been monitoring and documenting the technology's usage weekly, if not daily. And we should have requested that documentation throughout the course of our evaluation.
Ongoing Assessment
Ongoing assessment and impact evaluation are also critical. The former allows for midcourse corrections and relies on the types of fidelity metrics described above. The latter can help guide decisions about continued financial investment in the technology and possible expansion to other facilities.
We have established the value of partnering with a local researcher prior to deploying the technology. When doing so, corrections administrators would benefit from securing an agreement that the researcher is willing to embrace an "action research" approach rather than the more traditional evaluation methodology. Traditionally, researchers erect a firm firewall and do not report back implementation or impact findings until they have concluded their evaluation. Action research, by contrast, involves ongoing analytic support and engagement.
Some purists frown upon action research, saying the approach creates a Hawthorne effect of sorts, whereby the researcher's activities affect the nature of the intervention. However, researchers with a practical bent are increasingly embracing action research. Corrections leaders should demand this time-honored approach when deciding to team with evaluation partners.
Corrections officials would also benefit from including a cost-benefit component in their evaluation partnerships. This component can provide crucial data to help inform decisions about ongoing investment, expansion, and sustainability.
Summing It Up
Shortly after the release of our final RFID evaluation report, I found myself in the conference room of a corrections agency in a nearby jurisdiction. The director objected to our finding that the technology had no impact when implemented at its most basic capacity, as was the case at NEPRC, and he launched into a tirade about our so-called "substandard" evaluation. I learned later that his agency had just committed considerable resources to implementing its own RFID system, and he felt that our evaluation threatened the wisdom of that investment.
He is not alone. Far too often, we draw erroneous conclusions about the effectiveness of criminal justice technologies based on evaluations that yield no impact. But in this case — and arguably in many like it — the failure is not in the technology itself but in the deployment. As Thomas Edison once famously observed, "Just because something doesn't do what you planned it to do doesn't mean it's useless."
When deploying technology, corrections agencies should gather staff input, train and engage all users, and routinely monitor and document usage. Evaluation partners can help collect data and provide that all-important feedback loop, which can lead to midcourse corrections aimed at enhancing implementation fidelity. Following this approach will prevent false assumptions and increase the odds that the technology will yield its intended impact.
For More Information
Read the final RFID evaluation report, "Evaluating the Use of Radio Frequency Identification Device (RFID) Technology to Prevent and Investigate Sexual Assaults in a Correctional Setting."
About This Article
This artice appeared in NIJ Journal Issue 278, May 2017.
This article discusses the following grant: "Evaluating the Use of Radio Frequency Identification Devices to Prevent and Investigate Sexual Assaults in Correctional Settings," grant number 2007-RP-BX-0001.