With NIJ’s support, an expert panel on security surveillance technology has devised an “investment road map” to identify priority needs to guide development of formidable new video and other monitoring tools to fight crime and improve public safety. The panel’s final report identifies a set of priority innovation needs as well as a list of common objects and behaviors the new video technology should be able to recognize.
The explosive growth of security cameras in all corners of society promises to greatly enhance public safety agencies’ ability to stop or prevent crime, respond to accidents, and eliminate safety hazards. To be effective, however, video surveillance generally requires monitoring by humans exercising judgment, and public agencies are strained to detect and process all of the bad acts and mishaps those cameras capture. Field observations have revealed that one individual can simultaneously monitor ten video screens at most, and then only superficially. Research also has shown that a person charged with monitoring multiple camera feeds can only concentrate closely on one at a time. Moreover, people watching video monitors lose effectiveness later in their work shifts.[1]
Emerging machine-learning tools show vast potential for replacing inefficient human monitoring of security video with programs that reliably and immediately alert authorities when they recognize suspect or hazardous objects (a raised gun, for example) or movements (a fall from a bridge, for example). The research and development niche focused on these new surveillance tools is known as Video Analytics (VA).
Ultimate decision-making on how to respond to crime and hazard alerts will remain with human actors. But for initial detection, the formidable analytical power and speed of VA technology are of a different order than the human eye-brain connection.
On a parallel track, equally promising technology is refining ways to intelligently merge multiple data streams – for example, automatically aiming nearby cameras at an officer when biometric sensors signal that she is under stress or assault, or triggering a video zoom on a license plate whenever a nearby camera detects activity consistent with a veering or suddenly speeding vehicle. This area of R&D is called Sensor Fusion (SF). Together these synergistic new fields are known as “VA/SF.”
With the promise of potent VA/SF technology for public safety comes uncertainty about how best to invest in VA/SF development and how to apply these technologies for optimal law enforcement and public safety impact.
To help forge a responsible path forward for VA/SF research, a panel of policing and security experts convened on behalf of NIJ has charted a VA/SF “investment roadmap” for technology development that identifies needs for innovation. Guided by the RAND Corporation, assisted by the Police Executive Research Forum (PERF), the panel of 15 experts concluded that Video Analytics and Sensor Fusion are “extremely promising technologies for improving public safety,” according to the report Using Video Analytics and Sensor Fusion in Law Enforcement of the panel’s work published by RAND.[2]
The VA/SF expert panel initiative is one element of NIJ’s continuing commitment to delivering practical technology solutions to pressing law enforcement and public safety needs. The report and recommendations are the end product of the VA/SF panel’s two-day July 2017 workshop focused on four inquiries:
- What are the core public safety applications for VA/SF?
- What are the specific VA/SF tasks needed to carry out those applications?
- What security, privacy, and civil rights protections are needed?
- What technology, policy, and educational needs for education are most important to address?
The panel identified four key “business cases” – core law enforcement and public safety functions – for VA/SF:
- Real-time monitoring: Detecting crimes and their precursors, hazards and suspicious activities.
- Forensics: Providing data to investigations, with video and data management. Data undergoes automatic indexing and speech-to-text conversion.
- Auto-reporting: Automatic review of video and sensor data in temporary storage, checking for relevance to known cases. Automatic indexing and speech to text conversion.
- Performance monitoring: Assessing personal and agency performance; support of best practice training.
Officer well-being could also benefit from VA/SF technology, the panelists found. The report noted that a fusion of video and biometric sensors “could substantially improve the health and safety of officers.”
From the four business cases, the experts generated a list of 22 priority needs for VA/SF development, grouped in the following four categories as shown in the table below along with examples of each:
| Priority Category | Example |
|---|---|
| Core Recognition Research | Facilitate the creation of a continuously refreshed service for cataloging and sharing data sources that can be used for training algorithm models for law enforcement; develop best practices to ensure that the data algorithms that are trained on sufficiently cover the continuum of possibilities. |
| Computing Infrastructure | Develop systems capable of real-time indexing as video is collected or archived. |
| Security, Privacy, and Civil Rights | Develop data retention policies; develop standards for an audit trail, explaining why particular information was presented to an officer. |
| Innovation Supporting Specific Core Public Safety Functions | To support real-time monitoring, conduct R&D on the critical impacts to public safety (e.g., desirable municipal traffic outputs to monitor on video) and the appropriate level of human-in-the-loop presence for monitoring and improving changes. |
Video analytics depends on multilayer neural networks – machine-learning technologies employing networks of computational nodes, or neurons – that together can recognize a feature of interest (such as a gun or a human figure’s thrusting action, etc.) after being trained on a large set of exemplar images where the feature is present or absent. As the machine learns more, its ability to recognize and react becomes more precise, nuanced and prolific. The expert panel produced a list of crimes in progress that should be recognizable through applied VA/SF, including but not limited to:
- Homicides, shootings, and aggravated assaults.
- Sexual assault.
- Kidnapping.
- Vehicular homicide and hit-and-runs.
- Robbery and burglary.
- Aggravated assault.
- Arson.
- Drug trafficking and sales.
- Driving under the influence.
- Trespassing.
- Simple assault.
- Theft.
The new tools should also detect common criminal precursors related to assault, the panel urged, including, for example:
- Recognition of gun shots.
- Visually displayed weapons.
- Objects and behaviors indicative of a weapon’s presence, for example a bulging pocket.
The panel report identified additional precursors of different categories of crime.
The panel emphasized the need for restrictions on VA/SF uses, to protect privacy, security and civil rights. The first recommended restriction, the final report said, is that “these tools should collectively be used as passive sensors that trigger alerts or are consulted in response to other events, rather than persistent surveillance systems that are constantly monitored and spur immediate action.” Further, “[A]ny use of recorded video or sensor data will require a law enforcement predicate, such as a suspected crime or threat to public safety or homeland security.”
The 2017 workshop built upon strides made in video-analytics priority-setting during a June 2016 workshop of stakeholders called by National Institute of Standards and Technology (NIST). Two priorities to emerge from the initial NIST meeting on VA were:
- Development of analytic tools to provide solutions to content-centric problems related to increasing demands for video in public safety.
- Access to the most advanced technology and greater engagement with R&D communities.
Both the risks and the rewards of VA/SF are amplified by the proliferation of monitoring technologies now in common use. Closed circuit video, long the norm, is now complemented by large numbers of law enforcement body-worn and car-mounted cameras, and other types of sensors. In addition, the “Internet of Things,” household appliances and other common devices, like home security systems, now linked to the Internet, is greatly expanding the number of potential surveillance data sources. The panel observed: “Law enforcement officials will increasingly have streaming video feeds from numerous sources to help protect the safety of officers and bystanders. They may also have access to increasing numbers of privately installed cameras that stream over the internet.”
The expert panel stressed the importance of community buy-in before VA/SF tools are deployed in the field. The public should be consulted before, during, and after implementation, the panel said. Further, measures should be in place to recognize and protect officers’ rights when VA/SF tools are integrated with body-worn cameras, biometric sensors, and other sensors of police activities or conditions.
The VA/SF expert panel was sponsored by NIJ and the Priority Justice Needs Initiative. The initiative is a project of RAND, PERF, RTI International, and the University of Denver.
Although the expert panel found VA/SF to be very promising for fighting crime and improving public safety, the panel sounded a cautionary note on potential infringement of privacy and civil rights: “[T]he risks of VA/SF technologies are significant. The panel recognized that these technologies have great potential to be abused.”
With that in mind, the panel identified particular needs for innovation related to protecting security, privacy, and civil rights.
About This Article
The research described in this article was funded by NIJ award , awarded to the RAND Corporation, working in partnership with the Police Executive Research Forum, RTI International, and the University of Denver. This article is based on the grantee report “Using Video Analytics and Sensor Fusion in Law Enforcement: Building a Research Agenda That Includes Business Cases, Privacy and Civil Rights Protections, and Needs for Innovation.”
The report contributors are John S. Hollywood, Michael J.D. Vermeer, Dulani Woods, Sean E. Goodison, and Brian A. Jackson.