Few events in American life evoke stronger reactions across society than mass shootings. They are part of the broader phenomenon of mass violence that includes, for example, terrorist attacks and war-related events. But mass shootings are distinguishable from those categories of mass violence in that their underlying motive sometimes appears to be unknown. Typically, mass shootings occur in a public place, with a single shooter, and most victims are killed or wounded indiscriminately.[1]
Because mass shootings have a severe impact on victims and society, they are a national criminal justice priority. As the frequency of mass shootings has increased in recent years, law enforcement and researchers have intensified their efforts to understand and prevent this form of firearms violence.[2] But their efforts are being held back by two systemic deficiencies: (1) the absence of a uniform definition of mass shootings and related concepts and (2) the absence of consistent databases that gather, sort, and share essential facts on attempted and completed mass shooting incidents.
In an effort to improve understanding of mass shootings, NIJ science staff carried out a systematic literature review to identify the current state of knowledge suitable for use in preventing these incidents. They uncovered apparent inconsistencies in researchers’ definitions of mass shooting incidents. Moreover, they found that the analyses supporting the definitions often rely on open-source data that are unreliable, inconsistent, or both.[3] These inconsistencies may lead to mixed — or even contradictory — findings, suggesting a need to align data and definitions in a more unified, coherent approach.
NIJ also convened leading researchers and law enforcement practitioners to gain additional insight into the challenges surrounding mass shooting studies and prevention strategies. The experts offered recommendations on how the field should move forward to advance both the research on and the prevention of mass shootings. All of these insights will help guide NIJ’s leadership of mass shooting research and data management going forward, as key elements of its larger role in directing scientific investment to address violent crime and inform prevention efforts.
Inconsistencies in Definitions
To better understand the state of knowledge and identify gaps in research on mass shootings, NIJ science staff systematically reviewed the literature from 1997 through 2016.[4] Their analysis encompassed 44 research studies on mass shootings. Results revealed both consistencies and inconsistencies in the literature. Collectively, studies have yielded a number of high-utility insights on shooter characteristics, choice of targets, weapons, and other variables. Generally, however, the scholarship has been hampered by a lack of agreement on definitions of critical terms, such as “mass shootings” and “mass murders,” and by the absence of consistent sources of data on mass shootings.
The literature does not define “mass shooting” consistently, or even in similar contexts. The federal criminal code lacks a distinct mass shooting offense; this may help explain why researchers use different terminology, or types of criminal offense, in their analyses of the same phenomenon.[5]
Among the 44 studies analyzed, the most common definition of a mass shooting is an incident in which four or more victims are killed with a firearm in a public place (48%). Several studies defined the offense as an event during which as few as two (5%) or three (9%) victims are killed, whereas more than one-third of the studies more broadly defined the term as an incident in which multiple victims are killed (38%). Others either defined a mass shooting incident as having a minimum of five victims or did not specify a victim threshold.
The definitions in the analyzed studies include incidents that take place in publicly accessible spaces such as schools, workplaces, places of worship, and businesses. The incidents are also defined as a single, continuous event within a short time frame, but the specific time frame can vary. The definitions often exclude ideologically motivated terrorist acts as well as gang, drug, and other shooting incidents that resulted primarily from the commission of other crimes, such as aggravated robbery, familicides, and domestic violence. Some of these studies, however, do not specify whether certain types of offenses were excluded from the definition.
This lack of consistency in defining mass shooting events is reflected in contradictory findings across a number of studies. The differences noted appear to contribute to varying conclusions about individuals’ average age, motives, personality, suicidality, and target selection (i.e., victim, or victims, and place). Other notable differences in findings relate to the choice of firearms as well as the possible influence of news media coverage on mass shooting events and those who commit them. Importantly, wide variability in mass shooting definitions — in terms of the requisite minimum numbers of individuals shot and killed — casts serious doubt over the field’s ability to accurately capture all of the cases and analyze trends.
Limited Access to Consistent Databases
Compounding the problem is the lack of uniform, reliable data sources. The literature reviewed used 10 types of data sources, and the majority of the studies used more than one type of data source. Of the 122 distinct data sources used in the 44 studies, 65% came from secondary, open-source data. Open-source data refers to publicly available and accessible information such as databases, news and media accounts, or other widely available sources. Thirty-three percent came from official records that are publicly accessible for the most part, and 2% came from interviews (see exhibit 1).
It is evident that there is no single, primary source of data used across the research on mass shootings. Some of the official records, such as the FBI’s Uniform Crime Reports, Supplementary Homicide Reports, and National Incident-Based Reporting System, are often based on case files developed for the purposes of investigation and prosecution. Many times, however, they lack information on a wide range of variables that could advance prevention research. Such limitations often lead researchers to supplement the data with information from open sources or to rely solely on secondary data. Moreover, even if those standard, official reports were factually rich and complete, it is highly unlikely that they would be able to address many of the questions that are relevant to informing practices around preventing mass violence. For example, they generally do not include data on what the shooter did to prepare for the shooting, whether the shooter expressed some form of grievance, or whether the shooter had a history of mental health issues or had experienced a recent loss.
The factual limitations of official reports complicate the task of assessing the reliability of sources, raising questions such as how each data source defines the phenomenon, what specific information the source provides, and — in the case of databases — what the time frame is for including events. As with inconsistencies in the definition of mass shootings in terms of the number of victims killed, the use of different data sources obfuscates trends and the impact of policies. This is not to say that no study has produced valuable results and recommendations. But without a thorough analysis of the research design by a trained eye, the end users of research, such as policymakers and practitioners, may arrive at conclusions that are erroneous and that may produce more harm than good.
There are a number of ongoing efforts by researchers and the federal government to build or enhance mass shooting databases. However, the research community must identify the challenges in this line of research and determine a set of characteristics that would make any given mass shooting database more reliable and useful in informing prevention.
Heeding the Experts
In the latter half of 2018, NIJ held directed discussions with subject matter expert groups of law enforcement officials and scholars as part of its initiative to assess existing mass shooting research and gauge its shortcomings. Insights gained at those sessions can inform and refine research going forward.
The primary objectives were to:
- Assess the need for uniform definitions in mass shooting data collection and analysis.
- Discuss the benefits of establishing data collection techniques to consistently catalog all of the pertinent mass shooting information.
Law enforcement discussants (practitioners) were current and former members of federal and local law enforcement agencies. Researcher discussants (researchers) were a multidisciplinary collection of scientists from several U.S. universities.
Points of Broad Agreement
The practitioners and researchers agreed on certain discrete research and practice needs. For example, they reached general agreement that a universal definition of mass shootings would not solve all ambiguity problems but would be an important first step. A common definition of mass shooting should be broad but not tied to any fixed minimum number of victims (for instance, a rule that a mass shooting means the killing, by firearm, of four or more people). Some samples of relevant comments by discussants include:
- Researcher: “The number of people killed can be happenstance. … If you focus too much on [a] happenstance outcome, things might get lost. It seems arbitrary to say three or four or five victims minimum. That seems to be missing the big picture.”
- Practitioner: “That number [four] seems arbitrary. It should have less to do with efficiency, [that is, the] number of people in the room, etc., than the intent of the offender.”
- Practitioner: “You have to include nonfatal injuries. They all intend to kill, but if they are a poor shooter, you still have the same dynamics and personality — they just didn’t know how to operate the weapon.”
They also agreed that a mass shooting event is an incident where there is an evident premeditated intent to shoot to kill, regardless of the number of actual fatalities or injuries.
- Researcher: “But with the definition, I think we can discern that what we’re trying to get at is this event with this person who had the intent to kill large numbers of people.”
- Practitioner: “I think numbers are arbitrary and don’t matter. If the intent was to kill a bunch of people, it doesn’t matter. It would be counterproductive for prevention to exclude them.”
- Practitioner: “So, we get to the intent of the individual when they came to the incident. If they did [intend harm to a lot of people], it’s in; if not, it’s out. The reality is that if you include cases with only two or more victims, the offender in those cases might have been trying to kill more but didn’t.”
Points of Difference
On other issues, there was notable divergence between the practitioners and researchers. For one, practitioners tended to favor reliance on data and data sources that are objective and verifiable, whereas researchers tended to be more receptive to open sources as well as more subjective data related to, for instance, health factors. Key examples of where practitioners and researchers diverged include desired data sources for mass shootings and the time range for including an incident.
It is important to note that different data sources are designed for different purposes. Official data sources are often developed for investigations and prosecutions. Such sources have high value for answering some investigative questions, but may not be at all responsive to others. Official sources tend to focus on proximal factors related to the crime (e.g., time, place, manner, demographics, and other information that describes the criminal act and the individual committing it). On the other hand, media accounts (an example of an open source) are more likely to trace back further in time and look more broadly at other possible factors that influenced the individual committing the crime. That is, they may include information that is absent from official sources but is valuable for prevention purposes. At the same time, compared with official records, media sources may be more influenced by subjective judgments and errors.
Researchers tended to support a research approach that includes open-source data, such as media accounts. Several researchers said that because of a lack of access to official records and sensitive data, they often relied on open sources to fill the gaps and triangulate data. It should be noted that, if given the choice, these researchers said they would prefer to use official data sources. But they also see the value of triangulating information from multiple types of data sources for research purposes.
Practitioners tended to be strongly opposed to an open-source approach and to reliance on media accounts. Several practitioners said that in their view, media accounts are largely unreliable as primary data sources on mass shootings.
The sharp divergence in views between researchers and practitioners on data source preferences may reflect the distinct nature of their respective professions. Practitioners in the law enforcement field are accustomed to using official data, and their interest in determining accountability and culpability for criminal acts is often best served by data attributable to official sources. Researchers tend to seek answers to a broader range of questions, calling for broader data sources.
Some researchers and a local-level practitioner said they valued the collection of retrospective data (e.g., from the preceding 50 years) on qualifying incidents that were not sourced from media reports. They also emphasized the importance of collecting the same kind of data prospectively. Some practitioners, on the other hand, recommended a focus on data from 2000 forward, given the limited access to information prior to the implementation of internet technology.
Recommendations for Future Research
A primary purpose of the expert discussion groups convened by NIJ was to produce guidance on developing further mass shooting studies to improve prevention. Researcher and law enforcement participants voiced support for a series of recommendations:
- Partner with law enforcement agencies (both local and federal) and associations to better access official data on mass shootings through sources that include interviews with incarcerated persons, police investigations, and mining of information on multiple-victim shooting incidents that were not covered in any depth by the media.
- Examine data on averted attacks.
- Compare mass shootings with other forms of mass violence.
- Help identify and debunk misconceptions with scientific evidence (e.g., weapon choice, mental health, motivation, planning and preparation).
- Estimate costs of mass shootings and victim impacts over time.
- Develop guidelines and resources for identifying and managing people of concern.
- Create an analytical model to enable practitioners to engage in predictive analysis of mass shootings. The model would be based on the time (including date), place, and modus operandi of studied mass shooting events.
- Create models for information exchange among local and federal stakeholders.
Moving Forward
NIJ’s analysis of mass shooting literature and its structured engagement of experts point to the advisability of certain major action items for mass shooting research and law enforcement practice. First, there appears to be broad sentiment in favor of moving away from a number-based system of rigidly defining mass shootings and related phenomena, and toward defining mass shootings more flexibly. An incident should not be entirely omitted from a mass shooting data set where, for example, a shooter with evident intent to kill multiple persons opens fire in a park where 10 individuals are present, with several resulting gunfire injuries but three or fewer fatalities.
Law enforcement should have a more active role in the study of mass shootings and in translating research to practice — for example, developing detection methods and tips and educating and training bystanders, school counselors, and others. Beyond an expanded research role, law enforcement officers will remain the last, crucial barrier between prospective shooters and their intended victims. Law enforcement must enhance its capability to detect and intercept mass shooters, and educate members of the public to detect and report any warning signs of prospective shooters in their communities, if the threat to society is to be reduced.
Criteria should be developed to facilitate adopting uniform definitions and data characteristics across all databases. NIJ recognizes that uniform, consistent cataloging of past mass shootings designed to support future data entry is an essential first step in advancing research and prevention efforts. Additionally, NIJ hopes to glean insights from analogous fields that study rare incidents (e.g., terrorism) to replicate and improve on established data collection methods and sustainability.
Preparedness for mass shootings — deeply traumatizing social phenomena as elusive as they are disruptive — will require an increasingly focused and coordinated effort by the research and practice communities as we move forward.
For More Information
Learn more about NIJ research awards on mass shootings.
About This Article
This article appears in NIJ Journal issue 282.