Advancing Understanding, and Informing Prevention of Public Mass Shootings: Findings from NIJ Funded Studies, Part 1

MARY JO GIOVACCHINI: Good afternoon, everybody, and welcome to today's webinar, Advancing Understanding, and Informing Prevention of Public Mass Shootings, Findings from NIJ Funded Studies Part 1, hosted by the National Institute of Justice.  So at this time, I am going to webinar over to Basia Lopez.

BASIA E. LOPEZ: Welcome, everyone. Thank you for joining this webinar. My name is Basia Lopez and thank you, Mary Jo, for this information that you just provided. I am a Social Science Research Analyst in the Office of Research, Evaluation, and Technology at the National Institute of Justice. NIJ is a research and development, and evaluation arm of the US Department of Justice and its Office of Justice Programs. At NIJ, I oversee mainly federally funded research, evaluation, and data collection projects related to firearms violence, including mass shooting. I will share this webinar, through which we will hear from a team of renowned experts followed by a discussion that will center around the implications to finding [INDISTINCT] for the criminal justice system and on prevention. I am accompanied by NIJ's Graduate Research Assistant, Danielle Crimmins. Danielle, please introduce yourself.

DANIELLE CRIMMINS: Hello, my name is Danielle Crimmins. I'm currently a Research Assistant here at NIJ and also a graduate student at Purdue University. I'll be leading the Q&A session following a discussion on implications the findings have on the criminal justice system and prevention. Thanks.

BASIA E. LOPEZ: Thank you. Before we give the floor to our guest, I would like to mention that since the 1980s, our institute has sponsored research in an effort to assist our nation in reducing firearm violence by supporting rigorous studies and evaluation of programs to build knowledge and evidence-based practices aimed at reduction of firearms violence, building on various previous projects carried out by NIJ, including several projects funded under our Comprehensive School Safety Initiative. Since 2018, NIJ prioritized research on mass public shootings under the firearm violence portfolio and solicited for research projects on this topic through its investigator-initiated research and evaluation on firearms violence competitive solicitation. So, what you will hear today are some of the preliminary results, all the results already published by one of these NIJ-funded projects. This research team received an award in 2018 for the project titled “The Nature, Trends, Correlates, and Prevention of Mass Public Shootings in America 1976-2018.” Our panel consists of research experts in studying gun violence and mass killings, specifically mass shootings. Our first presenter, James Alan Fox, is the Lipman Family Professor of Criminology, Law, and Public Policy at Northeastern University in Boston, Massachusetts. Our second presenter is Grant Duwe. Dr. Grant Duwe is the Director of Research and Evaluation for the Minnesota Department of Corrections. And our final presenter is Dr. Michael Rocque; he is an Associate Professor of Sociology at Bates College Lewiston, Maine. So, with no further ado, we will start with Dr. Fox and his presentation on “Trends and Contagion in Mass Public Shootings.”

JAMES ALAN FOX: Thank you and thank you, NIJ, not just for the funding, but for this opportunity to speak today and to present some of our findings. We have one more year on the project and we have several other items that are in the works. Also, thank you for giving me the first opportunity in nine months to wear a coat and tie, given the fact that I've been home so much because of the pandemic. Anyway, let's get started. So it's nice to see a very large number of people here today. I know it's not because of us as presenters, but more the topic, but that's interesting because I started studying mass shootings and mass killings 40 years ago and Grant Duwe about 30 years ago. And back then, no one was interested in mass killing and mass shooting. Among criminologists, sure--certainly serial murder but not mass killing.

But then came 2012, all things changed back then. You know, bad things happen in threes, they say, and we certainly had three horrible things that year, we had the Oikos University shooting at a movie theater in Aurora, Colorado, and of course the Sandy Hook Elementary School Shooting in Connecticut. Things were so bad that year in terms of mass shootings that the Associated Press named mass shootings the top story of the year, beat out the other Sandy--Hurricane Sandy, and also beat out the presidential election of the year. Well, of interest, also, was high among academics starting in 2012. Prior to 2012, relatively little was published in the area of mass shootings. But since then, there's been exponential growth in publications in the area of mass shootings, by scholars. Now, unfortunately, there was no official database. Sure, there was the Supplemental Homicide Reports, and you could look at shootings of four or more people, but the data were terribly flawed. A lot of cases were entered that shouldn't be. Cases, for example, where one person was killed and three people were injured, and they were all listed as if they were homicide victims. And many cases that should've been in there that weren’t.

So, starting with 2012, 2013, a variety of academic organizations and news organizations started to create and launch these database projects. There were about a dozen, and I've listed some of them here. And they vary, as you can see, in terms of timeframe, in terms of the number of victims required for the threshold, and whether those victims had to be killed or just injured. So here are just three in particular. And as you can see, these very in terms of--there are four more victims as the threshold, which is the standard threshold that had been used for many, many years. And as you can see, they're in terms of whether the victims had to be shot or killed. Now you have, for example, the Gun Violence Archive, I'm gonna talk about that later. And we had an average of just over one person killed--one victim killed per case and others, of course, were shot and injured. And then we have, for example, the USA TODAY--the Northeastern--I'm sorry, Associated Press, USA TODAY, Northeastern University database, it goes back in 2006, and they include all of mass killings of four or more people. On average, 5.7 victims killed. And then we have public mass killings in FDR there, stands for Fox, Duwe, Rocque, that's the three of us. We are focusing just on cases that are in public places. We're eliminating family massacres, we're eliminating gang-related, drug-related, robbery-related cases. These are the more indiscriminate cases in public places. They're what frightens America. It's not the ones that are more common; it's the ones that are more deadly that frighten Americans. It's also, when people think about mass shootings, they associate it with these large-scale massacres like we had, for example, in El Paso and Dayton last year.

So mass public shooting, as we define as, again, four or more killed in public place. They're not the majority of cases. In fact, about half of mass killings are family annihilations. And then we have, of course, the felony-related cases that I mentioned. Most people don't worry about those, it's not their family. They're not in a situation to be involved in a gang-related shooting. What people worry about are the public cases because it can happen at any time, at any place, to anyone. So we focus on those. The other reason we focus on those, of course, was that the NIJ solicitation specifically indicated interest in mass public shootings.

So looking at the trends over the past 40 years, in the right, you have the rate per million. There's some increase starting in the mid-nine--mid-2000s until the middle of this decade, but clearly, there's been a spike in 2018 and 2019, which, of course, got a lot of attention in the press. In terms of the severity, one of the significant changes in the recent years is the average size of these killings in terms of victim fatalities. Prior to 2015, the average number of victims killed per case was six, and since then, twelve. Of course, part of that is the fact that four of the eight cases, in which twenty or more people were killed that are listed here, occurred since 2015.

So, these stories, or these cases, led many Americans to believe it’s an epidemic, as you can see in these quotes here. I'm not so sure it's an epidemic--clearly would when it happens a better definition of epidemic, but I kind of question that. When you think about the fact that we're talking about a crime in terms of mass public shooting, that represents, in terms of victim count, less--significantly less than one percent of all people killed each year in the United States. Of course, they do stoke public fears. And in terms of fear, these surveys clearly indicate that. ABC News showing that six out of ten Americans fear that there'd be a mass shooting in their community. And a USA TODAY poll, also in 2019, show that about one in five Americans avoid certain public places for fear that they'll be the victim of a mass killing. And, as you can see by the Chapman University Fear Index, the level of fear associated with mass shooting certainly has grown over the past few years.

So why is this a disconnect between the risk and fear? Fear is high, 60% fear mass shootings. Yet, we're talking about a rare event that occurs--in terms of mass public shootings, maybe average about six times a year, deadly though they may be. Well, to some extent, there's misunderstanding and confusion about the datasets and how they define mass killings. And then part of it is also the nature of the--of the media coverage. For example, the gun violence archive, starting in 2013, basically said, "well, maybe a mass shooting doesn't have to have people get killed, maybe just shot." So, they started collecting data on a number of cases of four or more killed--I'm so sorry, four or more shot. And it's valuable data to be sure, but it's not the same as a mass killing. And unfortunately, people get confused. For example, the CBS News image here, there had been more than one mass shooting a day in the United States. But if you look at the list there, they're all mass killings with large victim count, so no wonder the American public gets confused. So, they're certainly important, but some injuries are minor whereas death is definitive. So, we tend to focus on cases in which four or more people get killed. Also in terms of the Gun Violence Archive, where people hear one--more than one a day and think about an epidemic, well, no one can say how many there were in 2010 or 2005. We don't have videos that go back before 2013 in terms of these kinds of mass shootings. We do in terms of mass killings.

The other factor in terms of public perception of course is the nature of the media. We go back--you know, when Sandy Hook happened, people were astonished and shocked that an elementary school could be the site of a mass shooting. Well, it wasn't the first. Back in 1989, there was a mass shooting at an elementary school in Stockton, California. Very few people remember it, probably because it wasn't on television. Back in 1989, we only had the major networks, CNN was just really in its infancy, we didn't have MSNBC or Fox News back then. But things today are different. We have cable news channels that cover these cases around the clock, marathon coverage, and we have satellite trucks that now can be on the scene within minutes carrying these images of children being led away from their schools with tears still fresh in their eyes. Seeing is believing, and Americans, by seeing these things, believe it's rampant and an epidemic.

So, we try to look at what gets covered the most. Back in 2000, Grant Duwe published an article showing that public mass shootings get far more coverage than family and felony cases, not surprisingly. So, one of the projects we'd look at: what aspects about mass public shootings attract the news media? And not surprisingly, it goes with the higher death tolls, younger offenders particularly in schools, in churches, cases involved terrorism, cases involved predominantly white victims, and an assailant who's arrested so that you can have continuing media coverage. This paper, by the way, will be coming out shortly in Homicide Studies. Now, the massive media attention raises the issue about contagion. Is there a price to this news coverage? To an extent, is there copycatting? Of course, there are some iconic mass killings that receive a lot of attention, most of them don't. And this issue of contagion certainly got a tremendous amount of credence by a widely cited paper by Sherry Towers and colleagues at Arizona State University that basically show that mass killings tend to raise the probability of another event for about 13 days. It incites or excites further killings. Now, clearly, that seems to indicate a contagion effect. But the thing is they looked at USA TODAY'S data mass shootings, most of those cases are family cases, are felony cases, and they receive very little media attention, if at all. And, unfortunately, although it's a terrific study, it did not include any measure of how much media coverage was given to these mass shootings. So, what we wanted to do is to sort of repeat that approach but look at the coverage that these cases receive and how does that tend to create contagion, if at all.

So, we collected data on the amount of media coverage, daily media coverage, from the year 2002 to 2018, how much media coverage about mass shootings in major newspapers, 13 major newspapers weighted by their circulations, also Associated Press, and finally television news programs. And as you can see, there's been an increase in terms of the number and the severity of mass shootings over the years. And paralleling that has been, in the gray, an increase in the amount of media coverage. So, they both have increased. The question is how can we disentangle the cause and effect here?

So, this shows you the amount of media coverage in terms of major papers leading up to a public mass shooting and then afterwards. So, clearly, the mass shootings create immense amounts of publicity, but there doesn't seem to be any run-up in publicity prior to the shootings, so at least there's no short-term contagion effect based on media coverage. The same thing tend--we tend to see in terms of the Associated Press and television networks; the lag in terms of peaks are somewhat different based on the practicalities about how soon can the news be reported based on television or daily newspapers. In addition to this graphical approach, we also did look at a multivariate point-process model, very similar to what Towers et al did. But we included the amount of media coverage as a variable, and essentially what we found was that media coverage did not tend to excite further mass shootings. In fact, mass shootings didn't excite further mass shootings, at least in the short-term. But certainly mass shootings did tend to create a tremendous amount of media coverage as shown in these figures here.

Now, previously I—[INDISTINCT] I mentioned it, I noted the No Notoriety Pledge that about over a hundred fifty criminologists signed on to a letter urging the media not to show the face of killers, not to mention their names. Well-meaning, certainly that it is, and I understand that in the--in terms of the victims, at least, the victim's families, it adds insults to injury when these offenders get tremendous amount of publicity. But in terms of contagion and copycatting, what really influences other people and likeminded individuals is not the actor, it's the act. So, for example, when there's a massacre in El Paso of Hispanics, other white supremacists applaud it, but they don't necessarily applaud the person, the name. Many of them don't even know the name of the perpetrator. What they applaud is what that person did. So, it doesn't matter what the guy's name is and what the face is, it's what he did. And, certainly, the media is not going to stop reporting the crime. They should indeed report the facts, and part of the factual information about a case is who did it, where they got the gun, certain information about their background, basic information. Unfortunately, sometimes they do go a little bit over the board in terms of humanizing the killer by presenting information that's really not necessary for our understanding of the crime. For example, the Virginia Tech shooter shown on the New York Times, all the news that fit to print, I guess they thought it was fitting to show him with guns pointed out. Now that's quite different than just a simple head shot. For example, with the Las Vegas killer, we learned so much about that person, what he ate for dinner on the night of the shooting, what shoe size he had worn, what casino games he liked. We even had pictures of him on his high school tennis team, as if that really is important. We knew more about him than our next door neighbors, and that's kind of a problem. So, a certain line is often crossed when they turn someone from an offender to almost a celebrity. Also, I should say that the media should stop publishing it in full length their rants and stop calling them manifestos, which they really aren't because that implies a far greater importance than is deserved. Finally, they should focus on strength and resilience. For example, in El Paso after the shooting, the members of the community lining up to give blood, that's a story that people should be hearing, not stories about tennis teams, pictures of a shooter.

In terms of contagion, not all contagion is media related. In fact, maybe contagion has to do with a public obsession, constant discussion. So, for example, this is a similar situation. These are school shootings, not necessarily mass shootings, but back in the late 1990's, there were eight separate multiple victim shootings at schools. It was a tremendous amount of attention in America on the news. The Department of Education sent pamphlets to every school in America about the warning signs. President Bill Clinton established an advisory committee of school shootings. And then in March of 2001, after shooting in Santee, California, Dan Rather declared school shootings a national epidemic. But then there wasn't another one for four years. The reason, probably, was the fact that we had 9/11, the attack on America. All of a sudden, people weren't talking about school shootings. They were talking about a different kind of threat, threats from abroad. International terrorism, al-Qaeda. And once people's attention was diverted, it just didn't continue to reinforce this idea that schools are dangerous places, and kids bring guns to school to get even with their classmates. And so, for four years, there wasn't another one.

Now, can that happen now? Maybe. In 2020, we have had only two public mass shootings. Of course, public places have been closed, churches closed, no concerts. Restaurants are closed, so that has contributed to the fact that there haven't been very many public mass shootings because there aren't places for people to convene. Now I'm hoping that this external threat, the pandemic, will be a distraction, very much like 9/11 was the distraction for school shootings, and then hopefully, time will tell, the panic and the social contagion, the constant talking and worrying, obsession about mass shootings will not reemerge once we get back to a normal lifestyle. So, I've talked about trends in the past. And so now I'm going to hand things over to Grant Duwe who's going to talk about what the future holds in terms of mass public shootings. Thank you. And I look forward to hearing any questions, and hopefully I didn't speak too quickly.

GRANT DUWE: Thank you, Jamie. So, a lot of the research that has been done on mass murder or mass shootings has really focused on trying to describe the people who commit these acts, those who get victimized, the incident characteristics, but there really hasn't been a whole lot of attention that's been focused on trying to control and predict these types of events. And so, for my presentation today, I'll be talking about a study that I did with my co-panelists and Nathan Sanders that attempts to forecast the severity of mass public shootings in the future. And as I'll show in more detail later on, both the incidents and severity of mass public shootings have increased since the latter part of the 2000's.

Two of the worst attacks or two of the worst attacks during this recent increase, as we all know, were the massacres that took place in Orlando and Las Vegas. A little more than 100 victims were shot in Orlando, 49 fatally, while nearly 500 victims were shot in Las Vegas with 60 people who ended up being mortally wounded. And in this study, we consider a couple of questions. One is: what is the likelihood of attacks as catastrophic as Orlando or Las Vegas occurring in the next five years or ten years or twenty years? And perhaps what's just as interesting, if not more so, is what is the probability of an even worse mass public shooting taking place in the future? And so, for this presentation, I will show results that we've been able to gather from a study that we've been working on that's currently under review. And to address these questions, we draw on research from other fields that have developed valid estimates of the future likelihood of rare catastrophic events. For example, seismologists have used procedures to predict the likelihood of severe earthquakes or earthquakes that exceed 7.0 on the Richter scale. Or, in a more recent example, researchers have estimated the odds the U.S. would experience a terrorist event as catastrophic as 9/11 at some point in the future. And this research has shown that these phenomena tend to follow heavy-tailed distributions such as power loss, which is to say that most events are relatively small in terms of severity while a small number of events are very severe. And we know that mass public shootings, even though, by definition, it is a very severe type of event, these phenomena also tend to follow heavy-tailed distributions. And so, what we see is that most mass public shootings that take place involve four or five or six fatal victims while relatively few fortunately involve more than ten, or most mass public shootings we see involve--or at least the total number of victims shot is less than twenty-five. And so, given that mass public shootings follow this heavy-tailed distribution, it makes it possible for us to forecast the severity of these types of events in the future.

So, one of the questions that may arise is why is it important for us to try to forecast the severity of mass public shootings? So, in other words, why does it matter? And I think the reason why it's important is that the forecast estimates that we develop, no matter what they turn out to be, can have implications for how we go about allocating resources in the future. If the likelihood is very low, for example, then it might call into question any policies or practices that we would have that would specifically focus on preventing or responding to catastrophic attacks but if on the other hand the estimates that we develop show the likelihood is higher, is greater than, say, 10% or if it's 20% or 30%, then it might be more relevant for us to do things like modeling the trauma capacity of regional hospital systems, developing and understanding of the potential consequences of prevention or mitigation strategies and policies, or even something like assessing risk around large public gatherings.

So, before I begin to talk a bit more about the methodology we used in the results, I think it's important to take a few slides to explain how we define mass public shootings specifically and how went about measuring these events. And I think it's, you know, given some of the confusion that's arisen over the phrase “mass shootings” and the number of definitions that have come about. In fact, one of my co-panelists, Jamie Fox, has said that there's been mass confusion that's risen over use of the phrase mass shootings, and I would agree. And so, I think it's important to try to clarify what we mean by that, and so with the mass murder, we're talking about any incident in which four or more people are killed within a 24-hour period. A mass shooting, although it's taken on a number of different definitions, could mean just any gun-related mass murder. And so, with the mass public shooting, we're looking at those incidents in which for or more victims are killed with a gun in a public location. And so, in doing so, we're excluding cases that occurred in connection with other criminal activity. And the mass public shootings that these--partly because they involved more victims who are killed and who are wounded, they tend to be more newsworthy because they take place in a public location. That also increases their newsworthiness and, you know, as you'll see, that here's some infamous examples of mass public shootings like Columbine, Virginia Tech, or Las Vegas. And Jamie noted that they make up about 20% to 25% of all mass shootings but when we consider mass murders more broadly, a mass killing committed with any type of weapon, we see that they make up about 12%. So, mass public shootings are a relatively infrequent type of event, even within the context of mass murder, which is itself infrequent but extreme type of violence.  

So, in measuring mass public shootings, we used what are considered best practices in terms of data collection. We used a triangulated data collection strategy where we initially relied on the FBI's supplementary homicide reports from 1976 to 2018. And, even though the SHR has a number of flaws, it is a relatively comprehensive data source on all homicides that take place in the U.S. It enabled us to, at least initially, identify when and where mass killings have occurred. But it does not contain information on the number of wounded victims or the specific type of location, and that's why it's important to gather more detailed information that come from news coverage. And so, using those two data sources together enabled us to comprehensively identify mass public shootings that took place over that 43-year-period and this is a strategy by the way that has been used not only by the Congressional Research Service in their own report that they disseminated in 2015, but also by USA TODAY.

In addition to this strategy, we consulted both published and un-published lists of mass public shootings. And then we also did a consensus review, FDR, essentially what we did is we looked at all the cases that potentially met our criteria for inclusion, and then we came to an agreement as to which case is to include and which one is not. And we ended up with an overall sample of 156 cases from 1976 to 2018. And so, that works out to a little less than four mass public shootings per year over that period, a total of 2360 victims who are shot of whom 1092 were killed. And so, when we look at trends in the incidents of mass public shootings over that 43-year-period, and if we express it in terms of a rates per 100 million of the US population over that period, we do see that there has been an increase since the latter 2000s. And the black line that you see represents a five-year moving average. And so, when you look at that black line, the five-year moving average, we do see that there is--that there is an uptick in the latter part of the 2000s. Really, I would say after Virginia Tech and then, as Jamie noted in his presentation, there has been a spike in the last several years, especially from 2017 onwards. And this is reflected in one of the two severity measures that we consider. One is the number of victims who were killed in mass public shootings. And once again we see an increase from the latter part of the 2000s up to the present and especially over the last couple of years, where there has been a decided spike in the severity of mass public shootings. And then the other severity outcome measure that we'll consider is the total number of victims shot in mass public shootings. And, once again, this is a rate per 100 million that it controls per population increases over time. And, in this slide, in order to more clearly depict the trends that we see, this excludes the Las Vegas case. So, even if we exclude Las Vegas from 2017 we still see a large increase in the severity of cases over roughly about the last 15 years.

So, when we're generating a forecast of the severity of mass public shootings, one of the key assumptions that we make is that the likelihood of a catastrophic attack is influenced by the future incidents of mass public shootings. In their paper that I referenced earlier on estimating the recurrence of something like 9/11, Clauset and Woodard used three sets of assumptions in estimating how frequent terrorist events would be. And so, one of their assumptions was a pessimistic assumption where it's assumed that the incidence of terrorism would be really high. In addition, they had a status quo assumption which assumed that everything would kind of stay normal or average based on what's been observed in the past. And then they had an optimistic scenario which assumed that the incidence of terrorism would be relatively low. And we followed that same approach here where we had three sets of assumptions about the future incidence of mass public shootings over the next 20 years.

And so, we grounded these estimates in the historical data that we have from 1976 to 2018. And so, for our pessimistic scenario we look at the highest prevalence of mass public shootings over that period of time. While the status quo just looked at the average mass public shooting rates in terms of incidence and severity while our optimistic scenario assumed a relatively low incidence and severity of mass public shootings. And in order to generate these estimates we relied on U.S. population projections to calculate the future number of mass public shootings for each scenario. And then, in addition to the scenarios that we considered, there were several other parameters for our forecast. For example, we looked at several different types of distributions, Pareto, Weibull, and Lognormal to help us generate those forecasts. We looked at three different forecast horizons. Meaning we looked at the probability of severe mass public shootings over the next five years, the next ten years, and the next twenty years. And then, specifically, we looked at several different severity outcomes. For example, we looked at the probability of something as catastrophic as Orlando, involving 49 fatal victims, as well as a mass public shooting as severe as Las Vegas, involving 60 fatal victims. But then we also considered the probability of mass public shootings that were even more severe than those, and then finally looking at total number of victims shot. One hundred, which would be similar to what we observed for Orlando, two hundred and fifty, five hundred which is similar to what we observed for Las Vegas, and then a mass public shooting with one thousand total victims shot.

So, the results that we saw varied quite a bit across the different parameters that we used and we don't have time to go through all the forecast. But instead I just like to focus on the results for the forecast where we used a 10-year horizon for the number of victims killed, and also to provide you with a sense of the variation across scenarios I'll focus just on the optimistic and pessimistic scenarios. And what these results indicate is that even though we do see, you know, some variation across the types of distributions that we use, our results suggest that there's at least a 38% probability that the United States will witness an attack as lethal as Orlando over the next 10 years. And this is under the most optimistic conditions. The probability for something as severe as Las Vegas would be at least twenty-six percent for a mass public shooting with at least seventy-five victims, it would be sixteen percent, and then it would be nine percent for a mass public shooting with at least one hundred fatal victims. And then when we look at those probabilities under pessimistic conditions, meaning if we assume that mass public shootings, that the incidence would be relatively high, we see that that probability increases to 51% for something as serious as Orlando, 37% for something as serious as Las Vegas, 24% for one involving 75 fatal victims, and then 13% for a mass public shooting with at least 100 fatal victims.

And then this slide just provides an example of the results that we obtained for Lognormal distribution across three forecast horizons. Given time constraints I won't really go through those too much but instead I'll go ahead and jump ahead to our results for total number of victim shot in mass public shootings. And here we see that the probabilities, whether it's an optimistic or a pessimistic condition, is relatively high for an incident involving at least 100 victims shot. But when we go to 250 or higher than the probabilities are relatively low. And a lot of it has to do with the fact that even with something like Las Vegas, where we saw almost 500 victims shot that was exceptionally rare, even within the context of mass public shootings.

And so just to wrap up here, so regardless of the forecast parameters we used, the odds are very low that we'll see a mass public shooting with as many total gunfire victims as Las Vegas. But the odds are not trivial though, when we focus on fatalities. We see--even under the most optimistic conditions that it's at least 26% likelihood for a mass public shooting with 60 victims killed and that's over the next 10 years, or close to 10% for a mass public shooting with 100 victims killed. And so, this does have implications for resource allocation decisions when it comes to law enforcement, medical professionals, policy makers. It is worth pointing out though there are some limitations to our study. One is that it can't really tell us where or exactly when a severe mass public shooting may transpire. And we did see quite a bit of variability across the parameters that we used, that the estimates strongly depended on assumptions that we made about the type of distribution and the tail location. We did use a strategy that's relatively novel for criminology but we think it provides valid estimates that enabled us to predict the probability of mass casualty events. And so, at this point I will go ahead and turn it over to my other co-panelist, Dr. Michael Rocque.

MICHAEL ROCQUE: Thank you, Grant. I appreciate everyone coming out here today and joining us for our talk on mass public shootings. These two presentations have been pretty interesting, hopefully for you guys, as well. And I'm really looking forward to hearing your thoughts and your questions at the end of the presentation. Today, what I'm going to be talking about represents two of the studies that we were funded to conduct by the National Institute of Justice. The first one is an examination of state gun laws and whether they have any correlation with incidence and severity of mass public shootings. And then the second one is an examination of foiled or averted mass public shootings.

So, the first one had been recently published, and I'll give you the citation in just a minute, and the second one is very much in progress, so two different stages of progress here to talk about. The first study was motivated really by the idea that mass public shootings, as my co-panelists have mentioned, instill a lot of fear in the public. They get a lot of media attention. And really, you all know as soon as a--as a really severe mass public shooting happens, we instantly see commentary about what we can do about it, right? And they typically revolve around mental illness or gun control. And both of those things are very controversial. And neither of them have a large amount of data to support policies. We also know that gun laws vary tremendously by state. And this slide is from The Guardian. It's just an illustration of how gun laws vary across states. They vary tremendously. And there are a lot of different gun laws.

So, the idea is: do gun laws have any relationship with mass public shootings? And to do this, we can look at across United States, over time, and look at whether any changes in gun laws have any effects on mass public shootings, and whether different laws have effects on the incidence of mass public shootings, or also severity in terms of lethality and the number of victims. So, this study really theoretically was motivated by Phil Cook's work, where he basically made the argument that gun violence is driven by availability of guns. So, in other words, more guns that are available lead to a higher incidence of gun violence. And this has been a controversial claim as well. You know, there has been research that has shown the opposite. But with respect to mass public shootings, it's also even more a little bit fine grain, right? Because these are very rare cases, and there are different ways that gun availability or gun control could influence mass public shootings.

And here, we just have two basic stories, two basic mechanisms. The first would be from a routine activities perspective, you have a motivated offender, somebody who has it in their head that they would like to commit a mass public shooting. And then they need the tools to do so, so if they are in a place where they have access to guns that could allow them to commit a mass public shooting, then that would allow them to do so and then the mass public shooting is more likely. And another way of looking at this is that gun availability, or the ease with which people can obtain guns, could actually influence motivation. And motivation, meaning the idea that someone has in their head of what they want to do. And what they want to do, say, shoot up a school, is even feasible, given the restrictions on getting guns. So, gun availability could actually influence motivation which then could influence the incidence and lethality of mass public shootings.

There has been in recent years a few studies that have been looking at this. And again, it's not, you know, brain surgery to make the connection that gun laws might be related to mass public shootings and to look at this. But the studies that have done this are somewhat mixed in terms of what they found. There were some previous studies who look at--basically, they try to classify gun law permissiveness, and they have found a relationship with mass public shootings. These three studies that are relatively recent have also looked at states. So, the variation in terms of states and their gun laws, and mass public shootings, and they have all found that certain laws, for example, assault weapon bans, and then LCM stands for large capacity magazine bans, are related to the incidence and the severity of mass public shootings. We looked at these studies and they're well done except they rely on somewhat of an idiosyncratic definition of state laws. You know, what determines whether a state has a may-issue law for example. And what happens when that changes. And secondly, the mass shooting data has a few limitations in terms of the sources. And so, we tend to think that our mass public shooting dataset overcomes some of these barriers. And so that offers us a way to add to this conversation in a unique way.

So, the thing that is really interesting about this study is that we were able to partner with Michael Siegel at Boston University who has developed a very strong dataset on gun laws across states. This is publicly available; the definitions are very clear. You can take a look at it. And so, what he did, was he took those--it was over a hundred state laws from 1991 to the present. And because our study goes all the way back to 1976, he took 89 of those state gun laws and retroactively collected the data on the states. For the purposes of this study, we focused on laws that would have a theoretical connection to mass public shootings. So those are the eight here as you see: assault weapons bans, large capacity magazine bans, permits, extreme risk protection orders, universal background checks, may-issue concealed-carry laws, relinquishment of guns for those who are prohibited from owning a gun, and then violent misdemeanor prohibitions. Again, the MPS, the mass public shooting dataset, utilizes a triangulated approach, which we have heard in both of the studies so far. So, I won't--I will not belabor the point. But the citation for the study, which was just recently published in--on human behavior is low.

So, to look at this, we used two different types of models. One is a logistic regression, so that’s whether or not a state has a mass public shooting in a particular year and then zero-inflated negative binomial for incidence and number of fatalities and victims. Now we did look at number of fatalities and a number of victims separately. The findings are very, very similar. We also controlled for state cluster, and we used Robust Standard Errors, and we included trends for time. The controls here, what you can see is these are some demographic controls and also the controls for the logistic regression for incidence. The ones that I highlighted here in red were used in the number of fatalities models. And so just to really briefly, our results--at the--what we're looking at, we have a hundred and fifty-five mass public shootings that resulted in one-thousand seventy-eight deaths and sixteen--almost seventeen hundred non-fatal injuries. Now note, the 155 number is a little bit--is one lower than what Grant mentioned in his presentation because we are excluding D.C. from this analysis. And here is just a map where you can see the average rate per million across the United States during this time period. And so, you can see these are in quartiles. You can see places like Maine, where I am, never--did not have a mass public shooting over this time period. So, we're quite proud of that. But places in the South and in the West, and Alaska actually has the highest incidence.

So, the question is: what is the relationship between gun laws and mass public shootings? And hopefully you can see this graphic--this is just a plot of the odds ratio in the 95% confidence intervals. And an odds ratio above one, that doesn't cross one, is statistically significant, meaning that there is a positive relationship, and then an odds ratio below one is a negative relationship. So, what you can see, these are just the significant variables, the ones that were non-significant are not included here. But what you can see is that permits--requiring a permit to own a gun, or a license, is negative, so that reduces the incidence of mass public shootings. And then the last row here, large capacity magazine bans, that's in red. So that's the--that's the severity model. So large capacity magazines, so in other words, had--were--banning magazines that can hold a lot of ammunition that results in fewer deaths. And it also results in fewer non-fatal victimizations per incident, okay? So, the models that we included, models for--whether there was an incident and then the number of victims per incident. So that's the first study.

The second one is one that is ongoing, and we got the motivation from the work of Eric Madfis and Christine Sarteschi, among others. And I've been working with a couple of really fantastic students. Madison Gerdes is a PhD student with Professor Fox. And Madeline Clark was an undergraduate student with me at Bates. And over the last two summers they have been helping me collect data on averted mass public shootings. And the impetus for this study is that we know quite a bit about mass public shootings, even though we know less than what we might hope to know because of the varying definitions and the varying datasets so on and so forth. But most of the research has been on mass public shootings that happened, right? So, there's a little bit of a selection effect there, if we want to know what exactly is provoking these mass public shootings. And then what can we do to actually prevent them. One of the things that we have often found in mass public shooting literature is that there's often leakage. Leakage meaning the person or people who are the perpetrators have made their plans known, whether in writing or speaking with somebody, so there's a warning there. And it seems like that would be a place where we could intervene. And so, this study is looking to compare mass public shootings that were completed with mass public shootings that were averted, or thwarted, or foiled. Do they look the same? Are there places in which we can intervene to save some lives?

So, at this point, there have been numerous studies that have looked at pre-attack behaviors or averted mass public shootings. The pre-attack behaviors, this is an FBI study that tried to make sense of what happens before the mass shooting occurs, or active shooting, as the FBI is oftentimes looking at, slightly different. But what they found was a vast majority of the people were spending more than a week in planning the attack. And they exhibited fortified concerning behaviors. And this could be something like mental illness or even leakage, right? Or threats. So, there are--there's warning signs. And there's an opportunity to study averted mass public shootings. These are not spur-of-the-moment incidents, in other words. And then there have been a couple of papers that have looked at averted mass public shootings. Laura Agnich has looked at averted mass school shootings. Daniels and colleagues have also looked at mass shootings in schools. Eric Madfis is also focused on mass shootings in schools. This is where most of the research has been done. And again, finding that there is a place where we can intervene. There's leakage. There is one study that just recently came out by Jason Silva and that looks at comparing--completed to averted mass public shootings. And that study found that there were some differences, right? So averted mass public shootings were more often younger, white, they had more co-conspirators, and there was less of a--sort of a target-specific approach. In other words, they didn't have two or three people that they were really trying to attack.

So, our study is similar to that. There are a few differences however. Ours looks--has a different dataset as we have mentioned over the course of these presentations, we think it's quite comprehensive. We also use a different credibility assessment, and I'll talk about that in just a second. So, one of the things that you really need to focus on and you really need to understand is whether a threat is credible. In other words, when someone said I want to shoot up my school, are they actually intending to do that, or are they just kind of spouting off? So, we want to be looking at credible threats. And we also--our averted mass shooting data come from a few different data sources as well. But this is--this is just adding to the literature, or at least we hope it will, anyway.

So, our definition, we had to start from a specific definition of averted mass public shooting, and that's actually a lot more difficult than you might think. How do you define something that never happened? And so basically, we use the same definition for mass public shooting except that the shooting didn't take place. So, we use the term plot, planned, or threat to shoot four or more individuals. Now, this is a little bit difficult because if people say something like, "I'm going to bring a gun to school," does that mean that they were intending to shoot four or more people? So, we had to be pretty specific in terms of how we--how we collected that data. There had to be a very specific plan that was--that was clear that four or more people were intended as targets. We also--in terms of our credibility assessment, so Silva required a gun, or plan to acquire a gun, so--and that makes sense because, you know, the idea is that this is something that could actually happen. Ours is a--is slightly different. We say that there has to be some sort of a detailed plan, right? So, it's not just spouting off but there's some planning going on, maybe maps, or a list of targets, or something like that, or weapons must be accessible. So, it could be a less detailed plan if they had access to a weapon. So that's a slightly different approach here.

So, to collect data, starting not last summer, but in 2019, we started with data sets, and we were really lucky that the scholars Eric Madfis and Laura Agnich, and Christine Sarteschi, were willing to share their datasets. And Madfis and Agnich, again, were focusing on schools, Christine Sarteschi is actually focusing her paper, which is published a couple years ago, looks at mass violence in general. So, we looked at those cases and we reviewed all of them to see whether they fit our definition. We also used the Averted School Violence website, there’s a K-12 School Shooting Database. There's some online lists that are available, and we just did searches of the media. This is an open-source data collection strategy where we're mostly using media sources. There are some times when we had access to criminal justice records when we could fill in some of the blanks. Each of the case was reviewed by at least myself and one of the students. Mostly, we focused on whether a case is credible or not and then we came to an agreement if there wasn't, and usually, we were on the same page. So, we developed the codebook and started collecting data in 2019. Three of us right now have been involved in collecting the data, and we were very active in terms of trying to tweak the codebook when it didn't seem to fit some of the cases, and also to make sure that we were all on the same page in terms of definitions.

So, one of the things that was curious to me was whether we were defining different categories and different variables the same. So, we decided that we were going to do a reliability assessment this summer. We did a trial. We found out that the trial was not really getting at what we were interested in. And then we conducted a 10-case reliability assessment. So, 10 cases from the dataset, the same sources for each of us, and then we coded what we call subjective fields, fields that are not objective, like age or gender. So, this includes things like mental illness, the length of the plan, the response, motivation, and credibility. And then a quick percent agreement, it looked like there was overall pretty good agreement, but there was some variation. Specifically, we ran some analysis here with Gwet's Agreement Coefficient, which is recommended for reliability. And for the most part, we're okay. Threat, plan, and threat length and the plan length were a little bit difficult, so how long had the people been planning this. We initially had it in--code it in the number of days, but that was difficult information to really discern from the sources. So, we broke it up into categories and we have a pretty good handle on it now. But the length of the plan, we know, is a little bit sketchy in terms of the reliability of the information.

So just some challenges, again, there's lots of hidden information; we know that there's probably a lot more cases that we just haven't been aware of. These cases have to be, generally speaking, they have to come to the attention of the media, and there are probably cases that did not. There are also cases where people had plans and maybe didn't leak. These are often--these are cases that leakage occurred. And so, we had to figure out a plan for identifying cases, but then also when we were having challenges with data collection, we had to figure out when to move on, when we have what we can get. The big thing that we focused on was credibility. So, we wanted to make sure that we all are on the same page with whether a case was credible or not, and that--as I said, two coders are doing that every case. And I just want to really briefly mention here, WhatsApp. We created a group chat where we were able to stay in touch pretty consistently and meet on a daily basis, and discuss cases, and discuss plans. And it was really, really helpful, rather than having to be on Zoom all day long, or have a phone call, which sometimes can be a little bit less efficient. WhatsApp was--I really recommend that for people working in group projects.

So what we have, as of this month, we have 210 cases in the averted, the date range does not go back to 1976. The earliest one that we have is from 1999, it's generally incident structured. There are some quantitative and qualitative variables. And this is very, very preliminary, but a quick comparison with--between averted and completed, so what Madison was able to do was code the completed data set, some of the variables in the similar way that we had coded the averted. And so, what we see here is the age, right? Perhaps not unexpected, and this is in line with what Silva found, the average age is lower for averted. Now, again, our data source is focused primarily on school shootings, so that's not to be--that's not too surprising. What is kind of interesting is that the percent male for averted is lower and then the percent white is higher. And then for mental illness, we see not too much of a difference, a little over 61% for completed and 40% for averted. Now, this is a very expansive definition of mental illness, this is whether some sort of mental health issue is mentioned at all in the case, not a formal diagnosis, okay?

So just to wrap things up, because I know that we're running out of time, for the state laws paper, we see that a permit and a large capacity magazine ban, these are related to the incident and then the severity of mass public shootings. This is across state, and this is in line with recent research. We are not quite clear on the mechanism, we did have a measure in that study of household gun ownership rates, and that was not related to mass public shooting incidence, interestingly enough. So, we're curious about people's thoughts on that. The averted mass public shooting project is continuing on, we found some similarities and some differences across completed versus averted, and we will have, in the future, a more specific plan to compare. Although I do say--I do want to say that I'm a little hesitant to do formal statistical comparisons of completed and averted because they are not, by any means, random events. So, I'll end that here, and I'll just say thank you very much.

MARY JO GIOVACCHINI: Thank you so much, Michael. We will go ahead, and we'll go back to Barbara.

BASIA E. LOPEZ: Yes. Hello, everyone. Again, to our panel, thank you very much for this presentation. I would like to ask you a couple of questions to--for you to answer. The first one is, what are the implications the findings have for the criminal justice system and on prevention?

JAMES ALAN FOX: Well, I think each of us in our areas that we've covered can say something. I'm going to focus, again, on the media, since that's really what my presentation concerned. There's a lot of media bashing and blaming in terms of creating the surge and epidemic, and I think that's misplaced. Again, that's--you know, why should mass shooters be different than other murderers? I mean, the newspapers, they can name people who killed one, two, or three, but they kill four, they can't name them? Or how about serial killers? We do biopics about them and--through crime books. And what about mob killers? So, I think it's impractical to treat mass shooters any different. And, as I've said, I don't think we should. It's part of the story. It's part of the news. But, again, there's a line that they sometimes cross that--and that--and part of it is because we want to know why. But we don't always know why. In case like Las Vegas, we don't know why. But in the process of collecting all this information, more than is needed, more than is relevant, we do a disservice. We do a disservice to the victims, their families. And a disservice by making the person larger than life. So, I think let's stop blaming the media for creating an epidemic when there is not an epidemic, as I've said. But, again, there's a certain degree of control that the media should take on themselves. So, in terms of the no notoriety pledge, well-meaning. And I think if the idea is, "Let's limit the amount of attention to what's necessary," that's fine, and I agree with that. But in terms of the total blackout of information, like, it's become very common place now when you have police saying, "We're not going to mention the person's name." That's just not practical. Besides, the name is going to come out in the social media anyway.

GRANT DUWE: So, I think the implication--I think there are probably several implications from the forecasting severity study that we did. I think one is that we do see that our estimates are influenced by the incidents of mass public shootings, so I do think that whatever policies and practices that we can implement to reduce the incidents of mass public shootings, that it will in turn likely reduce the probability of a catastrophic mass public shooting taking place in the future. But I also think that even though our forecasts don't specify exactly when or where a mass public shooting will take place, or by whom, I do think it's--it behooves us to try to plan, to the extent that we can, for events that our--that our studies suggest is not a trivial probability. That something as serious as Las Vegas could take place in the U.S. at some point over the next five to ten years. And so, communities that, you know, whether it's the regional hospital systems or law enforcement in terms of planning for large events, in--that take place in a public, that it will--it will be important to try to plan for those, in terms of having resources available.

MICHAEL ROCQUE: And I will say, with respect to the state gun laws, the policy implications of that study would seem to be pretty clear in terms of permits reducing the incidents of mass public shootings, and then large capacity magazines appear to be related to the severity of mass public shootings. And so that's a consistent finding that all--that--at least two other studies have found. And so, I think it's something that we need to take seriously as a potential way to reduce the lethality. And as we've seen with the other presentations, the lethality and severity of mass public shootings has increased over time. And so that's something that we can perhaps take a look at. With respect to averted, that one is, I think, a very interesting project from a policy perspective. Because really, the idea is, if we can study those who have a plan to complete a mass public shooting then--and then don't, maybe we can gain some insight on how these things can be thwarted in the future. On the other hand, it almost--that entire dataset is 210 incidents that no one got hurt. And so that shows, you know, what policies are working. And so, schools are--and states across the country are really pushing for us to take seriously whenever anybody says that they are thinking about harming themselves or others. That seems to clearly work. Whether we can look at the completed versus averted and figure out other types of policies, I think maybe if we look at the differences between completed and averted then we can see which incidents and which characteristics of the offender are more likely to actually succeed. And so, the Silva study, for example, finds that completed offenders are more likely to have a criminal record. So that's a place where we can maybe devote some resources on focusing on people who have a criminal record and trying to reduce their--the chances that they might actually commit violence. But we're very early in the stages of this study, and so I'm looking forward to seeing what the results show.

JAMES ALAN FOX: Yeah. Just one cautionary point here though is, you know, we start looking at the background of perpetrators, and whether it be weapon possession, or domestic violence, or any of the characteristics that Michael is talking about, this is a very rare event. And in trying to predict and identify the next mass shooter is virtually impossible. There's--it's a huge haystack of--it's needles in haystack. A huge haystack of people who have lots of characteristics that are commonplace among mass shooters, but very, very few needles who actually turn their anger and angst into action. And we have to be very careful about false positives because if we start trying to predict who people are, based on their common characteristics--and, by the way, I know that tomorrow there'll be a presentation about the common characteristics, and it's something we're working on too. That's interesting to know what characters are common. But if you think you can use that to identify and predict, you'll be sorely mistaken. And it's frustrating. Of course, it's a base rate problem. Very few cases compared to the number of people who might fit a profile.

BASIA E. LOPEZ: Thank you very much for these answers. One more very quick question before we give it to the audience. Based on your research this far, what do you see as the future direction for the research field?

JAMES ALAN FOX: I think there's a lot of misconceptions about mass killings and mass killers. One I know there's a question here about domestic violence as a precursor. I think what we need to do in the next few years is be real careful about trying to identify and correct a lot of the misconceptions and myths that are out there. This is a research area that's growing, as I showed the trends in terms of scholarship. In the infancy of this literature, there is a danger for a lot of half-baked ideas to get into the public airwaves and be misunderstood and misinterpreted. So, I think that's very critical that we--oh, and one other thing. I think we need to come to some agreement about what a mass shooting is. There's an indicator--there's confusion about mass shootings, and people hear about one mass shooting a day, based on the Gun Violence Archive, and they think about El Paso, or they think about Dayton, and Las Vegas, and--that these things are happening every day. They're not. So, the future direction is trying to get clarity, consensus, and correct a lot of the misconceptions that exist.

GRANT DUWE: So, I would have to say I'm probably a little less hopeful than Jamie is about reaching consensus on a definition, but I do think part of Mike's presentation, focusing on averted cases, I think because that is a newer line of research where there hasn't been as much done. That's not to denigrate the work that has been done. I think there's been some good research that's been done. But I see that as an area where we can make significant inroads to prevention. And just because I think there's more for us to learn there. I think there's a lot more headway that we can make in terms of research for the averted cases.

MICHAEL ROCQUE: I mean, obviously, I agree with what Grant just said there. But I do think that, piggybacking off of Jamie, I think that description is really important. One of the substudies of our project is to simply describe these cases, right? What do they look like? What are some trends over time? As Jamie mentioned, is this an epidemic? Are they increasing? It seems to me that when different data sources are agreeing on particular patterns or trends then we can sort of be more confident that it is a trend, and we can do something about it. You know, when I first started being interested in mass shootings, there was so few, what would I call, empirical studies that you couldn’t really make sense of anything. And now that--I could--if you Google scholar mass public shootings or mass shootings, you'll see, you know, dozens and dozens of studies. And so, we're starting to build a knowledge base on which we can perhaps try to figure out what types of policies and approaches could be effective. But I think we're still building that literature, even though there are some things that we feel pretty confident that we know about mass public shootings. For example, it's very clear that it's a male phenomenon, right? But whether it's guns or not, there's going to be controversy, regardless of what side you're on.

BASIA E. LOPEZ: All right. Thank you very much to all of our panelists. I will ask Danielle Crimmins now to join us, to read some of the questions that are in our Q&A.

DANIELLE CRIMMINS: I'm going to alternate between the three participants. And this first question is for Jamie. What level or percentage of first-person video games were consistent in the lives of the accurate shooters you have researched?

JAMES ALAN FOX: There's always an attempt to look for easy explanations and easy answers. And one of those is this idea that mass killers, mass shooters absorbed by video games and video games turn them into killers. It's actually perhaps the reverse, that certain individuals are fascinated, obsessed with violence. So their fascination with violence which is reflected in constant video game play, is also reflected in their behavior. So, people will point to say the shooter at Sandy Hook, that he had a tremendous--spent tremendous amounts of hours playing violent video games. By the way, he also spent tremendous amounts of hours playing nonviolent video games, like Dance Fever I think it was called. But if playing violent video games were really a predictor and a cause of mass shooters, boy, we'd have far more cases than we do now 'cause millions and millions of Americans play endless hours of violent video games, and they don't even consider committing murder. So, I think that's a red herring. Now in terms of the percentage, we have not collected those data. And I know tomorrow--I think The Violence Project has. But the point is, it's very hard data to find. A lot of missing information. And I think it's a dead-end frankly, and no pun intended. It's not really a factor in understanding why people become mass killers. There are other reasons, but that's certainly not one. That's a reflection of their behavior, not a cause of it.

DANIELLE CRIMMINS: This next question is for Grant. Very interesting, all those scary forecasts of future mass shooting. In your opinion which condition, example firearm legislation, location, et cetera, have the most influence on these forecasts?

GRANT DUWE: Some of the results that Mike presented suggest that there could potentially be some firearm legislations specific to mass public shootings that might have an impact on the incidence. We also see that location matters too, in terms of--and I think it's important to specify that location means like the population density of the location. Meaning that we do see more mass public shootings on a per capita basis for, you know, those places that have larger population sizes. And that's consistent with the map that Mike presented too. And so, I do think that both tend to have an impact, or at least they could influence the impact of the forecast that I presented in those slides just because we do see that the incidence matters in terms of like how common we're assuming mass public shootings to be. But I will say that all the assumptions that we used are consists [INDISTINCT] moving average over a three-year period. Like, when was the lowest five-year period? And that is what we assume a pretty optimistic forecast. Whereas for the pessimistic forecast, we look and say, you know, a five, or ten, or twenty-year moving average of the highest prevalence. And so, when we make those different assumptions about the incidents, it does have an impact on what those forecast show.

DANIELLE CRIMMINS: This next question is for Michael. What are your thoughts about red flag laws? Are you aware that any studies detailing petitions submitted to the court prohibiting others from obtaining firearms on the basis of gender, race, national origin, religious affiliation, et cetera.

MICHAEL ROCQUE: I am not aware of any research. Like I said in the talk, the really sophisticated research that has examined state laws and mass shootings is only about five--or a little bit more than that, years old. But I have heard about the--you know, the idea that red flag laws might be something that could be potentially useful. But I'm not aware of any research that has looked at it in a rigorous fashion. And the other co-panelists can jump in if they agree or disagree.

JAMES ALAN FOX: There's research on suicide and red flag laws. It's kind of interesting because when you look at the motivation, why red flag laws have been inactive, it's always following a mass killing or a high profile murder in Indiana and Connecticut, which were the first to establish red flag laws. They were associated with murders. And, of course, since Parkland shooting, we've seen a lot of talk about red flag laws, but there hasn't been any research on red flag laws and multiple homicide. I'm hoping there will be. And clearly given the proliferation of laws in the past couple of years, there's a wonderful opportunity. I'm trying to get some of my doctoral students to take that on as a dissertation, as a quasi-experiment. But nothing yet. Hopefully, we will see something.

DANIELLE CRIMMINS: Moving to the next question. This one is for Jamie. According to several sources, more than a hundred people are killed with guns and two hundred more are shot and wounded on a daily basis in the US. Do you believe that there should be more news coverage detail--dealing with violence related to guns?

JAMES ALAN FOX: Yes, there should. And what's unfortunate is that mass killings that represent far less than one percent of gun homicides receive the overwhelming majority of attention in the press. Those are the hardest cases to deal with because they are so rare. And many of the proposals that we have for dealing with mass killings, may or may not work. You know, it's--and then we have something that happens about four, five, six times a year, it's even harder to determine whether there's an impact given the low level of frequency. A lot of procedures, in fact, that had been proposed, whether it be red flag laws, or changing the age related to purchases of weapons, or limits on magazine size, et cetera, et cetera, et cetera, these may be good ideas, but in terms of mass killers, these are very determined individuals, very difficult people to stop. They are willing to die to get even with whatever--at the enemy that they see in their lives. And oftentimes they will get the gun, or get the weapon, that they need regardless of what roadblocks you put in their path. But that doesn't mean we shouldn't try. We should. In fact, a lot of the proposals for gun legislation would likely have an effect on the majority of homicide we see every day in America. You know, the weekend of El Paso and Dayton, there was--thirty-three people were killed that weekend in two mass killings. That's the average number of people killed in gun homicides every day in America. That's what the focus should be, really, in terms of prevention. Those are the cases that we--perhaps to make the greatest inroads. So, the irony is the mass killings are often fuel and motivation for doing something about guns yet the least likely to be impacted by those same procedures. The ones that could be impacted are the homicides we see every day.

DANIELLE CRIMMINS: The next question is for Grant. Can you explain how these forecasts influence resource allocation and practice? If we cannot say when or where these mass public shootings may occur with any specificity, how can one prevent or mitigate these incidents?

GRANT DUWE: Yeah, so I think I sort of tried to address the question to some extent earlier. I think what the results from our forecasting study indicate probably have more implications for how we respond and potentially mitigate the severity of these incidents as opposed to prevention. And what I mean by that is, you know, like modeling trauma capacity for regional healthcare systems, you know, in the response to a mass casualty event, you know? Our result suggests that something on the order of a mass public shooting involving more than fifty fatal victims or more than a hundred victims shot that's something that some community within our country will likely have to deal with at some point over the next five to ten years. So, I think building up or at least having that capacity and planning for it would be important. And I think a similar principle applies to law enforcement when it comes to, you know, planning for large public events that our study suggests that, you know, those potentially could be venues where we could see a large-scale act take place.

DANIELLE CRIMMINS: The next question, I believe, is for Michael. It came in during your presentation but it is not directed towards you. The other panelist can feel free to jump in. Can you please define felony versus family versus public?

MICHAEL ROCQUE: Yeah, I'm not sure that was related to my talk, but the felony versus family versus public is--in terms of how we define our universe of cases here, we're focusing on mass public shootings. And that was defined as any event with four or more gun deaths, not including the perpetrator, in a public place. And we specifically said not related to another type of crime. And so, a felony-related mass public shooting would be something like if somebody walked into a McDonald's and attempted to rob the place and then four more people were shot. That would be felony-related. And then a family--Jamie called them family annihilations. These are cases where--and these are much more prevalent, where generally speaking a man is having some sort of an issue and then he kills his wife and children. And so that's a mass shooting but it's not a public shooting and it's a familicide, which is typically thought as much different in terms of motivation than a mass public shooting. And also, as Jamie mentioned, something that--those are events that they're not tend to capture our collective attention as much because they seem like problem for other people and not us.

DANIELLE CRIMMINS: The next question is not directed at anyone specifically. So, regarding the uptake in mass public shootings the past few years, is there a link between this increase and the use of higher capacity magazines and weapons and the choice of where a shooter chooses to commit the crime or both?

JAMES ALAN FOX: I think that was partially answered by Michael's presentation already. In terms of location though, I mean, the rise and the number of hate-motivated mass killings certainly has impacted location such as church shootings or shooting at the synagogue in Pittsburgh.

JAMES ALAN FOX: Someone asked about the role of domestic violence as a free person, and what percentage of mass killers have domestic violence in their background and can we use that as a predictor? Well, there's a lot of misunderstanding there. A few years ago, Every Town for Gun Safety did a report on mass shootings. And they indicate in their report that fifty-four percent of mass shootings involve--in four or more people killed involve domestic violence. All the news stories in the wake of that claimed that 54% of mass killers had domestic violence in their background. That's not what Every Town for Gun Safety reported. The 54% were either family massacres or occasionally a non-family massacre where there was domestic violence in the background. I mean, the 54% of domestic violence was the mass killing, not prior mass killings. If you look at prior events, in terms of mass killings, we're only talking about around--it depends on how you define domestic violence. If you're looking at official reports of domestic violence or just rumors, 25% give or take have domestic violence in their background. And most of those are family massacres where there was prior domestic violence. If you're talking about public mass shootings, less than 10% have domestic violence in the background of the shooter. In fact, many of these guys live alone. They don't have family around. They're not married and don't have kids. So, in terms of mass public shootings, it is relatively rare that domestic is precursor. Now is domestic violence an issue for society? Absolutely. And unfortunately there's millions and millions of cases of domestic violence every year, but as a predictor of mass murder, not so much.

DANIELLE CRIMMINS: I'm going to look at the questions that are just coming in and they're not directed towards anyone. So just feel free to jump in. Any advice for how law enforcement can play a more viable role in the interruption in these events?

MICHAEL ROCQUE: Well, I'll jump in here and I will say with respect to averted mass public shootings, a lot of the research has been done in school, and so the School Security Officers, or SROs, have a huge role to play in creating a culture in which students know that they shouldn't be silent, right? Eric Madfis, in his work, tends to show that there is this sort of expectation that if you're friends with someone who is maybe making comments that appear like they could be heading down that path, you want to protect them by not saying anything, right? This code of silence. And that's troublesome. And so, the School Resource Officer can be instrumental in terms of the culture that is created. But, also, they're there. They're someone who can intervene, and also teach the students warning signs and things to look out for. So, from that perspective, I would say that law enforcement can be really instrumental in reducing these incidents.

DANIELLE CRIMMINS: Here we go. Here's one for the group. I know this is related just for guns but what happens if the suspect uses a knife and killed four or five, do you consider this a mass killing?

JAMES ALAN FOX: It is a mass killing. In fact, I mentioned in the--if I can give a plug of it, Associated Press/USA TODAY/Northeastern University Mass Killings database. We have a database of every mass killing of four or more people killed. Now about 19% of mass killings involve weapons other than a gun. And what's interesting is--and I've looked at the amount of media attention that mass killings without a gun get compared to mass killings with a gun, and there is exorbitant difference. And that's probably because the whole issue of gun control is not debated in the wake of a mass killing when someone uses a knife. There's no discussion about knife control. So, it's amazing how much different attention there is between those two crimes. But when you consider the fact that people are still dead, it doesn't matter if they're stabbed or shot, they're still dead. And it still matters. And it also say about the other kinds of--Michael mentioned how family massacres get very little attention. And that's true. Family massacres are hardly reported in the press. Local press but not the national press. And let's also keep in mind that it doesn't matter if the--someone who kills you was a stranger or a family member, you're still dead. So, I think that we should be as concerned about family massacres as we are about public massacres. And we should be concerned as much about knife killings, and bombings, and deaths from arson as we do with shootings, it’s just that shootings get a lot of attention because of the debate about the Second Amendment.

BASIA E. LOPEZ: So, thank you very much, Danielle. And with this, we will start wrapping up. I want to, again, thank our panelist. Thank you, James, thank you, Grant, and thank you, Michael, for presenting and having this conversation with us. We're looking forward to other findings that your project will eventually produce. We anticipate that they will come within a year or so. For those of you who are willing to attend tomorrow's webinar. It is the second part. We will have different research teams and looking at different topics but they will all pertain to mass public shooting. With this, I want to thank you very much.

MARY JO GIOVACCHINI: Thank you very much everyone for attending, and we appreciate your time. So, on behalf of NIJ and all of our presenters, again, thank you for today.