NIJ-Funded Research on Mass Shootings to Advance Evidence-Based Policy and Practice

DARYL FOX: Good afternoon, everyone and welcome to today’s webinar, NIJ-funded Research on Mass Shootings to Advance Evidence-based Policy and Practice, hosted by the National Institute of Justice. At this time, it’s my pleasure to introduce Basia Lopez, Social Science Analyst within the National Institute of Justice for some welcoming remarks and introductions.

BARBARA “BASIA” LOPEZ: Thank you very much, Daryl. Before we start, I would like to bring to your attention this important notification. The research projects presented here are supported by awards--NIJ-funded awards, the numbers you can see here on the screen from the NIJ of the U.S. DOJ. The opinions, findings, and conclusions or recommendations expressed during this webinar are those of the authors and do not necessarily reflect the position or policies of the Department of Justice. Again, my name is Basia Lopez. I am a Social Science Research Analyst here at NIJ. And I oversee federally funded research evaluation and data collection projects related to firearms violence, including mass shootings. Before we give the floor to our panelists, I want to provide--I want to provide you with a brief overview of NIJ’s efforts in research on mass shootings.

Our institute pretty much since the ‘80s has sponsored research in an effort to assist our nation in reducing firearms violence in general, by supporting rigorous studies and evaluations of programs that build knowledge and advance evidence-based practices aimed at reduction, prevention, and intervention. Building on various previous projects on mass violence, such as terrorism and school shootings, carried out by NIJ including safe schools--safety initiative as the--as far back as the late ‘90s or early 2000s, and more recently, several projects funded under Comprehensive School Safety Initiative. Since 2018, NIJ prioritized the research on mass public shootings and solicited for research projects on this topic through investigator-initiated research and evaluation on firearms violence competitive solicitation. The funded projects focused on basic understanding of public mass shooting events, victims, and perpetrators. Findings focus on the--focus on the implications of criminal justice prevention and intervention efforts. Description of this project can be found on nij.gov website under current awards by searching mass shootings.

Outside of external efforts or extramural efforts, NIJ is also carrying out various intramural activities in this area. In 2018, for example, we convened with subject matter experts from the fields of both research and practice to learn these subject matter experts’ perspectives and identify pressing research needs in the area of mass shootings. The following year, NIJ convened a meeting with grantees conducting research on terrorism and mass shootings to discuss implications of studying rare events, which mass shootings are. Last year, NIJ and the FBI joined to carry out a project that focuses on interviewing incarcerated active shooters to further inform intervention and prevention of these incidents. Today, we will hear from the principal investigator, Dr. Karie Gibson on this specific project. But before that, we will hear from Dr. Grant Duwe and Dr. John Hollywood, who will report on their studies funded by NIJ in 2018 and ‘19. With this, I would like to introduce Dr. Rachel Davies, who is a Science and Technology Policy Fellow with the American Association for the Advancement of Science and otherwise known as AAAS. She’s currently conducting her fellowship at NIJ. And she will facilitate this webinar and Q&A session afterwards. Rachel.

DR. RACHEL DAVIES: Thank you very much. Now, I am pleased to have the honor of introducing our speakers for their presentations today. First, we have Dr. Grant Duwe. Dr. Duwe is Research Director for the Minnesota Department of Corrections where he forecasts the state’s prison population, develops assessment instruments, and conducts research studies and program evaluations. Dr. Duwe is the author of a book on mass murder, and he has published more than 80 articles in peer-reviewed academic journals on a wide variety of topics, including mass violence. Dr. Duwe has served as a consultant for think tanks, academic institutions, and the National Institute of Justice, where in that capacity he co-developed the risk assessment instrument currently being used by the Federal Bureau of Prisons. I’ll turn it over to Dr. Duwe now for his presentation. Please submit questions via the Q&A function and we’ll get to them in the final section of the webinar.

DR. GRANT DUWE: Thank you very much, Rachel. It is a pleasure to be with you here this--here today. And Daryl, if you could go to the next slide, please? So, for my presentation today, I’ll be--I’ll be talking--I’ll be sharing with you preliminary findings from a study that’s underway that I’m doing with my co-authors, Nathan Sanders, Michael Rocque, and Jamie Fox, where we are attempting to estimate the global prevalence of mass public shootings. And this study really focuses on two questions in particular. The first is, what is the worldwide prevalence of mass public shootings? But the second question, which is arguably the more interesting and important one is, what percentage of mass public shootings occur in the--in the United States? And that question really gets at the issue of is--are mass public shootings a uniquely American phenomenon? Or is the United States just like the rest of the world when it comes to mass public shootings? And before I dive into the preliminary findings from the study, and in talking about the data in the proposed methodology, I think it’s critical to go over what previous research has shown about this. Next slide, please.

So, this is a relatively new area of research, looking at--which is focused on the percentage of mass public or the U.S. proportion of cases for mass public shootings. And one of the earliest studies came out in 2014, which isn’t mentioned on this slide but in that study, it was--it was found that 66% of mass shootings that occurred from 1983 to 2013 took place in the United States. But there are three more recent studies that have attracted quite a bit of attention. The first one was in 2016 by Adam Lankford. And in that study, he found that the United States accounts for 31% of mass public shootings from 1966 to 2012, given that the U.S. makes up about 4% of the world’s population, one of the implications from the study is that the U.S. percentage of cases is about eight times greater than its population would indicate. But then in 2019, John Lott and Carl Moody carried out their own study in which they found that the U.S. accounts for only 3% of mass public shootings from 1998 to 2012. One of the--one of the notable distinctions with their study, compared to the one by Adam Lankford, is they included terrorism and acts of genocide. And as Adam Lankford pointed out in his rebuttal to their study, a lot of the cases within their dataset involved incidents with multiple offenders. And usually what we see with mass public shootings is, as Adam Lankford pointed out, is they’re typically perpetrated by a single individual. In fact, when we look at mass public shootings, it’s at least 95%, whereas only about 10% of the cases in their dataset involved incidents carried out by a lone individual. And so when Professor Lankford just focused on the single offender attacks, he found that the U.S. still accounts for about 30% of those cases. Next slide, please.

So, when we look at this research, we’re left with the conclusion that either the U.S. is just like the rest of the world when it comes to mass shootings or the rate of incidence is about eight times higher than what the U.S. population would indicate. But I do--I do think there’s reason to be a little skeptical of the findings from both sides, especially when we consider that both Lankford’s research as well as Lott and Moody’s research, that they had almost sole reliance on news coverage as a data source. With the Lankford study, he used news archive and internet searches and also published lists, which have in turn also relied strictly on news coverage, like with the NYPD active shooter list, excuse me. And Lott and Moody are relying on the Global Terrorism Database, which also relied largely on news coverage as a data source.

But when--we also know from research that has used multiple data sources that--studies that just rely on news coverage that they’re going to miss cases. We know this from research that was done back in 2013 by reporters at USA Today. We also know that from research carried out by the Congressional Research Service in 2015, as well as research that I’ve done by myself, as well as with my co-principal investigators for this NIJ-funded project with Jamie Fox and Michael Rocque, the FDR research where we see that when you combine the FBI’s supplementary homicide reports data with re--with news coverage, you’re going to achieve a more complete dataset. And so, when we compare the FDR data from our projects over the 1976 to 2012 period with both Lankford’s data as well as Lott and Moody’s data, we see that they’re missing a relatively large number of cases, and that missingness for the U.S. cases has implications for estimates that we make about the incidence of mass public shootings outside the United States. Next slide, please.

But before I get to that, I think it’s important to kind of clarify what I mean when I say mass public shootings. So, like, when we look at the definition of mass murder, so these are--these are incidents in which four or more victims are killed within a 24-hour period. The phrase “mass shooting”--there have been a lot of ways in which people have defined that. How we’ve defined it in--for our project, those mass--a mass shooting would be any gun-related mass murder. But with a mass public shooting, we had a more specific definition in which these included incidents in which four or more victims are killed with a gun in a public location, excluding cases that occur in connection with other criminal activity, military conflicts, or collective violence, and this definition of mass public shooting is one that I’ve used in my own research, that predates this strand of research that has looked at mass public shootings from an international perspective. And in using this definition, it would exclude a lot of the cases that Lott and Moody included in their own mass shooting dataset, like, those would be cases that I think they would certainly classify as mass murders or mass shootings, but they would not meet the criteria for a mass public shooting. The mass public shooting definition that we have used is very similar to the one used by Adam Lankford in his--in his research including the 2016 and 2019 studies. Next slide, please.

So, when we take a closer look at the Lankford and Lott and Moody dataset, so with Adam Lankford’s 2016 data, he found 292 mass public shootings that took place across the world from 1966 to 2012. So, of the 292, there were 89 that occurred in the U.S. and then because the FDR data, we started in 1976 and so when we focus on that 1976 to 2012 period, we see that there were 84 cases in the U.S. But then, when we looked at those 84 cases, there were two in particular that would not meet mass public shooting criteria because there is one that arose primarily out of other criminal activity while there’s another one that occurred mostly in private--in a private residence. But of those 84, there would still be 82 that would meet mass public shooting criteria. And then there were 203 cases that took place outside the U.S. And just to--just to make it clear that both Lankford and Lott and Moody included assertions within their papers that implied that the datasets that they had were very comprehensive. So, you know, as an example in Adam Lankford’s study, it was reported that it included all known mass shooters who attacked anywhere on the globe from 1966 to 2012, and killed a minimum of four victims. And then when we look at the Lott and Moody dataset from 2019, that contained almost 1,500 incidents over that 15-year period. Of those, there were 43 that occurred in the U.S. But when we--when we looked at each of those 43 cases, there was one that occurred in Wisconsin in a residence that wouldn’t actually meet mass public shooting criteria, but there were still 42. And in their study, Lott and Moody state, “We are confident that we have all the public mass shootings for the U.S. and perhaps for Europe.” Next slide, please.

So, when we look at the three datasets, the Adam Lankford’s dataset, Lott and Moody’s dataset, and then the FDR dataset, we see a relatively large percentage of missing cases. So for our dataset, we identified 165 mass public shootings in the U.S. from 1976 to 2020. But when we focus on the 1976 to 2012 period, our dataset contained 124 mass public shootings. And of those 124, 82 were identified by Adam Lankford. And so, there were 42 cases that were--that he missed during that 1976 to 2012 period. So, he identified roughly about two-thirds of the mass public shootings that took place within the U.S., and that’s roughly the same percentage that we see for the Lott and Moody dataset over the 15-year period from 1998 to 2012. So during that--that period of time, there were 67 mass public shootings that we found in the U.S. And then Lott and Moody found 42. So they missed 25 cases, they identified 63% and missed 37%. And so, you know, one of the questions that we had in looking at the cases that were missed is, were--are there any common threads running through the cases that they missed? And what we see is that the missingness is not random. Next slide, please.

So I’ll take a couple minutes in going over this slide, because this is--this is critical. And so on the left-hand side are the incidents by the number of victims who were killed. And then we see the Lankford columns, and then the Lott and Moody columns. And so what the hit column indicates is that this was a case included by Adam Lankford, the miss column, this was the case that he missed. And then the total column comes from the FDR dataset. So like when we look at cases involving four fatal victims, there were 51--of the 124 from 1976 to 2012, the FDR dataset had 51. Of these, Adam Lankford identified 21 but missed 30, so the miss rate was about 59%. When we look at the Lott and Moody dataset for four victim cases, we see there are 27, Lott and Moody identified eight of those, so they missed about 70%. We see that the missing percentage goes down when we look at cases involving five fatal victims, for example, there were 24 from the 1976 to 2012 period. Adam Lankford identified about two-thirds of those, whereas with Lott and Moody, there are 13 cases, they identified eight, missed five. They--both datasets only missed about 10% of cases involving six fatal victims. And then, when we look at cases involving more than nine fatal victims, we see that neither Lankford nor Lott and Moody missed any of those. So they identified all of the cases involving larger numbers of fatal victims. Next slide, please.

So, you know, what this information tells us is that--is that one, that the lower severity mass public shootings like cases involving four, five, six victims, those are more numerous. In doing so, they tend to follow what’s known as like a--like a heavy tail or flat-tailed distribution. In fact, we see about 77% of the cases in the U.S. involved between four and six fatal victims. The extreme cases, those involving 10 or more, they’re more rare and they’re also more newsworthy. And so, you know, one of the issues with the cases involving fewer fatal victims is that they’re not quite as newsworthy. They still attract attention, of course, from the news media, but those cases don’t get quite as much coverage. And also, you know, especially with Professor Lankford’s dataset, because it goes further back in time, you know, when we just ran a simple logistic regression model on the data, we see that, like, the number of fatal victims is a significant predictor of whether a case was missed or identified. But so was the year in which the incident took place. So, cases that occurred farther back in time were significantly more likely to be missed and that’s because news coverage, when you use internet searches, published lists, and so on, that they rely on news coverage, the farther back in time it is, the more difficult it is to access that coverage. And so that’s why we see that pattern with both the Lott and Moody and Lankford’s data, that they’re missing the lower severity cases. And in doing so, they actually underrepresented the incidents of mass public shootings in the U.S. So, you know, one of the key questions then is, well, you know, what are the implications of that for the non-U.S. cases? Because when we combine the data from Adam Lankford’s study with that from the 2019 Lott and Moody study, we see that they identified 218 non-U.S. mass public shootings from 1976 to 2012. But what does the victim count distribution look like? Do most of those cases involve less than seven fatal victims like the--like the U.S. cases? Next slide, please.

So, in this slide, the red line, this is the percentage of the 124 U.S. cases within the FDR dataset. And we contrast that with the 218 non-U.S. cases identified by Lankford and Lott and Moody. And so what we see is that for the U.S. cases, 41% involved four fatal victims, 19% involved five fatal victims, about 16% involved six fatal victims, and then under 10%, seven, and eight, and nine, and so on. But when we look at the victim counts for the non-U.S. cases, we see that less than 10% involved four fatal victims, less than 10% involved five. In fact, like when we look at the non-U.S. cases for the Lankford and Lott and Moody data, we see that about 40% involved less than seven compared to the 77% that we see for the U.S. cases. And so what this strongly suggests is that, that they’re missing for their non-U.S. cases, they’re missing quite a few cases that were lower severity. Next slide, please.

So--and so when we assume that what--that the Lankford and Lott and Moody data that it captured all of the extreme cases, like those cases involving 10 or more victims, so from their data over the 1976 to 2012 period, there were 49 non-U.S. cases with 10 or more fatal victims. So from the FDR dataset, we had nine U.S. cases with 10 or more fatal victims. And so when we just look at those, we see that the U.S. accounts for 15.5% of all mass public shootings with 10 or more fatal victims. And so what we tried to do is, is develop an initial estimate of missingness for cases involving less than 10 victims and so we assume that the victim count distribution for the non-U.S. cases was similar to what we see in the U.S. So, as an example, that 41% of all of the non-U.S. cases involved four fatal victims, 19% involved five fatal victims, and so on. Next slide, please. So when we look at the estimated global prevalence of mass public shootings from 1976 to 2012, we come up with an overall estimate of a little less than 800. And so the non-U.S. cases that we see an estimated total of 673, so what this implies is that the U.S. accounts for about 16% of mass public shootings worldwide, which is kind of somewhere in the middle between what Adam Lankford reported and what Lott and Moody have reported. Next slide, please.

So, just to wrap up here, the--these are initial estimates, we will be developing more refined estimates using hierarchical Bayesian models, in which we’re going to use the series of heavy-tailed distributions like power law, lognormal distributions, etc., to help estimate the missingness in the non-U.S. cases. And this is similar to prior work that we’ve done, where we forecasted the severity of mass public shootings in the U.S. And if the additional estimates that we develop, if they are similar, then, you know, what--what’s the--what’s the key takeaway? And I think--I think the main takeaway is that the results suggest that the U.S. is not like the rest of the world when it comes to mass public shootings. But also that mass public shootings are not a uniquely American phenomenon. You know, and still with a--with an incidence that’s about four times higher relative to the U.S. population size, you know, I think what it would suggest is that mass public shootings are one of the most visible manifestations of violence in a country that has long been more violent than other comparable countries. So, at that--at this point, that’s the end of my presentation, and I will turn it back over to Rachel. Thank you.

DR. RACHEL DAVIES: Thank you very much, Dr. Duwe. As a reminder to attendees, you can continuously submit questions using the Q&A function, and along with other panelists, Dr. Duwe will be answering questions after all presentations and the discussion section have concluded. Our next presenter is Dr. John S. Hollywood. Dr. Hollywood is a Senior Operations Researcher at RAND and leads RAND Center for Quality Policing. He has conducted decision science and systems engineering-related research in the areas of criminal justice, homeland security, and information technology, including research on terrorism and mass attacks, for close to two decades. His projects include a series of studies examining how U.S. terror plots were foiled and what went wrong when they reached execution, leading a technology evaluation on a major active shooting exercise in Grand Central Terminal, leading expert panels to determine science and technology-related needs for the Department of Justice and Homeland Security, and developing methods for fusing and prioritizing disparate reports about potential terrorist threats. He has written opinion pieces on best practices in counterterrorism and law enforcement technology for The Hill, United Press International, CNN.com, and the Charlotte Observer. With that, I’ll turn it over to Dr. Hollywood.

DR. JOHN HOLLYWOOD: Thank you. It’s great to be here today. Today, I’m going to be talking about a study that will soon be publishing an online educational toolkit on helping prevent and respond to mass shootings, as well as other types of mass attacks. With that, next slide, please. So I wanted to start out with after our team’s finding, coding, reviewing over 600 cases related to mass attack plots, over 200 references of dozens of expert interviews, I wanted to start by giving you some bottom line upfront, three things I’d like you to go away with here today. First, we call the Power to Prevent. Everyone, both professionals, practitioners, but also the public can help prevent terror plots. In general, we found that about two-thirds of our foiled plots, which was around 300, about half the cases, were found as a result of public reporting, someone contacting authorities with their concerns. We also found that most of the cases that were reported in were vetted, assessed, so some sort of intervention often by, you know, sometimes by law enforcement, sometimes not. And most of these were really handled outside of the justice system. So, you know, most cases did end up being handled often with people getting the help that they needed.

The second is what we call Relentless Follow-Up. And this is the importance of bringing together multiorganizational teams of not just law enforcement, but also necessary services, relevant people in the community, can be schools depending on the nature of things, medical support to be--to be able to assess and then follow up with people who’ve been reported, and that addition to determining what to be done with people, you also need to be able to follow up and then following that, reassess regularly to see how things are going to determine if additional steps are needed.

The third we’ve referred to as Prepare and Train, as you can see from--this is from one of our interviewees, heroes are made because they prepare. So this means to be able to have advanced planning, resourcing, and training, whether tabletop or live exercises, for all the major partners who are going to be involved in responding to mass attacks. Also need to do preparations for actions after the attack is over to help provide for bystanders, survivors, the first responders, as well as to build resilience from the community and learn from what happened during the attack. Next slide, please.

So that very briefly onto our--for the rest of today’s presentation, I’ll briefly talk about objectives and methodology of the study. I’ll then talk about findings, covering ways to improve the prevention and mitigation of mass attacks, and improve follow-up afterwards, then cover some findings that will be of interest to policymakers and executives who are in certain positions to oversee as well as provide funding. Then I’ll talk about where we’re going next with the online educational toolkit and dissemination. Next slide, please.

So first on--first on the objectives, and I apologize a little bit for the formatting here. Objectives are from an outcomes perspective threefold, to improve the prevention of attacks, to improve the immediate--to improve the immediate response to how we are able to respond to those attacks in progress and reduce casualties, and then to improve the actions after attacks. And this one actually was added as a result of our research. It just--we just learned about the importance of a post-attack follow-up. Also then wanted to improve the understandings and capture the findings and educational toolkit. That’ll be rather than writing a traditional academic report, wanted to make the findings much more accessible and easy to use for a full range of professionals, policymakers, and the public. The idea is to provide information at three levels, first top takeaways, but then also more detailed information for people in different roles who need it. And then outbound links to core references and guidances to provide further reading on topics that are of most interest and concern to the readers. Next slide, please.

Then a little bit on methodology. First part of this is doing a case review of looking at--we--identifying through combination of existing databases, as well as additional targeted internet searches over the past years, mentioned, discovered over 600 foiled and executed plots, with over 1,000 cases considered, over 60 fields tracked and coded. We then also carried out interviews of dozens of federal, state, and local SMEs from a wide variety of different kind--types of expertise. Reviewed well over 200 journal articles, guidance books, manuals, policies, then we’ve been able to synthesize all those together, filed through all the material that we’d assembled, curated it, and then used that to draft and program a--an online educational toolkit. We got feedback on that toolkit from about a dozen experts followed by an academic QA process. And then from there, went to formal publication, which is the stuff that we are now looking at professional editing and programing. Next slide.

So, turning to our findings, I think the first thing that we were able to glean from all of this research was how to think about the range of actions that are needed to prevent and respond to attacks. So we’re able to identify three broad phases of defenses against mass attacks, collectively forming what we call The Mass Attack Defense Chain. Broadly speaking, that chain has three levels, first of which phase one: things done to prevent attacks, identify, assess, and hopefully divert attacks from happening. For those attacks that then--that then--that then do occur, and from there, we get to phase two: attack mitigation, how do we reduce casualties on scene and minimize the damage. Then following attacks, how do we provide services, how do we provide for the--for the resilience of communities and how do we learn from attacks for future efforts. Next slide. So this is the full picture of The Mass Attack Defense Chain showing the specific activities that go on within each phase. I understand this is a bit of an eye chart, so I’m not going to go through these in detail here. These are how findings are going to be indexed and arranged within the toolkit. Next slide.

So I’m starting with phase one: prevention. I wanted to provide a bit more on the first takeaway on the power to prevent. So if I’m looking across, there’s 600 cases, about half of which were foiled, this is showing what types of reports or initial clues led to plots being foiled. So reports about people directly threatening to carry out mass attacks were a little over a quarter of how we ended up foiling plots. The next category and the largest tips about potential plots were reports from members of the community reporting about behavior or actions or things that others were doing that they--that they had high concerns about, and I’ll show you examples of what that looks like in the next slide. Suspicious actions, things that were directly being picked up on scene were another 3%. Then we turn for the remaining third, results of ongoing extremism and terrorism investigations led to about a quarter of the plots being foiled. And then finally, investigations and investigations of what’s appeared to be routine criminal suspicious activity were responsible for almost 10%. Basically, police and sheriffs out on their everyday business led to just finding things and following up on things led to about 10% of the plots being foiled. Next slide.

So we think about what those tips being reported in are looking like or more specifically, you know, we’ve all heard “see something, say something.” What has the most promising and the most actionable something been? We came up with what we called the--I’m looking through the clues. What we call the Die Out. That’s going to--going to have two parts, motivation, actions are words that reflects a serious, credible intent to attack, combined with preparation, actions being carried out for an attack. So you can see examples of what that looks like here on the motivation side. Things related to being--comments that are directly inspired by past attacks, want to be infamous too, to be inspired to truly fulfill the extremist cause, especially if this comes with having been pick out of extremist group for being too violent for that extremist group’s taste, or comments that have to do with--that someone or someones is so harmful or threatening but that there is no choice but to attack. They need to attack to overcome a threat. Then there are--were a range of things having to do in preparation, trying to recruit for attack, writing down plans to attack, conducting research on how to kill, have all been examples. Active coordination with known extremist, again trying to--trying to be--how to learn how to kill more effectively. Then the actual physical preparations, seeking arsenals, weapons of ammunition when there isn’t an otherwise benign reason for it, like hunting or gun collecting. Travel to get training to join an extremist cause or to get to a specific target as well as site probing and breaching. So all those together in different forms are leading to the warnings that have--that when investigated have led to foiling plots. Next slide.

So it is rare that an initial tip coming in is sufficient on its own to foil a plot, nor is it always the case that many of these are what many of our interviewees directly refer to as the gray zone, where reports of a certain behavior is coming in and it is not clear and it ends up never being clear to what extent someone was actually planning to carry out an attack. But hopefully we’re able to carry out interventions with them and deter them away, and prevent the attacks from occurring. And in fact, with our--from our SMEs, it was made clear that they think the vast majority of cases being handled that are coming through reports are being handled through the so-called gray zone, that they’re not being sent out for arrests.

So to do that, you need a multiagency team that can share information and collectively work to assess and follow up with people who are at risk, and you need a systematic process. So an example of what one look--would look like is shown here, where basically you need to be systematic about vetting incoming reports, carrying out a systemic threat assessment of taking into account both risk increase and mitigating factors, and then determining what kinds of things you’re going to do next to intervene with the person. One of which by the way can be watchful waiting. We just basically see if anything comes up later on. Then another key piece which historically I think tends to go beneath the waves a bit, which is the importance of then following up afterwards at regularly scheduled intervals or as new information comes in that is important and then reassessing what’s going on, and then to intuitively following up to take more necessary actions and then leading to them to either being--just being watchful waiting and phase out, or eventually need to get to more intensive services possibly involving the criminal justice system. Next slide.

So moving now into the third major takeaway onto phase two and to a certain extent phase three on attack mitigations and following up after the attack. I mentioned before on the heavy importance on needing the responding teams, which are going to include multiple organizations, many directly outside of directly responding police, fire, EMS to collectively plan, coordinate, and train. For specific areas within larger response planning, you can see some of our--see the top findings here, specifically on improving security for locations, identify, you can see in italics, the overall ideas. The highest casualties tended to result when it looks like shooters were able to surprise a crowd at close range without ways that they could escape easily. Or relatedly, if they could surprise a crowd that was dispersed a bit throughout sort of a warren of different rooms and hallways and spaces, but the shooter then because of the nature of that sort of labyrinth-like facility had a lot of time collectively to move in and out of spaces and surprise different parts of that crowd.

That is--so the main measures that seemed to really help are associated with reduced casualties, were ways to put distance and physical barriers between would-be shooters and crowds of people. As well as finding ways to let people move away from shooters whether these were things like barriers, having entry vestibules, doors that can be locked quickly. Again, basically the idea of, you know, put--giving additional time and space between would-be attackers and their victims. On bystander and security response side, I think for--certainly for bystanders, the basic strategy of run, hide, and fight, or alternate versions that are very close to that. Certainly, it showed value. Also talk--also and you’ll see this in our final toolkit, more specific cases in which different kinds of interventions are necessary. As one notable example in cases where a shooter was able to surprise a crowd at close range, the most effective measuring or data proved to be members of that group essentially trying to attack the shooter as quickly as possible. Then for law enforcement and medical response, again, the multiorganizational planning and training is very key, and then to support command and control heard repeatedly about the importance of to really implement the Incident Command System and NIMS and be very familiar with that. And then of course to get resource and support from the relevant policymakers and leaders. Next slide.

This phase--phase three and post-attack follow-up was added to the toolkit as a result of our research. We found this phase to be very important but generally not covered nearly as much and certainly not planning defensive exercises, which tended to focus just if anything on the immediate aftermath of the attacks, and essentially you would simulate attackers coming in and how you would response to the attacker. But essentially, once the attacker had been neutralized and the medical treatment of the simulated victims was underway, that’s essentially the point at which the exercises tend to end. So here’s a list of the various types of activities that we discovered are needed in the aftermath of the attack. These include both things for the immediate aftermath of the attack, both from public communications, of crime scene investigations, providing immediate term support, as well as longer term provision of health support community services, and doing longer term recognition and learning. One key thing that we--that we found was people mentioned the importance of doing recognition. Now, not making it not about the shooters, making it about those who acted valorously during the day for--and then recognizing, you know, both victim survivors and the heroes of the day as well as being able to do after-action reports to learn for future events. Next slide.

So for policymakers and executives, three overarching takeaways. The first, again, to reiterate the importance of being providing on some--on some ongoing leadership and support for the multidisciplinary teams that were both responsible for helping prevent attacks as well as responding to the attacks that do occur both immediately and then afterwards. I talked about internal authorities that they can use in order to support planning, budgeting, using regular budgeting cycles. And there are also, both local, state, and federal level, a lot of support and guidance, different kinds of resources, different kinds of teams that can be brought in to assist with training and educating people. We understand and frequently heard from our interviewees that maintaining vigilance and effort is a problem given how rare mass attacks are. I know in other work that Grant’s done, he’s pointed out that we typically, you know, a reasonable measure of the frequency of public mass shootings in the U.S. is on the order of attacks for 100 million residents per year, which kind of tells you how mercifully rare these things are, despite how traumatic they are when they happen.

So, a key solution idea here is to be able to use existing teams dealing with multiple issues whether it would be public safety, to help reduce violence and improve community safety in general, or to improve healthcare in general, and from there as well to leverage emergency response. There were certain areas where it looked like that there would be needs for improvements on procedures and training, where I think that there are, you know, definite opportunities to add new capabilities. These include public education and reporting, just again helping, you know, what is see something and say something, and how do you most effectively report it. Finding pre-attack site surveillance and gun diversion, I know assembling the arsenal that will be used in attacks. We certainly know that these things are being done. But fortunately, they ended up not showing up as much in terms of what the specific initial clues were.

Wellness checks is a key--is a key thing being done by law enforcement or certain times, you know, other responders, people of who are reported in as potentially showing signs of being at risk. A lot on the importance of doing this, not a whole lot that we were able to find on the specifics of how to do it. Then on threat assessment, huge numbers of articles have been done on what different kinds of indicators and risk factors are. Much less on the specific instruments to be used to assess risk, even some of the instruments were more--well, here are several dozen indicators that collectively might imply a person’s at risk. And then of course the improved coordinated response especially for partners in addition to core law enforcement, fire, EMS, and also to start training on the post-attack actions being done. The final thing is to help spread the word on what resources are available to get support for all these things. Federal grants are very commonly known. But there’s also oftentimes a lot of state and local sources and relevant local philanthropies and development groups that can be useful sources of funding as well. Next slide.

So as mentioned, we’re putting all this together. We went through about hundreds of pages of material that we assembled, culling them all into an online educational toolkit. This is based in an earlier toolkit that we did called the Better Policing Toolkit, which reviewed different types of policing strategies that are proven to be promising and keep public safety outcomes, along with tips on how to implement them. Given that collectively prevention and response require such a broad-based actions, we ended up framing this as a toolkit for everyone, TM, with specific content being called out for different groups of practitioners for leaders in the public and the toolkit. We’re arranging them sort of hierarchically starting with, you know, what you need to know now. These are the top takeaways. And then from there we dig into more specific research findings, specific tools and programs that you might find to be useful, case examples, and then for further reading on additional resources and references. Next slide.

So, for conclusions, coming soon, we hope to have the toolkit out in about the January, February timeframe. You can see an example of what the top page of the toolkit will look like now. This is in the early stage of development, so there will be a cool graphic going where that gray box is, but you can get a sense of what it will start to look like. I do ask that you remember the top three, the power for us to all prevent attacks, the importance of relentless follow-up to assess, determine appropriate intervention actions, and follow-up as new information comes in, as well as on a regular basis to see what people need next. And then finally, to do preparation and training both to improve attack response from all involved partners as well as for those actions that need to place--take place after the attack.

With that, this has been a truly massive project, so I really want to thank our sponsors at NIJ, and I really want to thank all the members of the team which included researchers from RAND, RTI International. RTI International, by the way, they did the bulk of the coded case--they did the bulk of the case coding and I want to give a special shout out to them because I know it was a very difficult process of coding 60 fields across hundreds of cases, and how difficult that was especially the attacks on children. Also we had expert support from the Lafayette Group and the [INDISTINCT] Associates, all made a very helpful work. And also I want to thank everyone that we talked to and who served on our advisory panel. As I said, we hope to have this coming out in January or February of next year. Thank you.

DR. RACHEL DAVIES: Thank you, Dr. Hollywood. He’ll have a chance to answer questions along with the other presenters after the next presentation is finished. Please continue submitting questions via the Q&A portal in Webex. Our third and final presenter today is Dr. Karie Gibson. Dr. Gibson has been a Special Agent for the FBI for 15 years and currently serves as a Supervisory Special Agent with the FBI’s Behavioral Analysis Unit 1 or BAU1, where she is responsible for providing behaviorally based operational support and training to federal state, local, tribal, and international law enforcement agencies. Dr. Gibson completes threat assessments, threat management strategies, statement analysis, interview and interrogation strategies, prosecutorial strategies, media strategies, and unknown offender profiles. With that, I’ll turn it over to Dr. Gibson for her presentation.

DR. KARIE GIBSON: Thank you, Rachel. Good afternoon, everyone. I’m very excited to be here today to talk with you about our Active Shooter IV Project. We’re going to get to the specifics of where we are so far in this project. Before I get there, I have to explain a little bit how we got here. We like to build upon our research here and so Active Shooter IV would not be the project it is without the prior research that came before it. So next slide, please. To understand where our data is coming from, I need to start first with Active Shooter: Phase I and really briefly this was a project--a research that we did here that looked at the study of active shooter incidences in the United States between 2000 and 2013. There’s a total of 160 events that were looked at. And really wanting to understand what happened during and after the shooting. How fast did police get there? Did the shooter commit suicide or were they involved in a--in a shootout with police? All things that could be helpful to our law enforcement partners. Next slide.

From that, we built upon the Active Shooter I research and went into Active Shooter: Phase II. And what Phase II was is we looked at those 160 cases that were used in Phase I and we really needed to look for cases that have substantial law enforcement files. And so we were able to come up with 63 cases that had significant law enforcement files in order to really look at the active shooters and their behavior before the attack. So for us, it was very important to be able to have law enforcement data to rely on versus open source to really kind of inform that offender’s behavior. And so we were able to do that in our Phase II research, it was through Andre Simons, Sarah Craun and Jim Silver were the main researchers on that project. And we were able to really find out more about the offender’s behavior before the attack. How long are they on the pathway? Research and planning, for example, how long are they in preparation? Specific things like that. And then we went onto the next phase of the research. Can you go to the next slide?

Active Shooter: Phase III. So for this, again, we’re really looking at all of our research and building upon with the real theme of prevention, right? What can we learn from these offenders and how can we inform our prevention efforts? And so we took those 63 cases that we were able to look at with the law enforcement files from Active Shooter: Phase II, and then we used them in our Active Shooter: Phase III where we compared them with 63 cases of person of concern, cases that had come through our behavioral threat assessment center where we actively did threat assessment and threat management. And we looked at the differences between these two groups, right? Trying to figure out really what would be helpful for threat assessment teams to know. Some key aspects that came out of that research was that you were 16 times more likely to be an active shooter if you were displaying concerning behaviors and your bystanders did nothing. So that inaction really proved to be very relevant in the attacks going forward. Another key aspect was humiliation. We found a difference between the groups as the ones that went on to commit acts of targeted violence had experienced a humiliating event within two years prior to the attack. So, very key findings that we wanted to incorporate into our next stage. Next slide, please.

So that brings us to our Active Shooter: Phase IV research that we’re very excited to be doing and being able to do that with the help of NIJ. For us, again, we’re trying to look at this from the preventative aspect, right? What can we learn and build upon with our research. And so now Phase IV, what can I say, we like numbers here at BAU. Our Phase IV research is going to focus on the active shooters that are still alive and in prison and going to be able to interview them. And so when we did our initial searches related to these active shooters that are still alive and their cases are adjudicated, we had 72 that we had found that we could go to interview. Of that 72, 13 were from the original Active Shooter II research that we--that I already discussed. We’ve made--started making contacts and trying to kind of figure out, you know, the next stages with the interview, and ultimately we know that some of these initial 72 are deceased, some said no, so we’re at 67 potential interviews to complete at this time that expand over 60 prison locations.

Another aspect of this research that we were able to do is we expanded the interview protocol to include those new findings that we found in Active Shooter III research. We wanted to really get at that humiliating event, get at the bystander inaction to really learn more about that. And so our hope is as we go forward to be able to interview these offenders, we’re going to have a better understanding of not only their thought processes and motivations, but those key aspects along the pathway, the planning and preparation, and their tactics that were used. Next slide.

As we’ve gone through this, we’ve started this process, as I said, we started going out to be able to figure out where we’re going to be able to do the interviews with those 60 prison locations. However, COVID hit and, you know, as everywhere, everything is slow. So in that--in that timeframe, when we’re waiting for DOCs over the United States to open up, we started coding cases that we had here with that new protocol, and these were some of the findings that we found initially with that initial review. And so we were able to determine that the majority had showed a desire to obtain weapons six months or more before the incident. So this is, again, very consistent with other research that we have seen where, again, there is time to disrupt these cases. We also found that the majority felt rejected and disenfranchised within the year leading up to the attack. That’s consistent with other research that we have completed with the Active Shooter II research, it showed within the year leading up to the attack, there was concerning behaviors and stressors that were noted. And so I think that as we go forward and we do these interviews, we’ll be able to expand even more on these initial findings. We also found that the shortest amount of time between the grievance and the attack was three to five months with the most being 25 or more months. Again, consistent with that we have time to disrupt these attackers.

The most common concerning behavior that was reported was the offender’s mental health issues, so we continue to see that emerge and looking forward to finding out more specifics when we do the interviews of the offenders. And then the most common concerning behavior that was observable but did not raise concern was the offender’s behavioral leakage. We’re really excited to learn more about that because with the findings that we had found in Active Shooter III, the bystander inaction and so for us, we’re interested to learn more about this through the interview because it seems to be having some of those similar implications where, again, we have concerning behaviors being seen, but the bystanders not recognizing that or not reporting it. Next slide.

So as we go forward, now that things are kind of getting back to normal here, we definitely are continuing that coordination with the different prisons, the 60 different locations, to be able to get in there and start completing these interviews. Interviews to begin around, you know, here at the beginning of 2022, we’re excited for that. And as more cases get adjudicated, we will continue to add them to our number of cases that we have and so we’re, again, looking to continue to grow upon this research and see how it builds. We will continue to code all the cases that we do have access to right now, those case files with that revised protocol. And then we also have a full-time researcher assigned to this project. So we’re, again, this is in the very early stages for us but very excited to have this go forward and continue to learn more from these attackers to be able to prevent these attacks from happening in the future. With that, I’ll turn it back over to Rachel. Thank you.

DR. RACHEL DAVIES: Thank you, Dr. Gibson. Dr. Gibson and the other presenters will have a chance to answer audience questions shortly. But before that, we’re honored to host a discussion with the presenters led by Dr. Lin Huff-Corzine. Dr. Huff-Corzine is a Professor in the Department of Sociology at the University of Central Florida. Her research on lethal and nonlethal violence spans three decades with work on homicide, examining topics including but not limited to lynching, domestic violence, regional variations, transportation effects, human trafficking, and more recently mass murder, mass shootings, and COVID-19 effects. Dr. Huff-Corzine’s publications typically result from working closely with colleagues and can be found in edited collections as well as journals such as Homicide Studies, Justice Quarterly, Violence and Victims, Social Courses, Victims and Offenders, Criminology, Deviant Behavior, and Criminology and Public Policy. I’ll now turn it over to Dr. Huff-Corzine for the discussion and I’ll take back the panel in about 15 minutes for audience Q&A.

DR. LIN HUFF-CORZINE: Thank you, Rachel. I appreciate the introduction and we are faced here with two, three wonderful papers, each dealing with mass violence in some form. And so I want to take each one individually and I’ll take them in the order that they were presented and ask a few questions from each. So I--one of the things that I’m concerned about continuously is how do we decide that--and this would be for everyone. How do we decide that a mass shooting, mass murder is based on four deaths in a public situation? So what is public? As another part of my question here, so dealing with the numbers, is that--is that as useful as it--as it once was? Do we--do we need to really understand why we’re asking for four or more deaths before we consider it a mass? Anybody?

DR. GRANT DUWE: I can go ahead and address that one. I mean, we--there is a point at which a line has to be drawn, right? So you could--you could potentially use a victim count of two or more. And some definitions of mass shootings don’t even require victims to be killed. They look at cases involving injured victims. And that’s not to say that there’s anything inherently wrong with those definitions. And I think that’s part of the challenge here is that--is that I don’t--I don’t think--I don’t think there’s any definition that’s inherently more correct than another. It’s just it will provide people with a different look at whichever phenomenon we’re trying to examine. In the case of our study, we’ve elected to focus on mass public shootings mainly because the incidents at least in terms of how we’ve defined it, those generally capture the events that attracts a lot of attention, that galvanize the public, that trigger all of the policy debates because generally speaking, it’s not cases involving four wounded victims. Or it’s generally not going to be a case in which a person kills their family members, four more--five or more. Those typically don’t trigger discussions about domestic violence. Instead, it’s--the mass public shootings as we’ve defined them, these are--these are the cases which are rare within the context of mass shootings or mass murder. But again, these are the ones that drive the discussion. And I would argue, also, that when you--you know, one of the things that, you know, has been demonstrated at least over the last couple of decades is that the number of victims is the strongest predictor of news coverage. And news coverage has been the dominant source of data. So if you’re interested in more accurately measuring the phenomenon that you’re studying, then you’re going to want to focus on victims who were killed and you’re going to use a higher victim threshold. So--and that’s not to say that using, like, three or more, that there’s anything wrong with that, it’s just you’re probably going to miss more data for those cases involving three victims. And another thing that our research has shown is that the cases involving three victims are going to be more numerous, too.

DR. LIN HUFF-CORZINE: Yes, they absolutely are. So we have a good stance for why we start with four and why we look at public offenses. Do we have the same sort of data available from other countries, is it more limited, I would guess, in many countries so that we aren’t as sure about the data in other countries? So that would be another thing that would be of concern is how can we compare with other countries since we are not as--we aren’t as sure about the data coming from there?

DR. GRANT DUWE: So Lin, do you want me to take that one?

DR. LIN HUFF-CORZINE: Briefly. Uh-hmm.

DR. GRANT DUWE: Okay. Briefly. Well, so you’re absolutely correct. And I think like the missingness problem that we identified within the datasets that--from the other researchers that we looked at, and that was just for the U.S. cases. And so eventually, like, we saw that, you know, the missingness was between, like, 30--34 to 37% whereas with our estimate, it was like close to 67% for the non-U.S. cases. And so to us, that seems about right but it--you know, what that also indicates is there are a lot of incidents that are likely taking place that just don’t--you know, they may get reported locally, but we may not necessarily know about it here in the U.S.

DR. LIN HUFF-CORZINE: Absolutely. Okay. Let me turn to the paper on--that John Hollywood presented. You have quite a lot of ideas for a package here to help reduce the amount of mass violence, variety of types. One question I have is, have you considered the architecture and problems that the types of architecture may provide for, I guess, either in foiled attempts or in other types of mass violence? Let me give you an example. The classrooms here at UCS and in some of our buildings will have two doors. That’s one thing that they often suggest. But both of those doors are at the back and the professor is at the front. So there’s no escape out the other direction. And those doors lock from the outside, not the inside.

DR. JOHN HOLLYWOOD: Yeah.

DR. LIN HUFF-CORZINE: Is that something that might be added to this type of, you know--you got a lot of different things added here.

DR. JOHN HOLLYWOOD: Right, right.

DR. LIN HUFF-CORZINE: And just architecture was one of those that I’ve noticed that I even had a student say, “Well, somebody told me I could lock the door from the inside by taking a belt and wrapping it around the [INDISTINCT].

DR. JOHN HOLLYWOOD: Yeah.

DR. LIN HUFF-CORZINE: Yeah. And that’s not locking a door. So is that something you might have in your toolkit?

DR. JOHN HOLLYWOOD: Yes, actually the phase two page in attack mitigation in the section on securing venues actually has some dropdowns a little bit on issues just like that. And one of the things you referenced on the locks for example, is that, you know, we reference earlier work by Alert at Texas Tech that was noting that this--their data said they don’t have any examples of a shooter breaching a locked door, at least not in the U.S. So that’s one thing we talked about, kind of these examples of what some of the specific layouts look like, you know, I kind of mentioned that there’s--this one’s kind of a constrained box, we’re able to [INDISTINCT] all at once and they can’t really get out. There are ones that sort of like--we call it a labyrinth or a maze of rooms which schools tend to fall on that category. And to the extent that if people can be trapped in those rooms specifically, you know, if they can be surprised quickly and then they can’t secure the rooms, they can’t escape the rooms, then you’re in a much worse situation.

DR. LIN HUFF-CORZINE: So, there’s some real serious policy implications in that--in that sense for a lot of places. And another look at this. In 2012, we had a meeting at--let’s see about what we might do about shooting since. One of the things was report, report. And I see reporting in yours as a major issue as well. Who do they report to? How can--how can we make sure that these are listened to? We didn’t come up with a good way to implement that. And I hope that, you know, there’s some way that you’ve come up with that the reports are heard and responded to.

DR. JOHN HOLLYWOOD: Yeah, we talk about this a--we talk about this somewhat. I mean, objective-wise, we picked up on the importance of making it as easy, accessible, and basically safe as possible to report. And also, and that means having multiple channels. So if it’s a student who wants to tell a teacher, they can. If it’s someone in the community who wants to tell, you know, a community leader, they can, you know, that there are--that there are all kinds of different channels to include, you know, when--you know, outside of the police. The goal then is to have those then basically, you know, referred up to whatever the local team, and different [INDISCTINCT] interviews, we’ll talk about that, there’ll be a local relevant team within that. I think it was for the whole jurisdiction, sometimes it was, specifically say, for schools, you know, either local or state school level. Sometimes it would be like a consortium of like building operators, and they’d have their own security staff would all confer. So what it was, was different, but the point was that you had a multidisciplinary team that was in coordination with different relevant authorities, and that was systematically meeting and going through these things.

DR. LIN HUFF-CORZINE: Okay. Yeah, I think there’s a lot of parts to this. And of course, how that gets implemented, in part depends on what Dr. Duwe was talking about in terms of prevalence. And so how many places we have to report and so on can certainly be influenced by the prevalence as well. So let me turn to Dr. Gibson on the Active Shooter Project. What--how do you plan to go in and interview--stick with Phase IV, how do you plan to go in to interview these folks? What--do you have a theoretical base that you plan to use? Or what do you plan to do?

DR. KARIE GIBSON: So we have an interview protocol that we’ve developed, that we’ve obviously expanded based off of the different findings that we have through the different phases of research. And so we plan to go in, have an interview team of two, people from the FBI’s Behavioral Analysis Unit who have, you know, done, you know, significant interviews of offenders before, they will partner and go in and complete the interview protocol. It’s a structured interview, you know, so they’ll go through and they’ll complete that structured protocol. And then we’ll obviously also code their files as well, as again, just the way to get as much information as we can to feed into this project. It will help as we go forward to understand their behavior.

DR. LIN HUFF-CORZINE: Okay. There’s work out there that as a--I think it has some good implications for this and that--that’s in terms of interviewing these kinds of people. And that’s the work of Lonnie Athens. Are you familiar with that piece? It really comes off of symbolic interactionism. And there’s a--there’s a part about generalized other in that work, and how those other people influence the violence that this person has exhibited and is in prison for. And what happens is, theoretically, they tend to think that the generalized other people are all going to tell them to be wonderful and tell them good things. It’s the same problem we have with Hirschi and some of his work on just kind of thinking that other people are going to tell them to do good things, not bad things. But that’s not the case for many people. And the people that are in the prison may have a different set of people that they have responded to and have grown up with. And so it would be, you know, in my view, would be wise to kind of get some background about where did they get these ideas? Where did all that develop from? And I don’t know if you have anything on that or not, but it would help with the understanding of the development of violence. That was the number one…

DR. KARIE GIBSON: Yes. No. The--that’s obvious--that’s great stuff, yeah. For us, we’re very interested in that as well. I mean, we see it, you know, anecdotally with the cases from a threat assessment management perspective, and the ones that go on to commit the attacks that we complete post attacks on where there is a shift, there’s a change, right, and really trying to pinpoint that so we can learn more about it for cases going forward. So yeah, we will be asking questions kind of about that evolution for them. And I’ll check out that research that you brought up as well, ma’am. Thank you.

DR. LIN HUFF-CORZINE: I’ll be glad to send it to you, Karie. No problem.

DR. KARIE GIBSON: Thank you.

DR. LIN HUFF-CORZINE: And how do you--how do you plan to compare with the general population? Or do you? You know, a lot of people have losses, a lot of people have disappointments, a lot of people are humiliated. If I’m not humiliated in a two-year period, that’s unusual. So, you know, how did we--how does this group compare with a general population would be one of my other questions that I would have so…

DR. KARIE GIBSON: Yeah, so that’s probably going to be Active Shooter Phase VI, we already have Phase V, we already have that committed. But no.

DR. LIN HUFF-CORZINE: Fair enough.

DR. KARIE GIBSON: You’re absolutely right. Like that--that’s why Active Shooter III was so interesting, because we were able to compare, you know, ones that committed attacks to ones who did not. And that’s really where you can really start to peel back those layers of the onion and really understand from a prevention standpoint, what can we do, because on the surface they look very similar, those two samples.

DR. LIN HUFF-CORZINE: Uh-hmm. Uh-hmm. Okay. I think my time is up. But I want to thank everybody for responding to the questions I was able to find by looking through your work, and I commend and enjoyed each of the presentations and look forward to those coming out and the publications that you will be getting out with. So thank you for your time. And I’ll turn it back to Rachel.

DR. RACHEL DAVIES: Thank you very much, Dr. Huff-Corzine, for that discussion. We will now have the opportunity to field questions from the audience. Our first question is for Dr. Hollywood, and this is from Cody. Cody asks, “Does the mass attack defense toolkit include considerations for private companies that provide safety and security training as part of getting the public trained?”

DR. JOHN HOLLYWOOD: Absolutely. Certainly the phase two page on attack mitigation, the sections on securing venues as well as some of the things on bystander and security response, that security response specifically, I think will be directly relevant for companies. And also as mentioned as well, many cases, there is a need for companies especially that have, you know, very large public exposure, they’re part of the partner process as well, because the security measures and security staff that are in place, you know, if an attack starts say at a mall or a theater or something like that, they’re going to be--it’s going to be part of the defenses first on scene.

DR. RACHEL DAVIES: Great. Thank you. Our next question is for Dr. Gibson. Katherine asks, “Can you expand to provide some examples of the bystander apathy that you identified?”

DR. KARIE GIBSON: Yeah, absolutely. So if you think about like somebody who’s displaying signs of depression or isolation and the bystanders around them don’t notice that or don’t inquire or don’t seek out any type of mental health care for them. Or change in their hobbies, they once, you know, were into different, you know, more suitable hobbies, and now they’ve changed into playing video games or shooting at the gun range, or shooting, you know, guns or being preoccupied with firearms, and really no questioning to that or talking about that, and trying to unpack it. And then also, these more dark themes of suicidality and homicidality and just kind of, you know, normalizing that to say, well, that’s just so and so, he’s just got a dark sense of humor versus really kind of asking and questioning, there’s been a change, let’s talk about it. Let’s--what’s going on? Can we get you some help? And so, those are--those are a few examples that we’ve seen.

DR. RACHEL DAVIES: Thank you. Our next question is for Dr. Duwe. Sylvester asks, “Can you tell us more of where and how did you approach your methods of obtaining data?”

DR. GRANT DUWE: Right. So what we did briefly is we used the FBI supplementary homicide reports data, which really the useful version of that goes back to 1976. And so, they’re--because it’s a voluntary program involving law enforcement agencies nationwide, there is some missing data. But it does provide a fairly comprehensive record of all homicides that take place in the U.S., and we use that to initially identify when and where mass murders have taken place. And then, after doing so, we then collected more detailed information on those incidents through primarily news reports. And then using that data to identify which cases actually met our criteria for mass public shooting and along with my other two co-PIs on this project, we actually discussed each case to determine whether--we came to a consensus as to whether all the criteria were met. But in using those multiple data sources, as opposed to just relying on news coverage, we believe it has helped us achieve a more accurate, complete dataset for mass public shootings.

DR. RACHEL DAVIES: Great. And as a follow up to Dr. Duwe, “Did you include the subject in your victim tallies if the subject died or killed themselves?”

DR. GRANT DUWE: No.

DR. RACHEL DAVIES: Okay. Next question. This one is for Dr. Hollywood. This was posted in the chat. “Data you provided indicates 20% of the foiled attacks involved direct threats. Do you know if these were reported by the potential victim? And can you speak to the proportional contributions of anonymous systems versus attributed reporting methods?”

DR. JOHN HOLLYWOOD: So for the latter part first, I can’t really speak to the prevalence of what was done through anonymous reporting versus attributed reporting, which is, you know, our data just largely tracks the basic type of clue and then the sort of the basic source. With regards to the direct threats, again, that included a combination of both people who were being directly threatened and passed on the threat. It also included a lot of threats that were made on social media that struck observers as being very credible, especially if they fell in the category of pointing out the gun and saying, you know, here’s the gun that I’m going to use to go carry out this attack against these people, you know, against my enemies, basically. So it was a combination of both of those.

DR. RACHEL DAVIES: Thank you. Our next question is for Dr. Gibson. “You mentioned exploring offenders’ mental health. Can you talk more about how you define mental health? Do you relate it to specific diagnoses and connections with the mental health system?”

DR. KARIE GIBSON: Sure. So in our studies, whenever we’ve talked about that, we’ve looked at it both ways. So from--we know from Active Shooter II that out of the subjects, only 25% had a diagnosable mental illness. But what we find is obviously, none of them are mentally well, and so really, we like to look at it from more of a prevention standpoint of mental wellness. And really understanding what does that mean? It doesn’t come down necessarily to a diagnosis. And I think sometimes people can get fixated on that. And really, it’s how is that person coping, it’s more of a global mental wellness. And that’s why we see for stressors for people to go on to commit active shooter events is that they have mental health stressors, as well as they show concerning mental health behaviors. And so really, from our standpoint, we look at it from that mental wellness standpoint.

DR. RACHEL DAVIES: Thank you. This next question was directed to the panel generally, and asks, “Is it fair to say the years from 2017 to 2019, there was an uptick in mass public shootings? And does your research look at or identify evidence for the reasons for that uptick?”

DR. KARIE GIBSON: I can tell you from our standpoint, we do see an increase that’s happened. As to the why, I think that’s still, you know, up for discussion and trying to kind of understand. But I can tell you, though, with each year, let’s say--let’s say there’s, you know, 30 active shooter events, if you’re going to be an active shooter, you have to be that much more egregious to get the attention now, because there’s so many happening out there. And so you start to think about how am I going to be the one that every--everybody knows or remembers. And that means usually, that attack is that much more over the top. And so I think that that’s something for all of us to understand is as we have more of these, as they increase, they’re going to get more and more egregious, and we’re trying to understand the why behind that. But again, there’s--it’s a lot of competition out there, and there is the potential that there--they--there is an acceleration that happens from these attackers seeing this and seeing the notoriety that each attacker gets once they complete an attack.

DR. RACHEL DAVIES: All right. With that, I think we’re approaching the end of our time for Q&A. So I want to go ahead and wrap up. Thank all of our presenters and our discussant again for their contributions to this event, and to the audience for their participation. And I’ll turn it back over to Daryl.

DARYL FOX: So I want to, again, thank everybody on behalf of the National Institute of Justice and our panelists, we want to thank you for joining today. This will end today’s presentation.