Opening the Black Box of NIBIN

DR. GREG RIDGEWAY: Good morning, everyone. Thanks to all of you for coming to our Research for the Real World. I’m Greg Ridgeway, Acting Director of the National Institute of Justice, and we’re thrilled to have this seminar today. And I want to first tell you a little bit about what we’ve been doing on the bigger issue of forensics. As many of you know, in 2009 there was a big National Academy of Science report on the forensic sciences. It was sort of an eye opener for many, questioning some of the underlying forensic science infrastructure and techniques in this country. But since then I think NIJ has really stood up to the challenge and brought research to bear on a lot of the questions. We formed the Office of Investigative and Forensic Science. I see many of our staff members here — our engineers, our physical scientists, our chemists, our toxicologists — who are trying to build better techniques and demonstrate reliability or unreliability of forensic techniques.

We had a fascinating study recently looking at ballistic evidence. We had — we got a research team to get 10 consecutively manufactured barrels coming right off a Glock manufacturing line. They were manufactured to precision. They were very similar, and then this research team fired 150 rounds through each of those guns and jumbled them up, distributed them out to firearm examiners around the country, and asked them to put that jigsaw puzzle back together — which bullet came from which gun. And surprisingly they were able to do it. The ballistics techniques, the pattern matching, the whole system for matching bullets and casings back to the original gun — the examiners are very good at it.

So it’s this kind of study that we’re trying to do to get to the fundamental science of the reliability behind many forensic science techniques. Probably by the end of this summer we’ll have invested $120 million in forensic science R&D over the last five years to answer this, as well as DNA questions, pattern impression evidence, and lots and lots of other issues.

But that science is only part of it. We actually need to get these techniques into the field and operational, and that’s where partnering with an organization — our sister agency, ATF — on how to make some of these techniques a reality, to make them operational, how it works in the field to get evidence into the hands of people who need it when they need it. And we’re thrilled to have Dr. Bill King from Sam Houston University and Special Agent John Risenhoover to participate in this seminar and share about their partnership and what they’ve learned.

Let me introduce both of our speakers. Dr. Bill King, he’s an Associate Professor of Criminal Justice and Associate Dean of Research and Program Development in the College of Criminal Justice at Sam Houston State University in Texas. His research focuses on the structure of police organizations and forensic crime labs and the role of forensic analysis in criminal investigations. And between 2005 and 2009, he helped implement reforms to the Forensic Crime Lab, National Police Service, and Homicide Bureau in Trinidad and Tobago.

Special Agent John Risenhoover has served in the position of NIBIN National Coordinator since August 2012, and in this role he is responsible for the development and implementation of the program, which includes training, strategy, management of over 150 partner sites throughout the United States.

And I understand we have many of our ATF friends here in the audience as well. Is that correct? Why don’t you stand up so we know who you are. We’re glad we have so many of you. Thanks for coming to our seminar today and for all of the good work that you do to improve our criminal justice system in the country. And we’re grateful to be research partners with you, and I know this is just the first research project in a long time, and we have many more to come, and I’ve already started to line up our next research projects with you. So there, be more seminars like this and more opportunities for you to come over to NIJ and work with us.

Without further ado I have a — oh sorry, I have a few housekeeping instructions that I’m told that I need to read to you first. So this seminar is being audio recorded. The slides and audio will be integrated and available within a few weeks. If you signed in over there with Ted, then you will receive an email when the seminar becomes available online, and we find that many people like to forward the slides and the seminar and the audio to their friends and post to their Facebook and Twitter and share, and we encourage you to do that. Share this widely to whoever you think might be interested in it.

At the end of the presentation, there will be a Q&A. If you have a question, please approach the microphones. We have them in the aisle and at the ends. Please approach those microphones; state your name and affiliation before you ask your question. Because this seminar is being audio recorded and published, one of our staff will ask you to sign a permission form allowing us to use your question in the published presentation. If you do not want your question posted online and prefer to not be part of our audio presentation, just let that staff person know, and then when we’re preparing the audio and the presentation we can just edit that out.

Without further ado, let me pass this off to Dr. Bill King.

DR. BILL KING: Thank you, Stan. Thanks for showing up. I had a great team of researchers and want to do some thanks to NIJ and the ATF for being great research partners. We did site visits at 10 crime labs and law enforcement agencies. They opened their doors to us. I’m going to name the names of those agencies here. I want to make it clear we’re really grateful to them for sharing a lot of information with us.

I assume you are like my students, and you’re going to read the slides and I’m going to do the voiceovers and I’m not going to read my slides to you. You don’t want to look at me anyway. So just read along here.

Quick overview: I’ll briefly explain what ballistics imaging is. I’ll describe briefly NIBIN, tell you about the research methodology on a research project, and I’m going to present the findings to you in four parts, nature of inputs, of hits, and policy recommendations for tactical and strategic uses. In some ways this presentation is a little bit historical, because the data I’m describing describe NIBIN circa 2009, 2010. And in the last year of the project, when we were wrapping up our final report in October of 2013, ATF was already talking a lot to us and they were implementing reforms. So the state of NIBIN today is considerably different than what you’re going to see here. Just bear that in mind. A grant generously supported by the National Institute of Justice.

Number two, if you don’t like something I said, it’s my fault, unless you have a lawyer, in which case it’s my fault and my coauthors’ fault. But I do not speak on behalf of NIJ or ATF or anybody else who has deep pockets. Thank you.

Briefly, what is ballistics? When a firearm is fired, the weapon and parts are unique; we call them individual tool marks upon cartridge cases and bullets. The most pertinent element of the bullet or cartridge is the cartridge case for NIBIN, as I’ll demonstrate in a minute. Starting at the bottom, we have a firing pin impression. This is unique from one firearm to the next firearm, including consecutively manufactured firearms. So we have firing pin impressions. Those are good. In the middle, we have breech face impressions. The cartridge case, which I’m going to refer to as brass for the rest of the presentation, breech face on the back of the cartridge case unique as well. And then usually ejector marks. I wanted to throw in a picture of an extractor mark just to keep it interesting for you.

In the olden days, firearms examiners would pick up the pieces of brass from crime scenes, and then they would have to visually compare one piece of brass to another piece of brass using a comparison microscope. Tedious, very time consuming, didn’t produce a great number of hits. Okay? Beginning with computerized systems, ballistics imaging, the computer takes a picture of the image, turns that image into a digital signature, loads it into a database, and then, as you load more and more brass — images of brass into the system, it compares each new input against all of the inputs in the database. Unlike on television where the computer tells you what the match is, that doesn’t happen with IBIS or with NIBIN. The computer produces a list of correlations of likely candidates ranked from most likely down to least likely, and then a firearms examiner or firearms technician has to examine the images on a computer screen. They look at them, they pick out the ones that they think are most likely, high-confidence candidates, and then they have to — that’s not technically a confirmed hit yet, so then the examiner has to take each piece of brass that they believe is a likely candidate, examine them under a comparison microscope, and then they designate that there is a hit. And so hits are the useful outputs for the system, because we’re matching ballistics evidence from one crime scene to another crime scene, or from a crime scene to a gun that’s been confiscated off of someone who shouldn’t have a gun, such as a felon in possession.

What we know from research by Braga and Pierce is that using these automated ballistics imaging systems like IBIS and NIBIN increases the productivity of firearm sections greatly. Okay? You can crank out hits with these things. The computer does a lot of the screening work, and examiners spend their time doing the minutiae of comparing a limited number of pieces of evidence under a comparison microscope. So this is a good technology.

My presentation is about how this technology gets implemented in individual crime labs and police agencies. So it’s a very organizational perspective on how forensic evidence is implemented. It’s also a presentation on how investigators use or don’t use the outputs of these systems and also how ATF oversees this rather loosely coupled system of technology.

What the heck is NIBIN? NIBIN is a program that is overseen by ATF. It’s a bunch of networked IBIS terminals. So now, not only can you compare the inputs from your local IBIS terminal, you can compare them regionally to different regions of the country. So Houston Police Department, Texas, can also compare their inputs against Harris County and Dallas and other agencies in the region. So you’re opening up the database; you’re making it wider in scope.

ATF has run NIBIN by itself since 2003. The number of NIBIN sites — let me back up a little bit. What happens is, ATF buys the hardware and the software and then they distribute it to the local police departments or crime labs around the country. Now if a local police agency or crime lab wants to become a NIBIN site, they can self-fund: Buy your own hardware, and you pay for your service contract, and then you can plug into NIBIN. The New York City Police Department does this. They’re a self-funding NIBIN partner site. But most sites are, their hardware and their contract, their maintenance contract, is provided by ATF. So ATF is distributing this technology out, and then ATF also runs the servers and upgrades the network and basically runs the system. But this is loosely coupled, because ATF can’t march into a crime lab and force examiners or technicians to put more evidence in or to confirm hits more quickly. And so there’s a loose system of accountability that, just, it’s based on the political nature of having a federal agency running a program that is ultimately staffed and run by people who are locals. Okay? And you’re going to see what that loose coupling results in, in part as we dive deeper into the exciting part of the presentation. This isn’t the exciting part.

There used to be a fair number of NIBIN sites; 236 is as many as I could count. A lot of these sites were culled in 2010 and 2011. NIBIN had budget issues. A bunch of sites, low-performing sites, were closed down. I’m going to show you data on those low-performing sites and high-performing sites. So bear that in mind, that a lot of the low performers have been culled out of NIBIN circa 2014.

NIBIN’s produced a lot of hits — more than 50,000 hits. Okay? That’s a lot. For your edification, a hit merely links a firearm. And so, unlike AFIS and CODIS, it doesn’t give us the individual characteristics of a person — my DNA, my fingerprint — it’s a gun. And even if you catch me with a gun — hypothetically; I’m not a criminal, okay? I mean I’m really not a criminal, but — hypothetically, I’ll tell you that I found a gun on my front lawn last night. I will lie to you about whether or not I possessed a gun. So there’s a little slippage between prosecuting individuals for firearms that we know are linked back to crimes. So this is a little more challenging, I think, than using AFIS and CODIS. Just bear that in mind, too, when we get into investigator interviews.

Two great studies of NIBIN: the Office of the Inspector General report from 2005 and the National Academy of Sciences report from 2008. I’m an egghead. I get paid to do research when I’m lucky, and this is my research project. Five sources of data; first, my coauthors and I did 10 site visits at different NIBIN sites. This is a non-random sample. We selected sites because five of them had to be close enough for one of my coauthors to drive there to control our cost. So they had to take their car, and then they got to fly to another one. Number two, we wanted to geographically represent the U.S. from east coast to west coast. We did okay with that. Three, we wanted to pick regional labs, because we were told that their inputs are different. When you’re gathering a bunch of inputs from local agencies, you don’t really control and have a clear accountability chain of command, and we wanted local agencies such as Houston and Austin, Santa Ana, where the majority of their inputs come from cops who work in the same agencies where the crime lab is located.

So my two regional labs are in Bowling Green, Ohio — this is a BCI NI lab. It’s run by the state attorney general in Ohio, and it is no longer a NIBIN site. They got culled in 2011, I believe. But a good lab system run by the state attorney general, well-funded, very professional lab. They produced three hits, which is why their terminal was pulled out.

The other regional lab is Onondaga County, New York, which is Syracuse. Okay? Except for Bowling Green, Ohio, these are generally pretty high-performing NIBIN sites. They all survived the cull of 2011, and they’re pretty high performers, and so when I present their data, you’ll see kind of what these high performers look like.

My coauthors and I surveyed every publicly funded crime lab in the U.S. we could find. We also surveyed every NIBIN site, past or present, we could find. I’ll show you a little bit of that survey data. We received a bunch of data from ATF. We got very detailed hit files for 19 different NIBIN sites. These give us the nature of when each crime happened in a hit dyad, when the hit was confirmed, type of evidence, type of crime. We also got monthly updates on how many hits, how many inputs, site by site. I’ll show you that. Don’t worry; it’s in a bar chart, so it’ll be quick to understand. And then we went on and we interviewed investigators who investigated a crime with a NIBIN hit at a bunch of these sites.

Okay? Don’t worry, this is all coming back, because here we go: overview of the findings, just four parts here. Inputs and hits for NIBIN sites as a whole; a new performance metric my coauthors and I cooked up on using a lab’s time as an indicator of performance for the program; and if you’re in federal law enforcement — I took a look at the roster, I think there are a bunch of other people who, ATF and other agencies — think about the programs that you oversee, and then think about how you measure their performance, because there is a lesson in here for you about how ATF used to measure the performance of individual sites and how ATF now does it. And, just a hint: Elapsed time matters. How helpful are NIBIN hits to criminal investigations and then using a NIBIN as a source for strategic intelligence.

First thing, just broadly in terms of what gets put into NIBIN by these local labs and agencies. Inputs can be either cartridge cases, brass, and they can be bullets, and then hits come out as either brass or bullets. A majority of the stuff going into NIBIN and the hits coming out — it’s almost all brass; it’s rarely bullets. Bullets are really hard to image, and so people tend to not do it, because it’s time-consuming, and then because the database isn’t populated with a lot of bullets, you don’t get a lot of hits out of it. So it’s not self-reinforcing. It’s almost a self-deterring system. There’s a new system for imaging bullets that FTI has put out that they think is going to improve this, but just so you know, brass is the modal input into NIBIN. Counting inputs and counting hits were the performance indicators ATF used up until about 2011 to measure whether you were a good NIBIN site or not. And so as long as you were putting in a minimal amount designated by ATF, you were probably okay, and as long as you maybe had some hits — but you’ll see some sites never produced a hit — you were okay with ATF. And that went away in 2011. ATF has raised the bar for performance at NIBIN sites.

What we find when we look at these data is there’s a lot of variation just in inputs. First thing, most of the inputs are brass; they’re not bullets. The modal — I’m sorry, the median input for a site was 4,700 pieces of brass and 463 bullet inputs. Most of the inputs are coming from test fires, and a test fire is when a criminal is caught with a firearm that they’re not supposed to have — I call them criminal firearms, and I know like attorneys go crazy when they hear me say that; sorry — these criminal firearms then go to a lab, and they get test fired, and the brass gets loaded into NIBIN. So the majority of these inputs, they’re test fires; they’re not brass getting picked up at crime scenes.

And then looking out across these 236 NIBIN sites, there’s a lot of variation. Three sites barely turned their machine on. They put in less than 100 pieces of brass. It’s really — that’s almost nothing. The lowest quartile is about 2,300 pieces of brass. In terms of bullets, lowest quartile is about 99 bullets, because people don’t want to image bullets. And then on the other end, on the right-hand tail of that distribution, you have a lot of really high-performing sites; 22 percent of sites had more than 10,000 inputs.

Inputs are a process indicator. Okay? I mean, the analogy I’ve used when I talk to ATF about NIBIN is if you’re of a certain age, you remember as a child a game called Hungry Hungry Hippos? It was the plastic hippo, and it ate a lot of marbles. NIBIN’s like a Hungry Hungry Hippo. You have to feed the doggone thing a bunch of brass and a bunch of bullets, or you’re not going to get inputs — I mean, you’re not going to get hits. So, the inputs are important. But what’s really important are the outputs of our NIBIN hippo, and that’s — those are hits. And so as we look across these 236 NIBIN sites, a bunch of which are no longer in the program, we see, again, tremendous variations. A majority of them never produced a bullet hit. Again, they don’t image a lot of bullets, we’re not going to expect a lot of bullets out. The most productive bullet site had 61 bullet hits. That’s it. Thirteen percent of the NIBIN sites never produced a brass hit. There are a bunch of sites that never produced a hit, period. The lowest quartile, nine or fewer brass hits. About half of the sites had, 50 percent was around 76 hits apiece, and then, again, you have a small number of sites on the right-hand end of that distribution: They’re hit factories. They’re really high performers. You want a visual, don’t you? There you go. Height of the bars represents NIBIN sites, and so the very first bar on the left is zero brass hits, and you can see about 28 sites never produced a hit out of NIBIN. The next bar is between 0 and 100 hits off of brass, and you can see that’s 35 sites. And then you see that distribution tails off fairly quickly to the right, and then you have some hit factories and high performers stretching out to the right with a maximum of what looks like about 3,100 hits. Not a normal distribution. All right?

Bullet hits, first bar on the left, these are sites that never produced a bullet hit. You can see the majority of sites never produced a bullet hit. And then the numbers of bullet hits just tail off all the way to the right.

One of the things we hit upon fairly quickly both from working in Trinidad and Tobago with their crime lab and homicide detectives — and I imported that when I took over this project — was timeliness of hits we think are really important. This is partially conjecture on our part, okay, if you read the report carefully. But I think it’s very reasonable to expect that criminal investigators need information on ballistics imaging hits very quickly. They’re going to work their case for a week or two weeks, and then they’re onto the next one, a week or two weeks if you’re lucky and it’s a serious violent crime. And so hits that come out later than that are probably less useful to investigators.

And this is I think at the heart of this NIJ grant, one of the things NIJ wanted to know: How do investigators use this ballistics imaging technology in their investigations? The first thing we have to tackle is, what’s the elapsed time? How long does it take NIBIN sites to produce hits? Data come from two sources. The first are out of survey data with NIBIN sites, and we asked them, “When evidence is submitted to your lab, how long does it take you to put it into NIBIN?” Okay? Ideally we want this to be hours, a day. I watch a lot of TV, I live in a fantasy world, because what we see is, the first bar indicates that the majority of sites indicate that it takes them between zero and 50 days, and the median for all of these sites is 25 days. And so it’s over three weeks. Number one. Okay? Because I’m sure there are firearms examiners in the room, or there are going to be some firearms examiners watching this, and their blood is going to begin to boil, and their blood should boil, because it usually maybe isn’t their fault. And so there are things that labs do and police agencies do that delay the processing time on evidence, and I’ll give you one of them. Some labs, when a firearm comes in, that firearm gets routed to DNA to see if they can get DNA off of it, and then it goes to fingerprints to see if they can lift prints off of it, and then it goes to firearms. And that probably weighs in here, right? These are things firearms examiners — they’re not lazy, they just can’t speed that process up.

Other cities, they allow a firearm to sit for five days, or 10 days depending on the site, and so the DNA or fingerprint people have five or 10 days to go down to property and grab that weapon, and then if they don’t, the firearms examiners come over and they say, “Give me all of the five-day-old guns.” Well, you’re introducing a five-day lag period just from the start through a lab process. Other sites, they’re not going to test fire weapons or enter brass unless the investigators specifically request that that is done. So we have multiple independent variables that are messing up our attempt to get information from this evidence put into NIBIN quickly.

Now, the second source of data comes from ATF, and ATF generously provided us with 19 very detailed hit files for 19 different NIBIN sites. They’re generally pretty high-performing sites in terms of hits. They were all active during 2011 and 2012. We have data on about 8,000 of these hits, and keep in mind that this is 8,000 hits out of about 50,000 hits. So it’s a sizable sample that we were able to grab. We have good dates for both crimes, because I need both crimes. Let me explain what I’m doing. I have a hit dyad. So I have, for example, homicide on January , and then I know that that hit with a second homicide on July 3. And so when I look at the data on July 31, I start a clock, and I count how many days it takes for that NIBIN site to confirm the hit. Okay? So I’m measuring this in days from when ideally you would find a hit within a day, how long does it take sites? And what we see is that there’s tremendous variation, as evidenced here. Heights of the bars indicate the number of days it took a site to confirm a NIBIN hit after that hit was available, after they had two crimes that were indeed linked by NIBIN.

As you can see on the left-hand side, these agencies are relatively slow. They’re taking more than 450, more than 400, more than 260 days on average, and this is the median, not the mean. Okay? So we’re trying to control for unequal distributions. This is how long it takes sites to identify hits. This is in a week. This is in two weeks. This is — the crime occurs in season one of your TV episode and then in season three, they get the ballistics imaging report back. Okay? And again, there are multiple reasons why.

On the right-hand side, we have sites that are fast. The city of Denver; Santa Ana, California; Syracuse/Onondaga County, New York — they’re pretty quick. Now I want to show you something, because this is data on 18 sites, and you say, “Hey, buddy, you have data on 19 sites,” and so I’m going to add a site, and it changes the nature of the distribution. And so I want to take a minute, because Stockton is currently calling their legal counsel for defamation. Right? Stockton, median time to identify a hit: more than 2,000 days. And let me explain why. Stockton struggled for years to find a firearms examiner. NIBIN hits can only be confirmed by a firearms examiner. They can’t be confirmed by a technician. So Stockton would, they would do the visual analysis on a screen. They would figure out what they thought were hits. They can’t confirm the hits. They send their evidence off to another lab in the state of California. That lab throttled the Stockton — and I mean throttled in both senses of the term, because they said, “You can only send us 10 cases a month.” It’s Stockton. They have a crime problem. They have a gun crime problem. And so Stockton had to carefully choose: What are we sending over to this other lab so that they can confirm our hits for us? And that’s why you see a lag time. It’s not strictly endemic to a problem at Stockton except they couldn’t get a firearms examiner to confirm their hits. Now, Stockton has hired on a contract a firearms examiner from Santa Ana, a guy named Rocky Edwards. Rocky is in there banging out hits like crazy. We’ve got a bunch of data on this. He’s doing a great job. But you can get an appreciation for why some sites are really slow and why some sites are relatively fast.

Part three of the findings: We wanted to know what role NIBIN hits reports played in criminal investigations. So, do they help you ID or arrest or charge a suspect? What’s their role? And so what I did is I went through these detailed data for nine of our 10 sites — because Bowling Green only had three hits, I couldn’t find case numbers in Bowling Green — so we picked our nine site visit sites, which are generally good performing NIBIN sites. Okay? Went through, and I looked for homicides. I picked a, basically I picked the population of homicide cases I could. Some of these cities don’t have a lot of homicides, so then I had to dig deeper, I had to pick aggravated assaults or robberies, but the majority of cases I’m about to present in the next three slides, they’re homicides. Most of these cases occurred between 2009 and 2012, and my site visitors sat down and conducted face-to-face interviews with investigators about their cases. Face-to-face interviews are, they’re time-intensive, and they’re — basically, they’re expensive. I have to spend time there. They’re a pain in the butt for agencies, because I give them a list of homicide cases, they have to look up who the investigator is, they have to figure out when that investigator is going to be into work, my site visitor is there for a week at most; it’s a big burden I put on these agencies, and I’m eternally grateful to them for producing investigators.

What I like about face-to-face interviews is you can read the investigator. A shrug, they look down, they laugh, they’ll tell you a joke — you’re not going to get any of that nuanced information from a survey and online SurveyMonkey thing. So I really like these face-to-face interviews.

So, we were able to conduct 65, we were able to conduct interviews on 65 of these cases. Here is what we found. At the point that the investigator got the NIBIN hit report, their case had already progressed. Half of them, they had already identified a suspect; in a third of cases, they had already arrested that suspect; and in 18 percent of cases, that suspect had already been charged. Now we wanted to ask them more pointed questions, because we know in some cases, let’s say you’re an investigator working a homicide, you have an idea that I’ve done it, you don’t feel quite comfortable in issuing a warrant yet, you get a ballistics hit report a month later, and you go, “That’s the straw that breaks the warrant back. I’m going to go sign out a warrant. I’m going to go arrest King.” That’s still good use. Right? NIBIN helps investigators arrest or identify. And so we asked investigators, “Did your NIBIN hit help you identify a suspect?” We find it only helped in about 10 percent of cases. “Did NIBIN hit lead to an arrest?” We only found one case where this was the case. “Did NIBIN help you obtain charges or a plea bargain?” Yes, in about 5 percent of cases. “And did NIBIN help with sentencing?”

About 2 percent. Now you see, the effect here is, it’s kind of curvy, like a Loch Ness monster. Right? NIBIN helps you identify people; it’s not helping you arrest them. It’s helping more in charging, and I think this is because of the time span at which investigations unfold. If you’re going to arrest someone you’re arresting them fairly quickly, within two weeks. So you’re going to do that without help of a NIBIN hit report. So you see why the arrest percentage is so low. But in some cases, you’ve already arrested them and they’re going to be charged, and so that hit report becomes useful. It becomes useful more for prosecutors and attorneys now. And so we have different time metrics. Right? Ballistics imaging reports take longer; investigations work in a shorter period of time.

Some quotes from our report here. And basically, we concluded that these NIBIN hit reports aren’t helping with tactical investigations to a great extent. They don’t help you identify or arrest; not really helpful in charging or plea bargaining. We think there are two primary problems. We think you could use NIBIN a lot more usefully, tactically in criminal investigations. The hit reports have to be identified more quickly. You’ve got to turn this stuff out quicker.

And then the second thing is that a lot of these hit reports just give investigators a date and a case number and tell you that it was linked to the same firearm involved in the date in this other case number. And those probably aren’t really useful to criminal investigators. They’re lacking a lot of nuanced information. And I’ve seen hit reports from the Houston Police Department. Houston Police Department’s hit reports are really good. They have addresses; they have the names of victims and the names of suspects; they have whether or not there’s a gang affiliation. There are a lot of really specific notes on these hit reports that I think are a lot more useful to investigators. They add the context, and basically they prevent investigators from having to go dig through a computer database and try and remember what the case was and where it happened.

Just to show you, this is just a little snippet of, it’s a snippet of 10 homicides committed by a highly violent criminal group in one city, a NIBIN site. I want to focus on their last four, well their last four homicides, which begins with a — it’s a homicide on September 12 with a 9mm, and then there’s a double homicide on November 25th with a .40 cal., and then there’s a third — well, I’m sorry, a fourth homicide with the 9mm and a .40 cal. And so you think on December 3, technically on December 4, we could have produced a NIBIN hit out of this. But about two weeks later ATF and the police go out, and they raid these bad guys, they make seven arrests, and they were able to string together 10 different homicides going back until early 2004 on these guys. And then you take a look at when the NIBIN hits come out, and the first NIBIN hit comes out relatively quickly. It’s two weeks later; it’s on December 31. But again, the deal is, these guys didn’t get arrested on a NIBIN hit. They weren’t identified off of a NIBIN hit, because the NIBIN hit didn’t come out until two weeks later, and that NIBIN hit probably contributes to them being charged. It probably makes investigators more comfortable that they really do have the group that they believe are committing these crimes.

The second NIBIN hit comes out at the median period of time for NIBIN sites, which at this time was 101 days. It comes out on April 4. This is very, very typical of how NIBIN’s used in criminal investigations, all right. It’s confirmatory, basically. And it segues into our fourth recommendation, and our fourth primary finding was NIBIN is a treasure trove of strategic intelligence for law enforcement agencies, not just ATF but for local police departments. Because if your NIBIN site is — if your lab people are regularly inputting evidence on gun crimes, you have a lot of information on where gun crimes are occurring, the weapons that are used. And if you’re able to associate other evidence with that, who and what and when and why and gangs and things of that nature, you have a large database of useful information. What we found was sites were not using NIBIN information at a strategic level: Only 8 percent are even passing their very sparse hit reports onto a crime analysis unit or a fusion center.

Now I’ll show you what happens when you start using NIBIN strategically. So not just to solve individual crimes, but to assemble an overall picture of criminal networks and targeting those criminal networks, and one of them is Onondaga County, New York, which is really good at this. This is Syracuse, and although they can’t necessarily pin individual use of guns on an individual, they can begin to figure out what the groups are that are involved. They are violent street gangs that are sharing guns among their members. Okay? Who wouldn’t want to go shoot someone with my 9mm handgun? What we see is, these community guns allow the police there and their prosecutors to outline the nature of the groups and then to prosecute those groups using RICO prosecutions. And there are other NIBIN sites and other law enforcement agencies that are doing this. I think this is a really fruitful practice. Partially, it doesn’t rely upon the immediate one-day or two-day NIBIN hit report; the timelines can extend a little bit more when you start using them strategically.

Now what do these networks of hits look like? This is a great little network analysis done by Andrew Fox at the University of Missouri at Kansas City. He used the NIBIN hit data from Kansas City, and what you see down at the bottom, these are just hit dyads. It’s a 9mm, or some weapon was used in two different crimes, or a test fire and a crime. These are just hit dyads. We’re more interested in what’s going on at the top, and especially in the upper left-hand corner, we see a whole bunch of either a gun or multiple guns used in overlapping crimes. You can see it’s daisy-chained into this rather elaborate network.

And what’s going on in Kansas City, in conjunction with their prosecutors and Kansas City Police Department and their crime lab, is they’re working on — it’s a lever-pulling project, kind of like a cease-fire project, and Andrew Fox’s role is he’s the analyst. He identifies who the trigger pullers are in gangs. And among the information, the criminal information he’s using, he’s using NIBIN hits to figure out who are these people who are involved in these shooting events, because these are serious violent crimes. Okay? And we want to incapacitate those people. And so this is just an example of the things you could do with NIBIN hit information as a strategic tool.

So my coauthors and I, during the last year of the project, we began talking a lot with ATF, and ATF began talking a lot to us. We were really fortunate in this regard. We began to have a conversation. Remember, the first time I went over to ATF, I sat down with Jim Needles, and I was showing him the lag times we were finding at sites. And I’ll be honest with you, I was kind of frightened that he was either going to argue with me and tell me I was full of crap, or he was going to shrug his shoulders and say, “Yeah, okay, whatever,” and basically usher me out of the building, which is normally what I get when I work with agencies. I’m pretty irrelevant, to be honest with you. Okay?

And I was surprised; Jim sat there, and I could tell he was getting it. He said, “Yes, yes of course, we know sites are slower now. You’re showing us this. We need to work on this. We’re going to implement some solutions to this.” And I could tell he was sincere. And when John Risenhoover gets up and talks to you, you’re going to see how adamant the ATF has been in implementing reforms. But those reforms began during the last year as I was writing a final report. And so if you read the final report, it’s very much written towards ATF as an audience in terms of reforming the program at the level of ATF. One of the things was we wanted ATF to find more performance metrics for their programs, especially in NIBIN. Be blunt about it. When you’re counting inputs, and you’re counting hits, you’re losing the picture. All right? It’s not an effective system, because you’re forgetting about elapsed time. You’re forgetting about what investigators are using. You’re not — you can’t measure what investigators really want to get out of hit reports to make it useful in investigations. And you’re also missing out on all the possible strategic uses of the data. We urged ATF: Build the capacity to start self-assessing your performance. Okay?

The second thing was that you want local sites to start using the hit information both tactically and strategically. Build this capacity. Basically grow your miniature Andrew Fox within a police agency or crime lab to start analyzing hit data at a strategic level, and start really pumping these hits out to get them useful at the tactical level for individual crimes.

We wanted ATF to build a research and development program to start experimenting with NIBIN and start determining what the best practices were. And then the last thing was there are different NIBIN sites that do things really excellently. So, Houston Police Department really writes good hit reports. Their hit reports are so good, my graduate students and I sat down with all of them, and we entered them, and one of my master’s students, named Jennifer Clausen, mapped them. No one that I know of has ever mapped NIBIN hits. Rocky Edwards does it with a software program, but he’s never analyzed it, and the only other research comes out of Israel. So Jennifer Clausen has mapped them, and she’s found some interesting relationships. You can ask me during the Q&A period, because I need to stop talking.

So we thought, you know, Houston has good hit reports. Santa Ana is really fast and has a great mapping program where they’re able to determine what stuff should go in first, and that software program is being used in Stockton now. It’s really improved performance in Stockton, and we’ve got data demonstrating this. So you have different sites that — they do different things really well, and so we want to learn why some sites are good at one thing and why this site is good at a second thing and diffuse those innovations and kind of best practices across all the NIBIN sites.

And so that was our goal. And so, now: Thanks. [Applause]

JOHN RISENHOOVER: I want to thank NIJ and all of the professors involved for doing this. It’s been a blessing, to say the least, to help us implement the new changes in NIBIN.

We go out on a regular basis, and we still get pushback from people who don’t want to make change, and we really view this as almost Moneyball. We’re freaking people out, and people are actually, I mean, I’ve been in rooms where I’ve had firearms examiners cry in front of me, because I’m destroying their world. All I’m trying to do is get down to a mission focus. And our mission focus really has to be one thing, and that’s what we’re going to do, and it’s going to be to lock up shooters. And based on that, that was our old mission statement with NIBIN. Don’t read it; it’ll just give you a headache. But that’s what you typically see when you drive something as a technology-based program. But does the American public give a crap if we have a technology-based program or we have a really neat network, or does the American public want something that they can wrap their heads around and law enforcement can wrap their heads around?

We’re going to lock up shooters. We’re not going to do anything else but that. Anything that gets in the way of that, we’re going to figure out a way to fix it. And if we’re not locking up shooters, then what are we doing here? This program was not instituted by the taxpayers and Congress so that ATF could have a really cool database and spend a lot of money on networks. They gave it to us so we could make the community safer. And we try to make that very clear, and we use this report every day when we go out to help us make that clear. We still have pushback. We still have people who don’t get it. We’re messing with their world.

I’m going to go through this real quick, because I want to talk to you more about stuff. And it’s all about crime gun intelligence. It’s all about what ATF can bring to the table. It’s not about taking guns away from people; it’s identifying the people who are illegally using firearms. And I’m very clear on that in everywhere we go and everything we do. We’re not about possession of firearms. Second Amendment is there; it’s legal. You can have guns. But you know what everybody can agree from left to right on? “Don’t shoot at people inside my cities. I want to go to bed at night with not having guns fired at me. I want my children to go to sleep.” Has anybody ever been into a house where they’ve put cinderblock walls in the bedroom? Anybody ever seen that? You’ve seen it. It’s a horrifying experience, guys, to go into a house and see a child’s bed with 3-foot cinderblock walls, because that’s where the kids sleep, so the bullets don’t come and kill them while they’re sleeping. And that’s what this is all about to identify. It’s basically taking old technology and the way we used to do business. We used to blow up cities trying to get to one factor. The reality is, with this technology and these techniques, we can precisely identify the one target in our community and remove that one target from our field of battle. Which, I hate to say it, is our inner cities. Not every gang member is a target. You know who is a target? The kid who’s shooting at everybody. And that’s all we’re trying to do.

I’ll pass this real quick. Basically, Bill did a great job explaining what it does. We leave marks on shell casings. We image shell casings, and we compare them with the images. We can try to make it fancier than that and sound cooler, but that’s all we really do. New technology is phenomenal. I mean, that’s the images we get now. Believe it or not, that’s a match. You have to look for a second, and you can see it. But a firearms examiner can see that. There’s people out there who can see that a mile away. But this is what it all comes down to: getting shooters off the streets in a real-time manner.

This is a great case out of Chicago, which goes to show when you don’t put stuff in what can slip through the weeds. Believe it or not, people get caught in Chicago periodically carrying guns. Shocking. But some kids got caught with a gun. No big deal, it’s a Glock 9mm. Of course, it belonged to no one, typically, which is true. They finally got around to tracing the gun, and when they traced the gun, they realized it was purchased eight years before it was recovered by law enforcement. Anybody in law enforcement says that’s a dead end. But luckily, ISP finally put that into NIBIN. And when they did put it into NIBIN, it suddenly had a different from eight years’ time to recovery from a one day time to crime. That gun was purchased and then used in a murder the next day. So, as you know, we have the trace report. We know who bought the gun. So we went and talked to her. Of course the statute of limitations has run, and we’re able to identify who the shooter was, and we were able to convict him and put him in prison. By mixing these two technologies together, we were able to remove a murderer from the community who was still out on the streets, very quickly and very easily.

One of these technologies by themselves wouldn’t have solved it, but tracing and NIBIN together did a great job in removing a homicide case that would have just sat out there forever. You know, again, we’re trying to give justice — I mean, I always hear “justice for the victims”; I’m just trying to prevent the next murder. I’ll get down to it. I’m just trying to prevent the next shooting.

Denver Crime Gun Intelligence, we talked about tactical, and they’re doing it on a day-to-day basis. This is a great case, and those of us in law enforcement can see this. A guy gets in a bar fight, somebody comes out, struggled around — you can carry a gun in Colorado for the most part — shoots the other guy. Of course, the shooter is going to claim self-defense; it was a fight. The DA is going to charge it at the most as involuntary manslaughter. But for some reason, this case went to life in prison. The reason why is that we get our stuff into NIBIN in a timely manner in Denver. We quickly identified that knucklehead liked to shoot at people at bars on a regular basis. This wasn’t a one-time event. He’s tried to murder people two other times. He finally just succeeded. And again, how often does it go down that it wasn’t an attempt, it was just sight picture and trigger control. That’s all it was. The difference in a shots fired and a murder can be down to that simple fact.

Cleveland. Now, Cleveland has changed their process. This was back when they were old NIBIN, and they’ve cleaned themselves up, and they’re doing a great job now. But we like to bring this up, because it’s a great example of what can be done. Domestic dispute; boyfriend shoots a gun in the air. No big deal. Cops show up; boyfriend is gone, gun is gone, girlfriend identifies the boyfriend, they pick up a shell casing.

A few weeks later, that same gun is used in a pharmacy robbery where they shoot the clerk in the leg. Now, if NIBIN was running real-time, I think you have a pretty good suspect when you only have a few days between the two crimes. You’re going to go find it. Unfortunately, at this time, NIBIN was running well over a year behind. So, everything you see here is historical. None of this happened. And then a few, a month or so later, we have a homicide with the gun. Well, it could be a community gun — the gun got passed around — except for one sad fact: Knucklehead got caught with the gun soon after that. So the boyfriend with the gun got eventually caught with the gun, and in between those two times, there was a pharmacy robbery and a murder. Can anybody guess what shouldn’t have occurred? The murder should have never occurred. If we run this in real time, we can actually prevent crime. We can get in front of these things before they can get to the point of a homicide. Again, this one is still ongoing. We have a couple of pleas still.

Since we’ve started implementing this program, again, I’ve told you about some labs pushing back on me. New Jersey State Police pushed back on me at first. And since then, they’ve actually changed their policy, and they’re trying to run NIBIN in a 24-hour time period. They’ve set up a statewide policy, because they see the potential of the program. They are willing to change the way they do business. But this can’t happen without constant review, having the academics constantly attack us and try to — I wouldn’t call it attack, this was very friendly, but — I’ll be honest with you; having somebody question the way you do business is not a bad thing. It’s a good thing. It helps you improve. Whatever we’re doing today, I hope in two years we’re doing much better. As good as we think we’re moving forward, I’m hoping in two years, Sharon’s going to be calling me going, “Oh, you all were so far off base. We’ve really gone a lot further than that.”

These are the four critical steps that we’ve really identified. Anybody who’s in the military has already seen these critical steps before. This ain’t rocket science, but believe it or not, for NIBIN it was. We really realized that you have to have comprehensive data collection. If you’re not going to do comprehensive data collection, what’s the point of all this? More often than not, we’ll go to places where the only NIBIN hits they have are on homicides. Can anybody guess why that is? It’s the only evidence that goes in the system. But the reality is, more often than not, a shots fired was an attempted murder. An illegal discharge of a firearm was an attempted murder. I’ll be honest with you. In most cities — I travel the entire nation, and most cities right now, if somebody shoots at you in the bad part of town and doesn’t hit you, it’s an illegal discharge of a firearm or a shots fired. No one is going to — a detective will never be assigned to the case. The case goes cold when the shell casings hit the ground. So when NIBIN is running a year behind on a case that never even had a detective, where’s that lead going? So suddenly, I have numerous shots fired because knucklehead can’t seem to get good sight picture and trigger control. He’s trying to kill people, but he can’t seem to engage properly. He never gets a detective assigned. Shooting, shooting, shooting, shooting. And what eventually happens? Unfortunately, more often than not, what do we see? Knucklehead doesn’t even get the guy he’s trying to hit; it always ends up being the 6-year-old little girl standing in the background. That’s the reality. Getting these guys off the streets, and we have to get this all in. Everything possible has to go in the system that’s suitable for the system. Plain and simple.

Next, we have to be timely in NIBIN. We did this in the state lab in Connecticut. We took them from a year and a half from a NIBIN hit to 12 hours. They have a shooting on Tuesday night; by Wednesday, when patrol comes back on duty, they have the leads on their desk. Do you know what most patrol officers will do if a detective hasn’t been assigned, and he has an active shooter in his neighborhood? He’ll go get him. But if I come back to him a year later, he’s not even running that district anymore. He’s not working that area anymore. If I’m providing him real-time intelligence, cops want to go get shooters. Don’t forget; everything we’re doing in here, and Bill will tell you, cops are sheepdogs. When they know there’s a wolf out there, their inclination is not to go back down in the corner; they want to go find the wolf, and they want to attack it. When you tell them there’s a shooter out there, their first inclination is to go find them and get them. And why timeliness is so important is we’re not DNA, we’re not fingerprints, we’re not CODIS, we’re not AFIS. We provide leads to detectives, and if we’re not real-time to do that, then what are we really doing? Because a detective solves a NIBIN hit. NIBIN doesn’t solve anything. It provides leads to detectives.

Follow-up. A simple thing, but believe it or not, you can do the first two steps perfectly, and if all that happens is when you provide that to somebody, they throw it in the trashcan, then NIBIN is a huge waste of taxpayer money. If nobody is going out there and arresting shooters and using this intelligence to make your community safer, then why are we doing it? As a writing exercise? The point is not to populate a database; the point is to reduce violent crime. We have to have follow up. But like you saw, the one in Denver where the prosecutor suddenly knew who was standing in front of him, it made all the difference in the world. If I think it’s a guy who has just gotten in a fist fight and hit somebody, and they fell down and hit their head and they died, yeah, that’s going to be involuntary manslaughter. It’s a big difference when somebody shoots somebody and then you realize he’s been doing this on a regular basis. The scariest thing about the Denver case is Denver PD only makes up 25 percent of the city, so the only thing that’s really participating in the Denver model right now, for the most part, is Denver PD. So knucklehead could have done several other shootings, but we won’t know for another year and a half from the state lab. And guess what happens if you take forever to get leads back? What do people finally stop doing? They stop submitting evidence to you. If they see no purpose for it, patrol won’t do it anymore either. So the importance of here is, there has to be a feedback loop to let people know why they’re participating in this program.

The military has been doing this for years, because it’s important for the guys on the ground in the Army to know why they’re picking up evidence on an IED bombing. I love to use the example, the Mod50 bomb that they had in Afghanistan. They thought they had several hundred bomb makers making this Mod50 bomb that was blowing up all of these soldiers. They didn’t know how to deal with it. But they quickly started doing good data collection. They processed the data, they put all the leads together, and they identified the fact that there wasn’t hundreds of bomb makers victimizing soldiers in Afghanistan; there were four. The nice thing about the U.S. military: I have a good feeling of what happened to those four individuals when they identified who those four guys were that were blowing up U.S. soldiers. Unfortunately, we don’t have that ability, so that’s what we have to do.***

But again, I sort of wanted to run through that. The important thing here, and everything we do, has to be to lock up shooters. And you have to understand how important this is, because — did anybody see what happened to Chicago on Fourth of July weekend? Eighty shootings; 94 people shot, Christy? Or was it —

CHRISTY: Eighty.

SA RISENHOOVER: Eighty people shot; 14 killed. We keep trying to approach this on, well, we’ve got to get the guns off the streets. Well, I go back to the point that there’s a reason a lot of people are carrying a gun on the street, if you’re on the south side of Chicago, because maybe somebody is shooting at you. Maybe the more important thing to do is to identify who the shooters are and get them off the streets. And we can all agree on that; whether we like guns or not, we can all agree it’s important to get shooters off the streets. But let’s think about this. We’ve tried to approach this from commonsense approach, but typical America doesn’t seem to get that excited for the most part about what’s going on in the inner cities. Let’s just be honest. Let’s finally get that out there. They’re not getting excited about that, but maybe they can get excited about the fact that of those 80 shootings that could average anywhere on the medical cost alone — medical cost alone is going to run anywhere between 20 and 40 million dollars for the Fourth of July weekend in Chicago, just for the hospital bills. Guess what? Most of those people don’t have health care.

Everybody in this room is going to end up being touched by that. Eventually, it’s going to touch everybody here. So if we can get the American taxpayer to understand — I can plead to the fact that no child should go to bed to the sound of gunfire. I can plead to the fact that the social, economical impact of this is horrendous, and we need to address it as a nation. But maybe I just need to plead to the fact that we can address gun violence economically through the use of great technologies like ShotSpotter, who help us identify where the shootings are occurring; through NIBIN, who helps us identify shooters themselves; and put these things together and actually save Joe Taxpayer some money. Maybe that’s the argument we need to take. I don’t know, but when we go forward, that’s why I like this academic study, that’s why I liked NIJ helping us. I want more. I’m not asking for less. I want to be challenged more on a daily basis, because what we’re doing here? Guys, this isn’t an academic study. We can show you pictures of dead children for hours and horrify you. But we’ve tried that, and it doesn’t seem to really horrify anybody in the American public for the most part. But maybe we can show you that by addressing gun violence in the long run, we can save the taxpayers money, or — and you’re not going to get the money back, let’s be honest about that — we can not spend money on other things, and we can spend it on something else. But if you’re spending it on medical care all day long, what are we really doing?

So that is my portion. We would like to have you all come up and some Q&A, because that’s where we’ll really get it. [Applause]

DR. RIDGEWAY: Thank you both, John and Bill. And as a reminder, we have microphones in the middle and on the sides. If you have a question, come on up. Line up behind one of the microphones. Please, first state your name and affiliation, and as a reminder, again, this is being audio recorded, and someone will come up and, if you ask a question, give you a permission slip. And if you don’t want to be a part of the audio recording, just let that person know. You can still ask your question, and we’ll just edit it out of the audio.

Okay, thanks. Come on up.

KELLY WALSH: Hi, thanks for the presentation. It was really enjoyable. I’m Kelly Walsh from the Urban Institute, and my question is, when you went out to the practitioners and explained, “Here is RAIN. Here’s this new system, and we’re going to take it from the timeliness numbers that your study showed to something new,” why the tears?

SA RISENHOOVER: A lot of people — labs get set in a very standard way. We’ve actually tried to move NIBIN out of being a forensic tool into an investigative lead tool. And basically, it just rocks people’s world. And we’ve done everything we can to remove every obstacle. We look every day for finding a reason to say yes to people, as opposed to the old days of saying no. So we’ve told people for the past — Sharon, what would you say? fourteen years? — that we wanted firearms examiners operating the equipment. Well, there’s only 850 firearms examiners in the world. They’re extremely talented, well-trained people, and they’re the only ones who can go testify in court. So more often than not, they’re getting pulled into court. But we also said, “We want two firearms examiners to confirm every NIBIN hit.” They are still in that mindset: If I make 15 entries a month, I’m a rock star, and I have to do everything. We’re telling them now, we’re willing to say yes to almost anything as long as it doesn’t mess up the quality of the database, because we want efficiency and process, and we want technicians to come in and do this. “Well, I want to do this. I like to test fire guns. I like to do everything the way I like to always do it.” And, again, it’s like Moneyball. It blows people’s minds, because for 15 years, we told everybody everything had to be a certain way, and now we are actually looking at people saying, “You come up with a crazy idea. We’ve got 150 sites. If it’s not going to mess up the database, let’s take a look at that and see if we can make that work, because you might be seeing genius while we’re not understanding what you’re trying to do.” But that’s what we get every time. And there are still several sites out there that they want to confirm every hit. Well, if you have a stop sign shooting to a stop sign shooting, you know, two random shootings that nobody ever is going to work or look at, they’re going to treat that the exact same way, as a firearms examiner, as a double or triple homicide. They don’t change anything.

And more often than not, we always hear the accreditation issue. Well, accreditation doesn’t allow us to do this. ASCLD, ISO, they won’t let us do that. We’ve already met with ASCLD and ISO, and they said, “don’t use us as an excuse anymore,” because accreditation means you do what you say and you say what you do. I always make the joke, and Sharon’s heard this a hundred times so she’ll be sick of it, is it’s like me going to my son and saying, “I need you to clean your bedroom,” but he says, “Well, Dad, I wrote a policy that I don’t have to clean my bedroom anymore, and only you have to do it.” Now, that’s when I hit him across the back of the head. Child abuse. But it’s fine; he’s 16. He’s big. But that’s what they do. They look for excuses to say no to me, and everything I’m doing is for a purpose. We’re trying to reduce gun violence, and timeliness really is an issue; efficiency of process is a very important issue. And what we’re doing today, I really am hoping in a couple of years, we’re going to even be more efficient than this.

My dream, and I think we all in the NIBIN branch dream about the day where you’re putting shell casings in at the crime scene, and two hours later, the detective who’s at the liquor store homicide knows that gun has been used in the following four shootings. So he knows which gang it probably is, he’s already calling those detectives, and by the end of the day he’s wrapping up a shooter.

DR. KING: I’m going to riff off of that, and you can set me straight if I get something wrong here. One of them is who is eligible to put stuff into NIBIN. And so, some of these sites, they’re spreading, they’re training firearms technicians from — they’re cops, and so they know where the terminal is, they’re putting the terminal in an accessible location. It’s not hidden away in a lab. And then, so they’re allowing more officers from other agencies to pound stuff into the NIBIN. And then, do you need to confirm every hit? Or if you have — if you look at the high-confidence candidate on a screen, can you release that possible, the unconfirmed hit, as a lead? And some sites have done that, and other sites I think are moving to that. They’re cranking down on inputs and timeliness. But some people are threatened by that.

SA RISENHOOVER: Detectives, you’ll ask them every time: Would you rather have a confirmed hit in a year, or would you rather have a lead tomorrow? They’ll take a lead, and where we’ve moved it up, I mean, New Haven, Connecticut. I can’t use the exact terminology the homicide sergeant used, but, “NIBIN has never been blah blah blah to me; now I see it as a real product that I couldn’t operate without.” Because now it’s real-time to him as opposed to a year later, when, again, shots fired are closed when the shell casings hit the ground. Agg assaults close in three days. If somebody shoots you in the bad part of Chicago, and you don’t know who it is, it’s probably closed in a day after the detective calls you. “Who shot you? Steve Miller. Do you know his DOB? Do you have a photograph of him and exactly where he is now? No? Okay. Closed.” Because they’re getting overwhelmed with shootings.

When I was in Phoenix, agg assault detectives were getting three cases a day. Three. They’re triaging agg assault cases because they’re overwhelmed with shootings. So the reality is if we can help them identify in a real-time manner while it’s still fresh on their mind, they’ll use that information. But if we’re coming back a year later, two years later, three years later? You’re just making them mad. That’s all you’re doing.

SAM BIELER: Hi, Sam Bieler, also at the Urban Institute. Excuse me. You talked a little bit earlier about some of the opportunities that could result when you’re using social network analysis from these NIBIN hits, but you also mentioned that mapping was an opportunity. I was wondering if you could speak a little bit more to the types of analyses you’re looking to conduct and what sort of strategic opportunities you think might arise from that intelligence.

DR. KING: Great question. One of the issues is, one, what sort of information gets input into the NIBIN database when firearms investigators are putting stuff in. And one of the issues is, old NIBIN, there aren’t any geolocations. That’s the bad news; if you want to map NIBIN hits, good luck. The beauty with Houston’s hits were, they were geocoding them on the paper as they come out, so they’re looking at the lems, figuring out what the geocodes are. There are some neat time and distance relationships between hits and how long the time elapses with the crime that confirms some research out of Israel. Are they putting geocodes in now?

SA RISENHOOVER: No. Again, we’re a sole-source vendor. We’re asking for these changes. We’re willing to pay for these changes, but there’s been studies that have shown that distance does make a difference and being able to map or even being able to condense your size. Right now, the system was set up basically on ATF field divisions, which could be very random. What you would assume we would do is, you would search out. Now we’re asking for that now, and everything that the studies ask for we’re trying to implement, so we can use — if shell casings are closer together, they’re more likely, that should give them a higher rank. Right? But if your field division is the Phoenix field division, which runs all the way up to, well it’s through Wyoming and Montana, shell casings that are 2,000 miles away probably aren’t the same gun. If they’re a block away, they might be more, because we’ll see that. If you actually map most of these shootings, even with ShotSpotter, when we link up with ShotSpotter and we use their technology to link where these cases are exactly found, you’d be surprised at how close — these guys, they don’t move a lot. They really don’t. Now and then you’ll, you do see some gangs go up and down corridors, but for the most part, most of the inner city shootings are in a very tight area, and they stay in their neighborhoods. But again, it’s key just identifying who those people are. And like in New Haven, even if we can’t arrest you, you might just get a knock on the door. In Denver, when they identify an active shooter and they can identify in that very small area, if they can’t get him on any real charge, we just got a guy who we put down for several shootings on habitual traffic offender. You know what the chief said? Good job. Because you got him off the streets for six months while we can build a case on him, because guess who nobody wants to testify against? The kid who’s still on the streets shooting at everybody. So getting him locked up just for a few months gives law enforcement breathing room, or much less stops him from shooting anybody, or much less, even worse, stops anybody from shooting at him. Because as we’ve seen so many times — it’s so horrific — they never hit each other, but who do they always hit? They’re always hitting the people behind them. The people that had nothing to do, I mean, how many times did we see in Chicago this weekend? How many of those people were the actual targets of the gunfire? So getting that — again, I’m going off topic, but we want to get that done.

Is - - in Stanford University did that study several years ago. They suggested — they said it’s a great idea, we need to do it. And we are going to try to get it done.

NANCY RITTER: Hi, gentlemen. Really an excellent job. My name is Nancy Ritter, from NIJ, and I would like to pull it back a little bit from NIBIN, but also I’d be interested in your response to this John, but mostly to Bill. You said that generally as a researcher, you walk into a place, and you’re considered irrelevant. I know that was only partly tongue in cheek, but I’d like you to take this opportunity to address, as a very seasoned researcher, how can we make researchers more relevant?

DR. KING: That’s a good question, Nancy.

MS. RITTER: [Off mic] Soft ball or hard ball?

SA RISENHOOVER: That’s a hard ball.

DR. KING: They all look like hard balls coming over the plate. All right, so I’m going to riff off of — David Kennedy gave a talk down at Sam Houston about six months ago, and one of the things he said in terms of working with agents, he said, “Yeah, professors are smart, but you walk into an agency, and they understand much more about what’s going on that’s relevant to their organization and their environment. So you should shut up and listen to them.” And what I and my researchers found was that we became much more useful to ATF when ATF began talking to us, and because they were talking, we were listening. We ended up with a really good relationship. I’m just speaking from this one, where they’re smart guys. They understand performance; they understand what this program is supposed to do, and they realized we could help them. We realized that they were paying attention to what we were doing.

The second part that I’ll riff off of is, something I learned from Ed McGuire was the mistake a lot of researchers make: You walk into the agency, you gather all of the data, you disappear; you come back, analyze the data, you present it to them, they look, they’re like, “Hey, this is pretty neat.” If it’s outside of their daily scheme of day-to-day activities — it’s a report, it’s a bunch of bar graphs, it’s a median, what the heck is that? — “Thank you, professor,” and they put it up on the shelf, and it gets all dusty. One of the tricks I learned from Ed McGuire was to try to implement an ongoing relationship of, let’s start doing this, let’s make a program, let’s make a unit, let’s have people, let’s periodically come back and assess the performance of that. You want to have an ongoing relationship. It’s kind of like the difference between a marriage and a one-night stand, to be honest with you. Writing the report and getting it put on the shelf is, you just got used. Right? As a researcher. Sorry. And in this case, actually, I get to go on further research dates with ATF, I hope. If that answers the question somewhat.

JOE BARTON: Hi, Joe Barton with Cornerstone Government Affairs. I had a question about the funding side of things. How would a police department go about receiving federal funds if they do not have the money to purchase equipment?

SA RISENHOOVER: Just getting equipment itself?

MR. BARTON: Right.

SA RISENHOOVER: They could apply through their local ATF office and write a proposal that they wanted equipment. Right now, there’s not a whole lot of equipment out there to be handed out. We’re currently redistributing equipment around the nation to make sure it’s being used effectively, and that’s our big thing we’re trying to make sure. But I’ll be the first to sit there and say it’s sort of good when people actually go out and buy their own equipment. We end up paying for the T1 lines. The T1 lines, which everybody is having to be upgraded, that alone costs us $15,000 a year. The equipment costs us $150; the service contract is $30. The bills add up really quick. It’s funny — Christy, this is yours. When you get on Southwest Airlines, and they throw you that bag of peanuts, what do you do more often than not? It’s not a very good bag of peanuts; you just shove it in the pocket in front of you, but you take it. If they asked you for a dime for that bag of peanuts, most people wouldn’t take it. If people have to put a little bit of their own skin in the game, I actually think it ends up being more effective. So that’s what we’re really trying to push as we move forward. We’re not trying to put more equipment out there; we’re trying to get the current equipment as effective as possible.

DAVID CHIPMAN: Good morning. My name is David Chipman. I’m senior vice president of public safety solutions at ShotSpotter. Thank you for your presentation, especially your candor. It wasn’t sugar coated.

I guess I wanted to start with just an observation. Two years ago, I was at a similar forum. COPS was putting on a presentation about fusion centers, and there was a line of people up asking questions, and this chiseled D.C. detective got up and said, “If you guys cannot produce a lead for me by the end of my shift, everything you’re doing is garbage.” And the room was like silent, and it made me really think that part of the reason that our relevance as researchers, as feds, has always been challenged is the challenge of real-time information. And, John, I wondered if you — or Doctor — if you could comment on how you might see that this new potential partnership with ShotSpotter providing real-time precise location of shootings to law enforcement in conjunction with officers who understand the value of NIBIN might take this program to another level.

SA RISENHOOVER: And that goes back to your centers of excellence, which we’re calling crime gun intelligence centers. And we currently have one in Denver. We’re implementing ShotSpotter into the program, too, to take it a step further. But what we’re really focusing on, and I’m glad the FBI is here, because I always compliment the FBI on how they run programs. They do a very good job. And I look at SafeStreets. I look at JTTFs. And we have JTTFs all over the nation right now, sitting by, waiting for a terrorism act and being able to act on it. But if we were able to put crime gun intelligence centers implemented with — well-staffed crime gun intelligence centers with staffing to make sure NIBIN gets done in 12 hours, that we’re responding to every shooting, that we have investigators to follow up and identify who the shooters are in your community and then do something about how to get them off the streets in a real-time manner, how many cities can we think of right now that they would be busy beyond belief from day one? Not just a little busy, but when we go into cities right now, where the chiefs of police tell us they don’t have the resources to really respond to all of the shots. They can’t even respond to shots fired. They’re responding to homicides. We’re talking to chiefs who say they don’t have the resources to work agg assaults. Where they can have a 90 percent solve rate on homicides, but they have a much less than 10 percent solve rate on agg assaults. It goes back to building a concept, just like you said with the fusion centers. We could do the same thing with NIBIN. We could do the same thing with tracing. We could do the same thing with the PD. And it’s not about who takes cases. That’s the biggest thing I push here. When we talk about tactical and strategic? The feds love their strategic cases. If you go down to New Orleans, where they do tons of these strategic cases and the charts are so big that the FBI will take a piece of the pie here, then the ATF will take a piece, and they go back and forth. So the concept is phenomenal, if we can build all of these assets together to attack this in a real-time manner.

MR. CHIPMAN: I just wanted to share one statistic. Last year, we conducted a study of ShotSpotter locations in over 40 cities, and something that shocked me — I was with ATF for 25 years, headed firearms programs; I thought I had a good sense of the firearms issue — we found out that on average, only one in five shootings were even being reported to police through 911. I thought that that number would be higher than that, not that low. And it made me think of, what would be a city that only had the capability to respond to one in five fires? What would we think of that fire department, and what would that do to a city? And so I think that, I appreciate your dedication to technology that I believe works. It’s the question of, can we get our customers to work it, and do they have the resources to do it? So, thanks for your time today.

SA RISENHOOVER: Thank you.

CHRISTINA GOODSEN: Hi, my name is Christina Goodsen. I’m a firearms examiner with Prince George’s County Police Department. Thank you for the presentation today. My main thing was, you mentioned the delay that can occur from having to go through DNA and fingerprinting before it can get test fired and entered into NIBIN. Do you have any proposed ideas on how — like, a lab that has to have it go to the DNA section and then the firearms, or the fingerprint section — how they can maybe speed that process up? Because some agencies do it really well. They have the crime scene techs do all of that, it gets to us extremely well, but then there’s other agencies out there that don’t. It has to go through each section. So I was just wondering if you had any ideas.

SA RISENHOOVER: It’s a matter of — I hate to, this is going to seem — it’s willing to do it. Because I was in New Orleans, and New Orleans had the same issue where they took three weeks. And I was actually in their lab for four months working with them. And I was able to take their process to two hours. That’s all it took. I said, “Well, what is holding you up?” And so I went over to the DNA people. “How long does it take you to wipe that?” Three minutes. “It takes you three minutes? Okay, how long does it take to hot glue something?” Two hours. “So where does two weeks come in?” So, it’s an ability to make change, and that’s where we go back to the whole thing, that a lot of people aren’t willing to make change. You have to address it. Why are you doing what you do in a crime lab? Are you doing it, and I tell you right now, guys, I’ve had lab directors tell me this — Sharon, you were next to me — “Why does everything take nine months? Well, because if we were to eliminate the backlog, we won’t get overtime anymore.” And that’s where Ron Nichols actually goes right at people and says, “Your acceptable backlog is costing lives on this stuff.” This is not theoretical. These are shooters. And once you start being, once you become Billy the Kid, you don’t stop being, you are Billy the Kid, and you don’t have a choice, you’re always there.

DR. RIDGEWAY: Can we end now on a happy note somehow?

SA RISENHOOVER: No.

DR. RIDGEWAY: At least, I mean, I think this is a great partnership between researchers and a law enforcement agency and the kind of thing that NIJ is really proud to support and hope to see a lot more of that. And let’s make that the high note.

SA RISENHOOVER: Okay.

DR. RIDGEWAY: Thanks to both of you for coming. I think the two of them will stick around a little longer if you have more questions. [Applause]

[END RECORDING]