Using License Plate Readers to Fight Crime, Part 4 of 4, Bruce G. Taylor, Principal Research Scientist, Substance Abuse, Mental Health and Criminal Justice Studies Department, National Opinion Research Center, University of Chicago

Bruce G. Taylor: Well, good afternoon. I'm going to discuss an experimental evaluation that the Police Executive Research Forum conducted. When I was the research director at PERF, I led this study. I had two colleagues, Chris Koper and Dan Woods from PERF, as well on the project.

I'm with a different organization now, the National Opinion Research Center, but I was the principal investigator for it, so I was asked to come and talk about the study, and I'm happy to do that.

OK. There were two phases to our study, and I'm going to be focusing mainly on phase one because we really won't have enough time to get to phase two, but, basically, phase one was with 117 what we called the “hot routes” of where we thought there would be active, sort of cars moving about that had been stolen. And I'll talk a little bit about how we came up with that term and how we defined these hot routes.

And then a second study was building on the first experiment, on things we learned. We designed a second experiment where we focused on a smaller number of routes. We worked with about half those number of — not the same routes but a different set of half of those routes where there were larger areas, and we did some similar experimentation.

I'm going to be reporting mostly on … today on some simple bivariate results, just some cross-tabs and some simple frequencies, but these results were confirmed with more complex modeling techniques. But I found the results were basically the same, and it's easier just to present some of the simple, straightforward percentages.

OK. So the first phase was a six-month experimental evaluation. Just by way of background with the experiment, I mean, generally, in the social sciences we consider the randomized experiments sort of the gold standard, you know, because we were able to sort of have equivalence between the treatment and comparison groups and be able to sort of isolate what the effects are, and that was our thought here. I'd been involved in a number of experimental evaluations in policing. I thought, “Ooh, well, this is an interesting piece of equipment,” didn't know a whole lot about it, had to learn a lot about it, but I thought, “Let's see if this sort of equipment can hold up to sort of the, you know, rigorous evaluation and see what kind of effects we might get from the equipment.”

So what we did was we identified … we were in Mesa, Arizona, and at the time, Mesa was in the top 10 for auto theft in the U.S., and so we thought, well, that's a good place; there will be a lot of activity for us to look at. And what we were able to do was we were able to identify 117 routes, hot theft auto routes. And, basically, what we did was we worked with a geographer to look at where the cars were stolen and where they were discovered, and she had a very interesting geographical spatial analytic method to connect those two dots. And it's more complex than I'm describing, but, basically, that's what she did.

And what we did was we identified a whole series of routes, but rather than relying just on the math, we also wanted to talk to detectives as well, and people who had been working the streets about where the routes were. I mean, sometimes the official information would be missing some other things that are going to happen in the study. So we identified an additional set of routes. A third of them came from talking with the detectives and officers.

And the idea there was that we we'd get a little more buy-in from the study where people thought they were contributing. So, basically, you get these 117 routes. You put them in a big hopper, and then you randomly assign them to different conditions. Now, just in general, these routes were about a mile in length, a mixture of residential and business areas, some highways.

We also had the officers grade each of those routes by ease of surveillance — were there nice places you could hide to put the equipment, or were there certain sort of difficulties with it, like if you have a … Mesa has got six-lane highways. They're really big. Those tend to be more difficult to set up because cars are going by real fast, and you can only cover so many lanes with the equipment.

OK. So we assigned the three conditions. Mesa had an auto theft unit that they had been using. This was a small spot of specialists who knew a lot about auto theft and been working these type of cases for a while. There were five people in the unit. And what we did was we wanted to look at how this unit used the equipment, and we wanted to use it when they didn't use the equipment, but we also wanted to have under normal patrol conditions.

So these would be … the auto theft unit, that's all they did. They didn't take calls for service. They would just focus on auto theft, and they would run out there and just be looking for auto theft. The question was, well, what happens when you give this unit this equipment, do they get better, or what happens when you have them just do standard sort of manual checking of license plates.

Then, we said, OK, we also wanted to compare that to what happens when you don't have a special unit, you just had standard manual checking of the plates, and notice how we set up the day. What we did was each route we assigned to get about an hour of sort of intense surveillance every day for two weeks. And what we did was the LPR team, they would have a route assigned to them, and then they would also have assigned to them, the same group of four officers would also have assigned another set of routes, which they wouldn't use the equipment, and they literally would turn all the equipment off.

We did some audited checking to make sure that was happening, and we had ways also with the LPR equipment you could tell if it's scanning, so that's not much of a problem. We could check to see that they were following the study design, and the officers were terrific, and really, you know, we explained the study and why we were doing it. I think it made perfect sense to them why we were doing what we were doing.

OK. Next piece to it. I'm not going to go into these advantages and disadvantages. We heard a lot about that already, so I'm going to move on.

Just to give you an idea of the geographic now, this is the city of Mesa. It's kind of hard to see there, but, basically, you can imagine … and you can see in the western part of the city is where a lot of the hot spots turn out to be, and, you know, so a lot of the routes end up being there, but they're all at least a mile in length.

OK. So, just in terms of the general tactics, what the officers would do is they would … when they used the LPR equipment, they had a route. They would first do an initial sweep of the area. They'd have the equipment. They'd check every little parking lot there was, every little apartment complex, run the roads, and just do an initial sweep of the area, and then they would try to identify, based on their experience, where a good place was to set up, and that way, you can get your maximum number of cars coming by, you know, going by. And they would pick those points, and they would move them during that hour if they needed to.

And there also was a supervisor involved who they would confer with in terms of what they were doing. Actually, I breezed by the other slide. Just so people know, there's a team of four officers that were working these cases, two marked cars, two unmarked cars, so you had four officers dedicated to a one-mile spot for an hour, every day for two weeks. You know, that's a pretty intense effort, and it's a pretty heavy commitment of resources by the Mesa Police Department to do the study.

Now, just in terms of any time you look at a randomized experiment, one of the questions comes up was did the experiment work; I mean, did you have equivalence between the treatment and the control groups. Well, yeah, we did. So, when we looked at … once we did the assignment, there were no differences in the groups in terms of what we said, pre treatment differences, the length of the routes, the speed limits; all those things were equivalent, number of arrests that were going on. There was just a whole range of things we were able to get equivalence through the experimental design.

OK. So what did we find? OK. First, not surprisingly, you know, the LPR leads to significantly more checks, about eight times the number of checks, and we had talked before this meeting, I think some of the estimates that had come up, some of the work may have been 20 times the number, but I think some of that has to do with probably where you set up, the kind of routes, the level of activity in cars. I think, maybe optimally, if you had a lot of cars flying by you, you could get maybe 20 times the number, but in our study, anyway, it was eight times the number. It was a still lot more checking than … and also, the other thing that it's important to keep in mind is that the officers who were doing the manual check were doing a lot of checks. I mean, that's all they were doing. They weren't answering calls. So they were just typing away. So you can type a lot, too, if you have people who are working hard. They can do a lot of typing, too. So we didn't find, you know, quite the 20 percent difference, but a lot more checks were done with the LPR equipment.

OK. The other thing we found was there were significantly more hits, so stolen car hits, stolen plate hits, other hits, total hits. We found that the LPR would give you more than the manual checks, and that was what we wanted to see.

OK. Also, we found more arrests as well. There was significantly more arrests than the manual checks for stolen car arrests and total arrests.

OK. The LPR leads to significantly more occupied stolen recoveries than the manual checks. So that was one of the things, of course, the officers were real interested in is they want to get a live body rather than just finding a car parked on the side of the street. So this equipment seemed to be helping with that as well during this time frame, so that's good news.

What we didn't find, at least in phase one, was we found no differences when you look at crime reporting through the UCR and calls for service. So you take these routes and you look at them across those three groups, the control group, the LPR route and the manual route, the control route being just standard patrol; we didn't find any differences. So we didn't affect the actual auto theft rates on those routes in the first phase.

Now, I'm not going to talk about the second phase, but in the second phase, we actually did find some differences that the LPR route did reduce the crime rate, so that was kind of interesting to see, but it was a little bit of a different approach that we took. But, in any event, we didn't find that here.

OK. Next thing. There were, however, less auto thefts during two weeks post period for routes with the manual checks compared to both the control routes and the LPR route. So, if you look at the far right side, the .05 and .08, at least on the … let's see. On the first one to the left, the UCR, the auto theft rates, there were differences for the LPR route versus the manual route. Then there were also differences on the ones that write the calls for service where we had the manual rate lower than both the control and the LPR routes. So that was kind of a puzzling finding. We said, well, why would manual checking lead to reductions in auto theft? We got a little bit of theory behind that, why we think that might have been.

No differences across the groups in all crime types during the intervention week. So this is where, instead of looking at just auto theft, we looked at other crimes that were occurring, all crimes, didn't find any differences there with the equipment, not that you necessarily would expect that, but we didn't find that, neither for the calls for services, for any calls as well.

Just a couple of notes about limitations about the study: Any time you do this kind of experimental work, it's important to recognize that we, you know, created somewhat artificial conditions was we told the officers you can only be on this route for this hour, and you had to move along the way we said, you know, and that's not how they work, of course. They'd much like to roam a much larger area. So the officers did feel a little constrained.

Now, the officers understood the study, and as far as we could tell from all the auditing we did that they stuck to what they were told to do. They were good soldiers, but, nonetheless, you know, I think that does go a little against what you would expect. So that might have, you know, in some ways, sort of constrained what the benefits you might see from the LPR because they weren't using it exactly how they would use it, but we needed to confine the study in this way to do this in an experimental way.

Also, we had 117 routes, which is a pretty big sample in this world of doing sort of hot spot policing, but, nonetheless, you know, it is a relatively small sample in the scheme of things. I mean, there's other studies done in criminal justice where you may have hundreds of cases that you look at, and we only had 117. So that limited the statistical power a little bit. So that's a limitation of the study.

LPR, I think, was only recently introduced to the agency. They had been working with it for more than a year, but, nonetheless, you know, the officers who were using the equipment had a fairly limited experience with it, and we gave them some pilot period to warm up to it. But, you know, I mean, it's safe to say that over time they are going to get it. The agencies are going to learn and they are going to get better with the equipment over time; whereas, the manual checking, well, they all had already been doing that for dozens for years. These officers already all had 10 or more years' experience, so they had a lot of experience with that. So there was some adjustment.

Also, Mesa, Arizona, you know, can run up to like 120 degrees in heat. I know when I was out there, it reached that, and, you know, this equipment, you know, held up pretty well, but there were times when they ran into some equipment failure-type issues that they had to bring in for service, and I think some of that we understood may have been heat related, so there may have been a little bit of down time during this.

I think also, you know, we noticed that there's probably a need for new technological advances. As Dale mentioned, you know, it doesn't scan every plate it sees, which, being a novice to this area of LPR equipment, I thought, “Wow! Doesn't it scan?” No, it doesn't, and I don't know the exact number of scans, what the percentages are, but maybe it's about half it scans or so or depending on the equipment?

Audience Member 1: It's on how many variations you see.

Taylor: Right. So it's probably on the context, lighting conditions, maybe a variety of things that kind of affect it, but that's an important thing.

Now, I would assume over time that the technology is going to get better. So, as the technology gets better, you read more plates. You know, the results may be better. So those are just a couple of things. You know, it doesn't handle multiple lanes very well. I mean, you know, basically, the officers … I think the manufacturer, this was the Remington equipment. I can get people the exact model number, if you're interested in it, but they had told us it could read like four or five lanes. And, no, it doesn't do that. You know, I think they were pretty comfortable with about two lanes in, and even that, there were, you know, real implications with it. You know, if you're on a highway and you've got someone flying by you at 80 miles an hour and you're in a standstill, it's kind of tough to go catch up to them, especially someone who wants to get away; at least, that's what the officers told us.

So there were cases where there were safety issues where the officers didn't feel they could confidently go catch some of these people, but they'll get better with those equipment. They'll come up with better strategies. Maybe they'll be setting up barricades and other things in the future, but they weren't going to do that here in this particular case. So I think that's just some limitations that are worth saying.

And just a couple of summaries of our findings: So, we found a whole lot more plate scanning that you did with the LPR equipment compared to manual checking. The LPR was associated with more positive scans for auto theft than manual checking for auto theft and stolen plates, more arrests as well. So that's all good news. Unexpectedly, as I mentioned, we found that the manual checking was associated with less auto theft two weeks post intervention than the control group, and we found no evidence of crime displacement as well.

So just a couple of notes here. I know I'm kind of running out of time here. I think these are some of the implications of our findings is that a specialized auto theft unit conducting intense manual checking is associated with lower auto theft compared to standard patrol, shows less frequent plate checking. So I think that's good news, and that when you do broad-based license plate checking, there is some value to that, regardless of how you're doing it, with a manual checking or you're doing it with the equipment. So that's one of the things we did find.

I think what the trick's going to be is to figure out some modifications in the strategies. I mean, one idea that we sort of kicked around about why the manual checking reduced auto theft in this case and the LPR equipment didn't was just how you're using the LPR equipment compared to how you're doing the manual checking.

The manual checking, basically, what you had to do was you had to travel with traffic to do a lot of plate reading, but just sit on the side of the road, pretty tough to read plates as they come flying by, so you have got to do that. And you create a little more visible of a presence; whereas, while we did that initial sweep with the LPR equipment, we'd basically set up in a place that you can kind of hide, so the bad guys hopefully wouldn't see you and they won't redirect around some other way.

But I think what happened is that the manual checking, when they're doing the manual checking on those routes, they probably created a greater presence potentially in those areas, and that greater presence might have actually helped reduce auto theft in those particular routes. I mean, that was one idea we had talked about with detectives, some of the officers, and we have to think about it some more, and we have those phase two results that we've also been analyzing as well, and that may help yield a little bit there.

But one idea that came out was that we might need to think of some new strategies, and one of them was that if you have the LPR equipment, you know, there may be ways that you may want to be a little more visible in some cases with the equipment. They really were focused on real covert sort of activity with the LPR. Maybe that's not the best thing to do, and there may be more of a mixing of the manual checking with the standard scanning. There's no reason why the officers couldn't have the scanning going and doing other stuff at the time, I suppose, and that was one of the things they started thinking about was just sort of observing their settings around them and other kinds of strategies. And it's real interesting to hear what the other two studies and they're already thinking about a lot of that stuff, and I think as we move forward, this kind of work, it's something to think about, some revised strategy to enhance maybe some of the auto theft reduction capabilities of the equipment.

I pretty much hit that. So I think I'm basically out of time. I will go ahead and wrap up. I'm not going to really have time to go to phase two. So we'll go ahead and wrap it up right there.