Geoff Barnes: The Probation Department had a one-size-fits-all supervision strategy in place. Every offender was getting roughly the same amount, coming in approximately once a month, seeing their officer for 20 to 30 minutes, maybe. It just wasn’t a whole lot of actual contact with the offenders. And, clearly, their budget wasn’t going to go up. They weren’t going to get any more officers.
They really wanted to utilize the resources they had available at the time, which weren’t going to increase, but focus those resources on the people that it made the most sense to focus upon. So identifying those that present the biggest risk to community safety and focusing on them, and at the same time looking at people who really present no risk at all, but had been sentenced to community supervision. Something had to be done with them, but maybe let’s not expend so many resources at the lower end of the spectrum, and instead take what would have been wasted on those people and put it on the people that present the highest risk.
Jordan Hyatt: So what this prediction model does is it predicts, using information that the probation department largely had available already, the likely conduct of any probationer for the first two years of their term of probation supervision. So there are three outcomes for this particular model. The lowest level of risk suggests, or it says, that the offender won’t commit any new offenses during that two-year forecasting period. The moderate level of supervision says that the offender will commit a crime, but not a serious crime. And the highest risk of supervision includes those offenders who are forecasted to commit a serious offense, which is generally defined as murder, attempted murder, aggravated assault, rape and arson.
And so what the Probation Department has done is, based on these forecasting outcomes, they supervise offenders in units based on those risk classifications, so the highest risk offenders, those most likely to pose a danger to the community, are supervised most intensely, while the offenders who are predicted to commit no new offenses or relatively minor offenses get a decreased level of supervision.
Hyatt: And I think that gets an important point, though: the difference between the researcher side of what we’re doing and the practitioner side. All that the model is doing is saying that based on the information available about this individual, this is what they’re likely to do over the next two years. It’s up to the agency, in our case the Probation Department, to decide what to do with that information.
So here they decided to supervise dangerous people more intensely, but that didn’t have to be the case. That was their decision to make and that’s the policy half of this equation, of this partnership.
Barnes: I mean probably the most crucial thing, at least in this project, has to have been that we had researchers and practitioners, but it wasn’t that there was one group on one side and one group on the other. It all had to come together, and it all had to be a partnership, and it had to happen in concert at every single step. We couldn’t have built a model without knowing from them, from our partners at Probation, exactly how many people could they possibly deal with being labeled as high-risk without knowing their capacity to supervise people, and exactly what they wanted to do, and how many officers they could devote to that, without knowing that it couldn’t go much above 15 or 18 percent, we could never have built the model in the first place. Without their data, we could never have built the model in the first place.
Barnes: One thing that seems to be a very big improvement in random forest modeling, as compared to stuff we maybe did in the past, maybe stuff that other jurisdictions tried in the past and weren’t very pleased with the results, one of the things is that the amount of information that we can use, you don’t necessarily need to go into the modeling process knowing, well, we think this is important and so we absolutely must go get this.
Lots of different things, lots of different values can be used to predict future behavior, even things, which probably this sounds very strange, but even things that don’t predict future behavior very well, can be included in the model. In traditional statistical procedures, what you typically would have to say is, “Well, we can only have a limited number of predictors to forecast future behavior.” In random forest modeling, you don’t have to be so choosy. You can afford to put things in that maybe don’t work well for older offenders, but work very well for younger offenders.
But I think the important thing is that each individual jurisdiction has access to things probably that they haven’t even thought about. They put this information in as a matter of course, as part of their day-to-day routine, and never really realizing how enormously powerful it could be with just a few edits, with just a few manipulations of it to convert it into a set of numbers that could forecast future behavior.
Well, all this forecasting technology can look overwhelming in a lot of ways. I think if you look at our report, you see something like, ah, well, the model makes nine million different decisions.
Hyatt: Yes, 8.4 something million decisions.
Barnes: Yeah, you know, you look at it and you say, “How could we ever get to that? It looks so complicated.” But I really think that the reality is different. I think that with the exception of maybe the very smallest jurisdictions, the data are available. It’s just a question of making use of stuff that you already have to build a customized model that fits your particular jurisdiction and your particular needs at the particular phase of the criminal justice system that you are interested in.
Barnes: We have to be very careful about how we allocate the precious resource we have, and the most precious resource is time. Every public employee, whether they’re a probation officer or a police officer or a corrections officer, they all only have a certain amount of time that they can devote, and attention that they can devote, to their jobs. We have to make sure we allocate those resources in ways that make sense.
But I think the other reason why we really have to focus on prediction is that, chances are, it’s happening anyway.
Hyatt: So probation officers or anybody in criminal justice, people are making judgments about the relative risk that an offender or a probationer poses already. What risk assessment, and specifically actuarial risk assessment, lets us do is ensure that we’re making those predictions in the most fair and equitable way possible. So by using a prediction model like the one in Philadelphia based on random forest modeling, we can be sure that we are identifying the most dangerous offenders in the most accurate way possible, and that we’re doing that in a consistent and fair way. And it ensures that we’re both preserving resources, but also the people who are subject to the policy decisions based on those risk assessments are being treated in a fair and consistent way.
Barnes: So one of the reasons to really want to bring this forward in the criminal justice system is that in a lot of ways, it makes the system fairer. It not only is more accurate, but at least you know that however you got put, or if you’re coming into probation and you have to come in once a week because you got put on high-risk probation, at least you know that the decision that put you into that situation was made the exact same way for you as it will be for the guy that comes after you and the guy that came before you, and everyone who comes into probation gets assessed on the basis of the same criteria.
You may not like being on high-risk probation, but from a procedural justice standpoint, you at least know that the decision was made the same way for everybody.
