Wrongful Convictions: The Latest Scientific Research & Implications for Law Enforcement

Greg Ridgeway: Good morning, and welcome to today's seminar in NIJ's Research for the Real World seminar series.

I'm Greg Ridgeway, acting director of the National Institute of Justice. I really appreciate you taking the time out of your busy schedules to make it here today. Thanks.

Today's presentation, titled “Wrongful Convictions: The Latest Scientific Research and Implications for Law Enforcement,” will feature Dr. Jon Gould, which he is still here. I'm glad you made it. We're also going to have Mr. John Firman, the research director at the International Association of Chiefs of Police, and they will discuss findings from the latest, most scientifically rigorous study of wrongful convictions that has occurred to date. In addition, with the addition of Mr. Firman, we're going to get to discuss the implications for law enforcement.

This study is part of NIJ's growing body of research on wrongful convictions. We view this as a really important area. We're trying to cultivate and encourage researchers to think about these topics and these problems, figure out the difficult way we can study them, because they don't happen a lot, but when they do they're shocking and surprising. What sort of things can we uncover, what sort of evidence-based approaches can we take to figure out how they happen, and then what sort of evidence-based approaches can we come up with, develop, to prevent them? So if you want to learn more about this, we have other related studies at our website, NIJ.gov.

So, of course identifying and understanding the causes of wrongful conviction is critical to maintain the integrity of our criminal justice systems. Our wrongful convictions of innocent persons undermine the public's confidence in our justice system. Of course it harms the individual innocent individuals who sometimes are put away for decades. It deprives the victims of justice that they expect from our justice system and of course it potentially allows the guilty parties to go free with risk of victimizing others.

So, glad all of you are here today; you are stakeholders in this regardless of what your role is in the justice system. If you're just passing observers, part of the members of the public, or you're part of our Department of Justice, there's something for you to do in this system. So my hope is the information you receive today, you'll learn something that you can take away and figure out how and wherever you are, whatever the career is, wherever you work, that you too can make a difference and use these efforts, these research efforts, to help what you do.

Now it's my pleasure to introduce our speakers. Dr. Jon Gould is a professor of law, justice, and society and Director of the Washington Institute for Public and International Affairs Research at American University. His work focuses on civil rights and liberties, justice policy, and legal change, helping to make academic research relevant and accessible to policy making. His first book, Speak No Evil: The Triumph of Hate Speech Regulation, was a co-winner of the 2006 Herbert Jacob Award for the best book in law and society. His second book, The Innocence Commission: Preventing Wrongful Convictions and Restoring the Criminal Justice System, was named an Outstanding Academic Title for 2008 by the American Library Association. Dr. Gould holds a Ph.D. from the University of Chicago and J.D. and M.P.P. degrees from Harvard University.

Mr. John Firman is the Director of the Research Division of the International Association of Chiefs of Police, IACP, the world's oldest and largest police leadership organization. It was founded in 1893. The IACP currently has 22,000 members representing 100 countries. Mr. Firman's duties include development and implementation of a national and international law enforcement, policy, research, and evaluation program for the association. He helped create and currently manages the National Law Enforcement Policy Summit Series for the IACP, addressing current and emerging issues in the policing profession. Prior to joining IACP, Mr. Firman was an appointee of the Governor of Illinois, serving as Associate Director of the Illinois Criminal Justice Authority from 1985 to 1994. Mr. Firman also serves as an adjunct professor at American University, where he teaches the advanced seminar in policing in the graduate school. Mr. Firman holds a B.A. in sociology from La Salle and an M.A. in sociology from Temple University. Please join me in welcoming our speakers, Dr. Gould and Mr. Firman.

Jon Gould: Well thank you all for coming out today on such a lovely spring day with snow on the ground. Now I don't know about you, but my calendar says that spring began last week, which is about the same time that we got word of yet another exoneration, this one taking place in Brooklyn, New York. The Brooklyn DA's Conviction Integrity Unit there concluded that a man who had spent 23 years in prison needed to be released because the prosecutor's office was not convinced that they had convicted the right person. And in fact, we are at the point now where we continue to read regularly about cases of wrongful conviction. But even with this growing body of cases and case facts that we are now aware of, we are largely lacking a large-scale systematic evaluation of the causes of wrongful conviction. Today what I'd like to present to you are the results of a three-year project using empirical methods and a control group that really brought social science methods to the table to examine the causes of wrongful conviction.

I think what's important about this study is that first of all, we have a control group, which I will talk about later, but second of all, what we're able to do in this study is distinguish those sources of wrongful convictions that explain why someone is indicted or enters the criminal justice system, and we're able to distinguish those from other sources that end up helping to convict someone once they've been indicted. In being able to distinguish those sources, I think we're going to be at a point where we're able to create better responses to the sources of wrongful conviction. And indeed those responses need to be joint measures. They need to involve police, they need to involve prosecutors, defense lawyers, forensic scientists and the like. And for that reason I am particularly pleased that we have the IACP and John Firman here today to talk about the work they've done on this, because we really now are moving to the point where there are a lot of groups that are getting involved in coming up with solutions to a problem that has been with us for quite some time. In fact, it's actually quite relevant that we're talking about this issue now in 2013, because we are at the 100-year mark of the first literature that was published on wrongful convictions.

Edwin Borchard, who was a professor at Yale University, is really seen as the father of wrongful conviction research, who in 1913 published the first work on this, at that point looking at cases in Europe, but in 1932 he came back with another study of 65 cases of wrongful conviction. We went roughly half a century from his landmark work until we really saw the age of the modern study of wrongful convictions, and that was in 1987 when Hugo Bedau and Michael Radelet came out with their law review piece in Stanford Law Review and then later on published a book that gave hundreds of cases that they claimed were wrongful convictions, and indeed they suggested that many of those people who had been wrongly convicted had actually been executed. Now this was at that time of fairly controversial study, because the next year, in 1988, Stephen Markman and Paul Casell published a response in the Stanford Law Review. Now for those of you who know the names Stephen Markman or Paul Casell, well, they were in the Justice Department at that time. Mr. Markman is now Justice Markman of the Michigan Supreme Court; Paul Casell later became a U.S. District Court Judge in Utah and is now back on the faculty of the University of Utah. And they challenged Bedau and Radelet on their definition on wrongful conviction, saying essentially that the academics were using their own interpretation of who was wrongly convicted, but did not have ironclad proof that the people who these two professors claim had been wrongly convicted were in fact innocent of the crimes that they had been charged with.

Well, that issue then began to become settled in the 1990s, when DNA came on the scene. And as we know, DNA testing is able to show whether biological evidence found at a crime scene comes from — with almost certainty — a particular individual. The rise of DNA then gave the kind of proof to show whether in fact individuals had been wrongly convicted of crimes. So we saw this work begin with journalists, beginning to some extent at Northwestern University in a journalism class there with reporters also from the Chicago Tribune, the Innocence Project sprung up — we all know of their work. And we are now at the point where there's a national registry of exonerations that's been created by Northwestern University and the University of Michigan Law School, which has hundreds of cases of wrongful convictions.

Well, from that work, we now have what some folks call the usual suspects or the usual story of causes or, I should say, sources that have been identified with erroneous convictions. Now I'm going to be using the expression “erroneous convictions” and “wrongful convictions” almost interchangeably. I think to some extent the better term is “erroneous convictions.” “Wrongful convictions” implies that there was intent on the part of the individuals who were involved in the cases and certainly from our research, we don't see that quite as often, but I recognize that the term of art has become “wrongful convictions,” and as such I will be using those terms interchangeably.

In any event, much of the research to date has identified these seven sources as connected to cases of erroneous convictions. So problems in eyewitnesses — being able to identify whether someone was the actual suspect. False confessions. This term “junk science” is used — it is a little bit of a misnomer. We're talking about kinds of biological evidence that have now had questions raised about them. So for example, hair analysis, to some extent handwriting analysis, to some extent fingerprinting, mainly the hair analysis and earlier blood testing that has been supplanted by DNA testing. Other sources include snitch testimony, police and prosecutorial misconduct, poor defense lawyering, and then of course something we all are familiar with, tunnel vision.

But here's the problem with some of the research today: It's largely been case studies. Now, I have nothing against case studies, and I have nothing against the folks who have done that work, mainly journalists, law professors, mainly those groups. And indeed I teach law, so to take full attack on law professors in this work is not going to be in my best interest, but here's the concern: Because these are case studies, because they lack a control group, because these are not, oftentimes, using social science methods, what we have in these studies are findings that are able to say that many of these factors occur in cases of erroneous convictions, but we're unable to say whether in fact those same sources are found in other cases that are not erroneous convictions. In other words, are these specific to cases of a wrongful conviction or are they shared by other cases in the criminal justice system? For that reason, I have real problems with these being talked about as being causes of wrongful convictions. These need to be thought of more appropriately as correlates or factors that are found in cases of wrongful conviction, but these are not necessarily causes unless we are able to use other kinds of methods to address these cases or to examine these cases.

And in fact, that's what we did in this study. This study was in response to a call from the National Institute of Justice to use social science methods in examining why it is that erroneous convictions occur. And what we did in this case is we created a control group of what we call “near misses” — and I will talk about them in a moment — but what we're trying to do here or what we were trying to do in this study is answer two questions. And that is, what factors explain why certain innocent suspects are erroneously convicted in particular cases, but those innocent suspects have their cases dismissed or they are acquitted in other cases? In addition, there's a policy question, which is, if we know this, then what kind of policy interventions can help the criminal justice system identify the innocent early in an investigation to prevent wrongful convictions?

I need to explain from the beginning that this case is largely exploratory, in the sense that there has not been prior research on near misses. Almost all of the research concerning erroneous convictions has looked simply at why it is that people are wrongly convicted. So what we have here is essentially an inductive study, meaning that we are trying to build theories for why it is that a case will come in one way and become a wrongful conviction, and come in another way and be — this is an innocent person — have their case dismissed or be acquitted at trial on the basis of factual innocence? We were using the erroneous conviction literature as a guide here, but really what we're doing is building variables, we're building explanations as we go to explain why it is that the cases go one way versus another.

Our case criteria had four definitions. So first of all we are looking at cases of violent felonies where the person was convicted or indicted by a state. These are all post-1980. Why post-1980? Because those are the cases where we were able to find the kind of information to be able to then build our database. It has nothing to do with whether anything specific happened before 1980 in the field of wrongful convictions. This is simply the fact that we could get data for cases after 1980. Then of course defendants had to be factually innocent. And in this situation we used a very conservative definition of factual innocence. And we were using one that we were hoping would address the concerns that Markman and Casell had in the 1980s. So our definition had two parts to it: Number one, the defendant had to have an official statement of innocence. What does this mean? It means either a judge had to have declared him factually innocent, a prosecutor had to have said he was factually innocent, a governor or a juror. What does this screen out? Here's what's not in our database: So you could imagine the cases where someone has his conviction overturned or the case is dismissed, but the judge gives the prosecutor the option of refiling. Those cases are not in our database unless a prosecutor also announced that he was not going to refile because he was convinced that the defendant was innocent.

Now we say throughout the report that this likely means that we are underreporting — and probably by a certain magnitude — underreporting the cases of innocence here. But we wanted to make sure that we were dealing with cases where no one would be questioning whether in fact we were dealing with someone who — there weren't enough facts to convince a jury beyond a reasonable doubt that the person was innocent as opposed to this person being factually innocent. That's why we used that first criterion.

The second criterion is that we ourselves after doing investigation of these cases had to be convinced that this person was indeed factually innocent. So again, two prongs: One, an official statement, either an exoneration or the case was dismissed or acquitted, and second of all that we ourselves had to be convinced that this person was factually innocent.

Work like this is incredibly time-consuming and requires a number of different people to help bring it together. And in fact, this was a project that had four faculty investigators and nine graduate students involved in the research, and I think we have two of them here today — Katie and Jackie, would the two of you mind raising your hands for a moment? So these are the people who really did the lion's share of the work on a project like this, along with seven other of their compatriots. I should say, if I had known three and a half years ago, when I bid on this proposal, what it would involve, I'm not sure I necessarily would have done it, because this is incredibly intricate, but the opportunity to work with such talented folks as Jackie and Katie really has been a boon here.

In addition, we had tremendous institutional support from a number of organizations that are involved in these cases. So the Innocence Project allowed us access to their database that the law firm of Winston & Strawn has put together that has the case files for many of the cases of erroneous convictions and the ones that they have listed on their website. In addition, as I'll explain later, we could not have done this work without the Police Foundation that helped us in building a scale about the facts in the cases. We were then assisted as well by the Association of Prosecuting Attorneys, the NDAA, and the NACDL, who gave us access to their members so that we could get the facts and the information that we needed to do these cases, I should say to investigate these cases. And for that we are eternally grateful. This work really needed folks on all sides of this issue to come together and be of assistance; otherwise we could not have done this.

So this is a lot harder an enterprise than we might think to try to identify cases. There is no database of near misses. None. It does not exist. We were able to identify one study of near misses that had been done. Now fortunately, there has been a good deal of work on wrongful convictions. As I said, we were able to find some cases through the Innocence Project. In addition, we looked through academic publications, other databases, we did searches through news media, and then with the help of the NDAA, the APA, the NACDL, and other organizations whose acronyms I cannot recall at the moment, we sent out broad-scale solicitations, to lawyers, to police officers to help us find these cases of near misses, to find additional cases of wrongful convictions. And what we asked people to do was nominate cases and we asked them to some extent if they wanted to nominate them anonymously or confidentially so they did not actually have to tell us anything more than, “We think there may be a case in this jurisdiction involving this particular defendant.” And we then had to go find these cases and we had to do a whole of investigation of these cases to see whether we could verify that the cases qualified, first of all, as a wrongful conviction or near miss and to verify whether in fact the defendant met our definition of innocence, of factual innocence. And in fact we ended up discarding more than half of the cases that were brought to our attention, because they did not meet our criteria. And in the end, we were able to include 460 cases in the study. As you see, 260 of them were erroneous convictions and 200 of them were near misses.

Now I'm going to show you a slide next that gives you the distribution of these cases. For reasons that I still do not understand, this does not come out on the screen very well. It looks very good in the report. So if you have problems seeing this, and I suspect you will, go online and read our report. But what I can tell you is that the circles are the wrongful convictions, and the triangles are the near misses, and here's what's particularly interesting to us as we look at these. First of all, the near misses are more evenly distributed geographically across the country than are the wrongful convictions. If you look at this, you'll see the wrongful convictions are mainly along the eastern seaboard, from Boston on down to Washington, D.C., another group in North Carolina, in Florida, Chicago, Houston, and then to some extent on the West Coast. As Marvin Zalman, who's a professor at Wayne State, has written, this phenomenon could be seen as hot spots but two different ways. One possibility is what we're seeing here is a distribution of where wrongful convictions typically occur, that they occur more often in, say, New York or Chicago. The other possibility, and I think the one that is probably more likely, is that what you're seeing here is centers of activity to investigate wrongful convictions. So the Innocence Project is headquartered in New York; a lot of the early journalistic activity and academic activity to identify wrongful convictions was in Chicago at Northwestern University. There have been problems in Houston with its crime lab — that may very well be the kind of hot spot we think of when we talk about crime, but we need to be mindful that when we're talking about wrongful convictions, any study of them, we are limited to where these cases have already been identified. And so, I cannot go out and find anything that has not already been identified, and it may well be that when we are talking about geographic differences, we are not talking about what causes wrongful convictions, but rather what sources then lead to the investigation that identifies wrongful convictions. The near misses, though, seem to be a little bit more evenly distributed across the country. But again, we recognize that what we may have here is an artifact of where people are willing to tell us cases exist rather than in fact where cases come from.

Now we used a number of different methods to do this research. We started, as I said, with gathering case facts. We recorded them in, originally, simply descriptions of what happened in the cases. These went on for many pages, and we then used those to be able to distill these down into roughly 60 or so variables that we were able to put through a quantitative analysis using statistical software. But we also were able to benefit a great deal by a scale that the Police Foundation has created to look at the strength of the evidence. And here's the phenomenon that they were looking at and here's what were interested in, which is it's one thing to say that a case had a mistaken identification in it or there was a false confession, but those are essentially what we would call “zero-one variables” — it either happened or it didn't happen.

What we're interested in as well is how strong would you say the evidence is that the prosecution had here, or that the police were able to develop, that would justify a conviction or not? And the Police Foundation, as I said, allowed us to borrow from their scale; we took it down from a five-point scale to a three-point scale and we applied it to all of our cases, both erroneous convictions and near misses. We also crafted a scale to look at the quality of the defense that was provided, because it is certainly the case that in many of these instances, whether someone ends up being wrongly convicted or not depends on the quality of the advocacy that the defendant had representing him.

So we have these variables that we coded. We also have the strength of the evidence scale. We put them through, at first, what's called a chi-square analysis, it's a simple quantitative analysis that's essentially — if any of you have ever seen what's called a two-by-two table, a variable is one or zero and then it's connected to another variable, one or zero. This is really to — except for maybe two studies, this is the extent to which quantitative methods have been applied previously to wrongful convictions.

We went from there to logistic regression, and then from there we use qualitative research. We brought together a group of experts — and here again we owe thanks to the APA and the NDAA and the NACDL for helping us to identify experts in police investigation, in prosecution, in forensics, in defense lawyering, in other researchers who have looked at these cases. And we brought them together over the course of two days and we gave them narratives from — I can't do the number off the top of my head, but I'm going to go with around 30 — and we had them look at these cases and help us to understand from their professional perspective, how it is that the cases went one way or the other and how strong the evidence was from their perspective. And meshing — what this is called — qualitative analysis with the regression analysis, we were able to say a whole lot more about why it is that the cases went one way versus the other.

And the first thing that we found is that many of these cases, the wrongful convictions and the near misses, had a number of factors in common. Now, when you hear “factors in common” you think, probably, “Oh, this is great news!” From our perspective, that's not necessarily great news. In other words, it's not absolutely bad news, but what it tells us is that these factors do not distinguish why an innocent defendant is convicted in one case or has his case dismissed or he is acquitted in the other. So false confessions were prevalent in roughly the same rate in our erroneous convictions and our near misses. In the wrongful convictions, false confessions were present in 22 percent of the cases; they were present in 29 percent of the near misses. Similarly, on eyewitness error, there was eyewitness error in 83 percent of our wrongful convictions and 75 percent of our near misses. That was not statistically significant. However, when we broke eyewitness error out into something we call “intentional eyewitness identification” versus “honest” or, simply, someone happened to make a mistake, then this variable did explain some of the difference, and we'll talk about that later.

Another thing that did not explain why an innocent defendant is acquitted or has his case dismissed in one situation or is convicted in another, was whether the crime that he was charged with was similar to his criminal history. So it is not the case that someone who, say, had a past rape conviction was going to be any more likely to be wrongly convicted or dismissed or thrown out of the system if the case in which he was being charged with going forward was a rape case. In addition, police and prosecutorial misconduct does not explain these two sets of cases, except for Brady violations, which I will explain a little bit later. And finally, perjury or snitch testimony doesn't distinguish these cases either.

So the way to understand these is these are prevalent, each of these factors is prevalent at more than just a small level in all of our cases, but they don't explain why a case goes one way or the other. And you remember in the beginning I was saying that the problem with some of the past research is that it can't distinguish those factors that are specific to wrongful convictions and those that may appear in other cases. These are those factors. Because what's happened in the past is, people have done case studies of wrongful convictions and say, “Oh look, false confessions pop up a lot in wrongful convictions cases,” and we say, “You're right — they do. But that doesn't necessarily mean they are a cause of why a case goes to a conviction versus not.” And we will talk near the end about how these do relate to why a defendant ends up in the criminal justice system, but they don't necessarily explain the result of the case.

Now I would distinguish these factors from those factors that don't appear much at all in these cases. So some of the things we've heard about in the past are things like forensic fraud. The Houston crime lab scandal — it happened, it exists, and it happens in some place, but nationally this is not that prevalent in the cases and frankly in our study does not explain wrongful convictions.

Race. Now when we have here on this slide, it doesn't mean that these are small numbers; rather, race — and this is a key point — race does not explain, in our study, wrongful convictions. It is not the case that one group of defendants on the basis of race is being wrongly convicted as opposed to another. To me that's good news, and that's something that comes out of our study.

Abuse to elicit confessions. If any of you are familiar with some of the cases that came out of Chicago Police Department where they had essentially beaten confessions in the past out of some defendants. That concept does not appear in many of the cases. Again, to me that is good news.

Last two things: Gang membership — not very prevalent — and, finally, whether the defense uses experts or not was something that we tested, and this doesn't appear very often. To me that's bad news, because not only — well, part of this I'm agnostic on, so part of it in terms of whether it explains the outcome, I need to be agnostic. But what we also found is that defense lawyers are not using experts that often in these cases. That does go to another finding that we have later about the quality of defense.

So I have what's not important. What is important? Now, don't worry, this is what we call the “trust me” slide. This is the slide that shows that in fact actually we ran our logistic regressions. So this, in a different form, appears in the report, please don't worry, I'm not going to require you to come up here and take a look at this. I have these results — just to let you know where we're going here — I have them three different ways. So I have them this way, I also have them that way. If this is too much math, I also have them that way. So we're going to back up and do the middle case. I'm not going to require you to get them this way.

So what we have here is we have 10 factors that the logistic regression shows to be predictive of a wrongful conviction versus a near miss. They are things including — and I'll explain these when we get to the next phase — death penalty culture, age of the defendant, strength of the defense case — I don't need to run through these; you can read them and they're on the next slide. What I want you to look at here is that the sliding scale here shows you how confident we are about — for those of you who know your statistics — the coefficients that we've come up with in the regression. But when you see a larger line here, it does not necessarily mean that we are less precise on the particular variable. Some of these variables are by variant, meaning we scored them zero-one, others are in a larger range. So, for example, the age of the defendant is on a much longer range and that's why there's a greater spread in terms of how confident we are about the coefficient. But this is the one I think we can focus on that helps us to understand what's going on here.

Now let me remind you, what we are talking about here are those factors that distinguish a wrongful conviction from a near miss or an acquittal at trial on the basis of factual innocence. Someone comes into the system, their case goes one way or the other. What explains it? I'm going to do the first one last because it's probably the most problematic, but let's start at the second one. So, being a younger defendant ends up being associated with a wrongful conviction as opposed to a near miss. Why is that? We hypothesized — well, actually we didn't hypothesize — looking at the cases, there are essentially two reasons for this: Number one, some of the defendants are not as sophisticated at a younger age as they are at an older age and were unable to be as helpful to their defense attorneys in being able to come up with alibis to explain what it is that they were doing at a particular time. Also, they're less likely to be employed, and because they are less likely to be employed, they don't have the kind of corroborating evidence that older defendants who are employed have — they have time cards. They clocked in to work at a particular time, and there is corroborating evidence that they could not have been at the crime scene because, “Look, here — I was at work.”

The next one down is any prior criminal record. Now you will recall a few minutes ago I said a criminal record that is similar to the crime being charged was not relevant. But here what it means is having any criminal record means that a defendant is more likely to be wrongly convicted. So being known to the police or having your photo in a mug shot book means that you are more likely — if you are innocent — to be wrongly convicted than to have a near miss.

Forensic errors are to be distinguished from forensic fraud. Forensic fraud is when a forensic examiner makes something up — that's not this factor. This is where the test is simply done wrong, or more likely the forensic analyst over-testifies to the conclusions. Over-testifying would be, for example, in hair analysis where the only thing that really could have been said is that the defendant's hair was consistent with the hair found at the crime scene, and in fact, the forensic scientist at trial testifies that this hair had to have come from the defendant.

Prosecution withholding exculpatory evidence. These are Brady violations. This one I think is fairly self-explanatory. If the prosecution has facts that show that the defendant is likely or potentially not involved in the case and the defense doesn't have that, it is harder to be able to prove the defendant's innocence.

Next one down: honest eyewitness mistakes. You will recall a few slides ago I said that when we looked at witness identification as a whole, it did not predict wrongful convictions from near misses. However, when we then break this down a little more closely and we compared what's called “honest eyewitness mistakes” versus “intentional eyewitness error,” honest eyewitness mistakes tended to lead to a wrongful conviction. And the way to understand this is that if an eyewitness is intentionally misidentifying the defendant, these cases are usually easier for police to be able to figure out, because they can figure out that the eyewitness had a motive to lie. When the eyewitness is simply making a mistake, “Look, I was there, this is what I saw.” I have no motive to lie; it's simply the usual problems with eyewitness identification: lighting, the ability to do cross-racial identifications, the emotional anxiety we have in the midst of a crime. All of those sorts of factors are those that are related to honest eyewitness errors and those are the ones that are associated with a wrongful conviction.

Weak evidence by the prosecution. This was where we really benefited from the Police Foundation scale. This is a variable that went completely against what we were predicting. So what this is saying is, in those cases where the prosecution's evidence is weaker — in other words, we look at these cases using the scale that the Police Foundation has helped us with, and the evidence is just not as strong in a certain group of cases. Those are the cases that are more likely to end in a wrongful conviction. We thought it would be the other way around. We thought that if the cases were not as strong, these would be the ones that would be kicked out of the system earlier. “Your case isn't that strong? Well, why are we going to trial?” or, “Why is this not being identified earlier?” But in fact, the weak cases are the ones that are more associated with a wrongful conviction. We'll talk about that in a moment.

The next one down, this one's intuitive I suspect to almost all of us: a weaker defense associated with wrongful conviction. Lying by a non-eyewitness, also associated with wrongful conviction. This includes two things: One of this is traditional snitch testimony that we're all familiar with, but these also include false rape cases. That can be in there as well.

Then finally the top one, which is death penalty culture or, as we were modeling it, state punitiveness. This is the one that we are least confident in, but what it suggests is, that in states that end up executing a larger percentage of the population than in other states, the cases are more likely to go to wrongful conviction than near miss.

Now this is what our quantitative evidence showed, this is what the regression showed. We took these findings and we went into the panel with our experts, so we went and did the qualitative analysis as well. And we came out of that with a better understanding of how these fit together, and what we ended up concluding is that really three of these factors work hand in glove. Now the three that I would highlight the most are the weak defense, the weak evidence by the prosecution, and the Brady violations, because together what they are telling is a story of tunnel vision, and it looks something like this.

In cases that have weaker prosecution facts, we have one or two facts that come in with a case, and police really have nothing else to go on, but they start down that path and they become more and more invested in that theory of the case, that they don't see other facts in the case, they don't consider as many other potential explanations. In so doing they may overlook exculpatory evidence or it's not turned over to the prosecution, and in turn that exculpatory evidence doesn't make its way to the defense. And the defense then cannot investigate those facts and in many cases doesn't investigate facts anyhow. And what we end up then with is a combination of systemic error. So one of the key findings to take away from all of this is that it's not any one of those individual facts by itself. Now these are all relevant, but they really act together, and the more of these that you have together in a case, the greater risk of a wrongful conviction. But again, I would highlight three of them as being representative of the phenomenon of tunnel vision, and those are the weak defense, the Brady violations, and the weak prosecution facts.

What this ends up then leading to is a question of how do we fit these together with theories of prior research, or findings of prior research, that have found things like false confessions related to erroneous convictions? And I think the way to understand this is essentially with a funnel. With those factors that explain how it is that someone enters the criminal justice system, and those factors that then explain once someone enters why the case goes one way versus the other.

So what we have at top here are three findings, not only from our work, but findings that are consistent with other work, that explain how it is that someone gets indicted in the first place when they're innocent. These three things that we have up here are false confessions, snitch testimony or tips, or criminal justice official error — police and prosecutorial misconduct or error. These are things that are associated with why someone who is innocent gets indicted. But once someone's indicted, these things do not explain why a case is going to go towards a wrongful conviction or why a case is going to go to a near miss. Different phenomena at play — those that get you into the system and those that get you out of the system, or those that have you then ending up being convicted.

Now one of the questions I've been asked by reporters from this study is, “Well, which of these things are you saying is most important? Where do we need to put our energy?” And my answer is — an answer of course that they don't find very satisfying and I suspect you won't either — my answer is, it depends. It depends on what we're more concerned about. Are we more concerned about preventing the innocent from being indicted, which takes us to the bottom of the funnel, or are we more concerned about preventing the innocent from being convicted once they are indicted? Because if what we're worried about is making sure that the innocent are not indicted, then false confessions and police and prosecutorial misconduct are things we ought to be spending our time on, and snitch testimony as well. But if we are concerned about preventing the erroneous conviction once someone has already been indicted, then those 10 factors that I showed you before — those 10 factors are key, but again I would focus this on three of those factors as being most operative together: defense lawyering, strength of the evidence that the prosecution has, and then the willingness of the police and prosecution to turn over exculpatory evidence.

I said at the beginning that I am particularly pleased that the International Association of Chiefs of Police has gotten involved in these issues. And in fact, we are seeing a number of police departments and prosecutors take interest in these issues and really try to prevent them, because 100 years after we begin to get evidence of wrongful convictions, where the scholarship begins, it is time that we start to implement measures to prevent these. And these are really issues of professionalism — police professionalism, prosecutorial professionalism. And I've said in an interview before and I'll repeat the story here again: Before I came to American University I taught at George Mason University for 12 years, and I was in a department that had an advisory committee, and the chair of that advisory committee was former Attorney General Ed Meese. Now for those folks who are involved in the Innocence Movement, people might be surprised to find that Ed Meese was actually quite supportive of some of the work that I was doing investigating erroneous convictions. And the reason was — and I had this conversation with him many times — is he said, it comes down to police and prosecutorial professionalism. Learning from error and preventing that error in the future is part of what it means to be in a professional agency. And for any of you who've heard me speak before, you know that one of the things that I am most concerned about is that in the criminal justice system we largely lack what is found in other professions.

So for example, if you were on an airplane that crashes, within roughly an hour or two, the National Transportation Safety Board will have a team of investigators in the air, on its way to the site to do an investigation to understand what went wrong, to create a report with recommendations to prevent those kind of errors in the future. If any of you have relatives who have died unfortunately in a hospital, particularly a teaching hospital, you may find that after that fact, doctors convene what's called an M&M conference to go over the case, to figure out what happened — not to point fingers, but to learn what went wrong to prevent those errors from happening again. But in the criminal justice system, where the stakes quite literally are life and death at times, we seem to be scared of that kind of review to figure out what went wrong and to prevent these things from happening in the future.

And prevention is going to have to require all of us from across the spectrum: police, prosecutors, the defense community, forensic scientists. And we had a number of those people together in our expert panel and we asked them, as part of that discussion, what kinds of things do they recommend to prevent wrongful convictions? And a number of these things that we have up here are things that they recommended to us, but these are best practices that are being used in a number of police agencies and a number of prosecutor's offices today. And many of these don't cost that much, and particularly in a time where we are in limited resources, any kind of reform I'm well aware is going to only be effective if we are not talking about just opening up purse strings, and a number of these things can do it.

So what kinds of things did they recommend? Number one, they talked about having checklists for investigations. And these are checklists that they recommended not simply for police and prosecutors, but also for defense lawyers, particularly when their defendant has an alibi. To run through a checklist of the kinds of things that need to be investigated and to be figured out before a case goes forward, to make sure you looked into, “Hmm, the defendant says that he was at work at that time. Well, number one, did he have a job? Number two, where was it? Number three, can anyone vouch for the fact that he was there at that time?” If you can do that, you can stop some investigations earlier than others.

Second one, and this is something I would have never thought of before, but we had a very, very good forensic scientist with us on our expert panel, and one of the things that this person said was, when DNA technology first came on the scene, it was being used often later in an investigation. So police officers would have someone already in mind and they would begin to run down the case, and then later on the forensic testing would come back. Certainly before DNA testing, there were questions as to whether those findings were as solid, if you will, as the other facts that the police had already acquired. If they come in later in a case, police and prosecutors may already be wedded to their theory of the case and would be more likely to discount those findings.

Now, with DNA evidence available, it is essential that that testing be done earlier in a case because certain people can be weeded out from the beginning. And I recognize that that testing is expensive, but the testing is likely going to be used anyhow in these cases, so you might as well do it earlier, kick people out who are innocent earlier, and thus forgo additional time of police, prosecutors, and defense lawyers in investing a case.

A number of prosecutors and police departments are right now experimenting with putting a prosecutor at a police department to help evaluate evidence as it's coming in, essentially to be a devil's advocate to some extent, but to make sure that the prosecutor is satisfied with the quality of the evidence that he or she has so that cases that are not as solid can either be fixed earlier or the case can be not charged. Similarly, having senior prosecutors assigned for intake or charging means that evidence that might get past a more junior colleague will not get past those prosecutors who have more familiarity with what things count and what things don't in terms of the kinds of evidence they'd bring to trial.

Certain prosecutors' offices have experimented with open files discovery; I realize that this is an issue that some prosecutors have concerns with, but in those offices that have used it, it almost immediately does away with Brady objections from the defense. But it also raises a concern that not enough defense lawyers, when they have open files discovery, are actually going and investigating the full file. And just as I said earlier, one of the sources of erroneous convictions was poor defense lawyering. This is an area where if the prosector's going to have open files discovery, defense lawyers need to be trained — just as we had checklists for investigations, a checklist for going through the file to make sure that the defense has all of the facts that could potentially be exculpatory and then those that are not as well.

This notion of a threshold for lineup. The idea here — and actually this idea came from one of the police officers, it was a sergeant who was on our panel — who said, if you are going to put a suspect in a lineup, to some extent what you are doing is you are putting that person at jeopardy that he is going to be wrongly identified by the witness, without the witness having any, oftentimes, motive to do anything other than pick who he or she thinks is the correct suspect. But if you're going to do that, essentially if you're going to put a suspect, as the sergeant said, in jeopardy of the mistaken identification, then there needs to be some threshold of evidence that the police could even show to a court for why it is that that person was someone that they wanted to put in the lineup.

Finally, I talked about post-error review. This is the one that, as you can tell, is my hobbyhorse issue. We need to be willing to look at these cases together after the fact to learn what went wrong to make sure that these problems do not happen again in the future. Then, finally, this is an idea that has come out of some police departments. This is a tricky one because how you immunize the reports for 1983 actions becomes a little tricky, but if what you want is to identify error, to prevent it, then you need to make it easier for officers or prosecutors to be willing to come forward and say, “You know, I think there was a mistake here. It's something that we ought to be taking a moment to step back and learn from and see what we ought to do about it.”

So that is the study essentially top to bottom. I'm going to look forward to hearing what John Firman has to say about this, and also to hear about the important work that the IACP is doing. This is something that I think we all ought to be applauding that the police community is working on, and then after that I know John and I will be very interested in taking your questions and we'll see where the two of us agree or disagree on these issues. So thank you very much and, John, it's all yours.

[Applause]

John Firman: Good morning. First of all, you all note that Jon hails from Harvard and I hail from La Salle College in Philadelphia, a very small Philadelphia school. It's hard to level that playing field, but I do want to note just as a housekeeping that if you look at the NCAA men's tournament, you'll notice that the folks from Harvard have gone home and my boys from La Salle will be playing this week. It's my best I can do.

The other housekeeping point I would start with is that my wife this morning said, “What are you doing at NIJ?” and I said, “I'm going to react to Professor Gould's excellent work on wrongful conviction.” And she said, “That could be difficult for you.” And I said, “Why?” And she said, “Because you typically overreact to everything.” She said, “Maybe you could, like, knock it down a notch and just talk to them about this.” So I'll try, in honor of her and Jon, I'll try and do that.

My other difficulty is, as you all know or most of you know is, I represent the International Association of Chiefs of Police. We have 22,000 members in 100 countries, so I have to speak for them. So I am — and obviously this is a practitioner response to the work that Jon has done, because I represent 22,000 chiefs, and in fact I must do that well. That's not an easy thing, because I don't know what they all think, and very often they think differently around the country on things, so it's a difficult job, but one I enjoy immensely.

The other thing is that we talked about — and Jon, you mentioned — that this work has been done, the 1913 work, but the IACP is driven by committee, so I want to just mention three committees right now. We have an investigative operations committee chaired by Pete Modafferi, in Rockland County District Attorney's Office, and investigations operations — their job is to say, what are the best practices in investigations. We have a forensic science committee chaired by Stephanie Stoiloff at Miami-Dade County Police Department. Their job is to say, how are we applying and leveraging forensic science when we go at these issues, particularly wrongful conviction. And finally we have a professional standards, image, and ethics committee chaired by Chief Ron McBride. Those committees have been together for seven, eight years, working hard on this issue of wrongful convictions. So I want to make it clear and be representative that they are the leaders in this, at least from an IACP perspective, for the last seven or eight years.

I think the best approach today, very simply, is to do what Jon suggest and that's — we ought to compare notes here. The IACP has been looking at this, we just had a national summit, and so I want to thank, by the way, obviously Director Ridgeway and the staff of NIJ, Eric Martin, thank you. We've had a lot of great counsel and advice from NIJ as we ran out our summit on this topic, so I want to certainly thank them. But I think it's good to compare some notes and look at the themes from the summit. I'll talk about the summit by the way. Jon has research; I think it's probably many, many, many pages of research I read. I didn't count them, but a lot of pages. I have a summit report, and by the way this is just to prove that we actually have a direct report. I can't release it yet. Our summit report is the result of bringing 75 subject matter experts together, throwing them in a room, in a very intense environment for one day and saying — this ranges from people like Barry Scheck, who was with us, to Ed Davis, the Commissioner of Police in Boston, victims, advocates, scientists, researchers, prosecutors, and saying, “Okay you've been at this for a long time. So you've got one day to put it all together and come out with some serious bottom-line recommendation and say, if we want to fix the problem or address it more aggressively, what's the best way to do that?” So a summit experience is a very kind of weird one, but also a very intense one. I think it's good at this point just to simply say, we did that, so what did they say and how does it compare to what Jon has learned through his research.

By the way, I will mention, back to my membership, I got a call about your research just so you know. Ron McBride, the chair of our image and ethics committee, called me a few days ago and said, “Have you read Gould's work?” And I said, “Why yes I have.” He said, “I'm having some very strong reactions to Professor Gould's work.” I said, “Okay what are those reactions?” He said, “He's right on.” He said, “He's picking topics, issues — his issues of how these things can go wrong and how they can be fixed,” he said, “they align perfectly with what IACP thinks and what we think.” He said, “So congratulate him when you see him.” That's from Chief McBride.

What were the broad themes of the summit? First of all, no surprise, everybody agreed it is a system problem. This isn't one person's problem, it's not one group's problem, it's not somebody's fault, it's the problem and the fault of an entire system and a whole bunch of people. In the beginning of the summit, people also said that most things, most investigations, most cases, most witness ID, start with good intention, the good intention of getting the bad guy and putting them behind bars, the real criminal, the real person who did the crime. The problem is that that often turns, and we know, from Innocence Project data, that turns to, in many cases, in certainly some number of cases, massive damage. We've had panels at IACP over the last two years, we've had panels talking about wrongful convictions, and we have the victim, we have the exoneree, the prosecutor, the chief, all of them up on the same panel. And to see that pain in that room, and to see that difficulty, and see the damage done, it's enough to make you certainly drive to fix this.

So what did the summit folks talk about? The first thing they went to was learning from research. What Jon has done, the IACP and our membership must learn from. What the Innocence Project and Barry Scheck and his staff have done, we must learn from that. There's no question that we're almost talking, well we are, we're talking evidence-based policing here in its broadest terms. We need to learn from these things if we're going to change the way we do business.

Summit participants were straight up front, immediately about the pressurized investigative environment. This doesn't track totally with what you've done, but I think it's right there. The folks doing police investigations work under an incredibly pressurized environment: governing body, the media, citizens, supervisors, the chief himself or herself. So clearly that was an issue that was raised immediately.

Funding and resources. Many things you'd like to fix — whether it's training, whether it's getting more DNA evidence, whether it's getting it faster — a lot of this stuff takes a lot of money, so certainly money, dollars, and resources were discussed generally, policies and training.

Organizational culture. We look at police organizational culture all the time. We are fairly positive about that culture, we think it's a heck of a good culture for professional law enforcement doing a good job out there protecting American citizens. However, there are parts of any culture that need to be taken a look at, and the thing that Jon mentions I go to is openness to new information. If you have invested in your general direction of a case, then the information coming over here from another direction has to be just as interesting to you, just as important to you as that that you've been working on over here. So I think openness to new information certainly was right away what our summit folks talked about.

Preventing and correcting. There was also a clear drive at the summit to say, “Let's prevent as much as we can, but let us go back and correct when we need to.” The last thing and fairly obvious was, the justice system begins with law enforcement. I know one statistic: 100 percent of everybody arrested and investigated, arrested and charged with a crime, are arrested by the police. So law enforcement is in the position to be at the very front of the continuum, so responsibility rests with law enforcement, certainly to start that process. And if they have started that process incorrectly, erroneously for whatever reason, it's going to process through the rest of the system in a troublesome way. So law enforcement's front-end job was certainly there.

So what are the solutions and how do they tie to Jon's checklist? I think you'll find this to be hopefully some very good news. Communication. Who's talking to who, putting prosecutors together with the police in a more aggressive fashion. Are we doing that? Reexamining the investigative culture. If we're saying to a detective, “Get your guy, get him fast, don't let anybody influence you, get the job done,” that's a very different message than saying, “You need to work in a team environment, taking in all kinds of information and having checklists and readdressing and reinvestigating and reassessing what you're doing.” Those messages are critical and that's certainly going to appear in our summit report. So in essence we're talking about collective ownerships of cases. It's not just my case, it's the department's case, it's the chief's case, it's the supervisor's case, it's the forensic folks' case, it's everybody's case.

Now, how did we do when we presented the arrestee to the prosecutor, how good a job did we do? Post-arrest case review — and I love this one. You talk about risk assessment checklists and we're going to go right there. As a matter of fact, one of the recommendations that I'll preview here that will come out of our summit will be a wrongful arrest assessment tool, where literally the team will sit down — and that team will be an expanded team — and the team will sit down and say, “Let us go back and look for any potential red flags, where we can say that even though we've hung our hat on this case and we think this is the right person, this is the right offender, then the bottom line is, let's go back, let's stop, let's just stop everything, get the checklist out — just like pilots do, just like doctors do — and say, ‘What do we have here? What does this case really look like?'” So I saw that as a direct link to what Jon has done.

Investigative protocols and training. Police live by good policy. The IACP has a National Policy Center that puts out I think some very good policies. And then you have to train on those policies, so police do well when they have good policies and they follow those policies because they were trained to do so. So clearly our summit report will talk about those issues and how important and how critical they are. And then supervision. We've had round tables at IACP, focus groups with investigators and chiefs, talking about critical major crime investigations. And one thing there can never be is a gap between where the officers investigating are coming from, how the supervisor's involved in that case, and how the chief is involved in that case. No one of those three important levels of policing can say, “Well, I'm going to go hands-off here. I'm just going to let them do what they're doing.” This needs to be a collaborative effort, so we're going to talk about sort of the family approach to how we investigate things.

Eyewitness ID. I don't think there's anything left to be said about that. There's so much research, so much excellent research done by Dr. Wellford at Maryland and a lot of other folks that talk about double blind, sequential. So one of the messages I think that'll come out and emerge when our summit report hits the street is to say, take that good research again, and go ahead and put it in place. If you're holding off changing your policies because you're waiting for more research — and I agree, by the way, with Jon that while a lot of this stuff has been laboratory research and we need to and the summit will call for additional research in the field with police departments — that's fine, but don't wait now, don't wait around for more research to come. When we know certain things seem impeccably logical, let's go ahead and do this.

Informant testimony. Again, just as Jon found, an issue to be looked at. Forensic science. Evidence and analysis. How badly, how much do police need the best forensic analysis and the best forensic science, they do. Are there issues? Yes. And are those issues resource-driven in some cases? Yes they are, but nonetheless we have to push for that.

I'll give you what I call the overarching themes of the summit, and again you'll see echoes and linkages to what Jon has done. Openness to new information. These are the things that everybody walked away saying we all agree on. Openness to new information. A culture of investigative quality versus speed. Rightful arrest. Avoiding errors. Detecting and correcting errors. Wrongful arrests. Terminology is — by the way, we call it the “wrongful conviction summit.” You'll notice our language in the summit when it's released is always going to say “wrongful arrest, prosecution and conviction.” We named this problem, years and years ago, from 1913 on, we named it wrongful conviction, but obviously it starts at the front end, so talking about doing that. Maximizing the value of forensic science and opening the doors widely to reexamining closed cases as well.

So let me wrap up here because we wanted to leave some time, and I hope we did, for questions. I'm going to summarize here. Jon and the IACP's work, in my view and I think the view of a lot of people and obviously Chief McBride, our ethics committee chair, we're on the same road, and we went there from very different ways in terms of the research and how we did it, but we're on the same road. I think Jon's work is echoed at the summit from what you've just heard and what I've shared with you. We are talking again about evidence-based policing, the need for law enforcement. We have 22,000 members, we have 18,000 agencies. Is every one of those agencies waking up this morning and saying, “I'm going to be influenced by and change my policies and my direction and my organization based on the best research in the world coming out of GMU or Jon's work at American or maybe La Salle too.” But yes, the answer is — we know the answer: No. The IACP's job is to reach as many of those agencies as we can and say, “Here's a message that you can learn from and use.” So the bottom line again, that everybody has a critical role, but law enforcement's at the front. I would leave you with this, it's a quote from — I wasn't allowed to release anything today too much because, by the way, the 85 summit participants — all of them, including Barry Scheck and Ed Davis, the Commissioner in Boston — all get to review this draft before we can release it to the public. But having said that, I will say this, because I think everybody would agree. And I quote: “Positioned at the front end of the justice continuum, law enforcement identifies, interviews, arrests and charges the defendants in all crimes. How appropriate did law enforcement in its natural leadership role in every American community take a lead role in stopping wrongful conviction before it happens, by eliminating the arrest of the wrong person?” That's all for me. Thank you very much. Glad to be here. I guess we're going to go to questions.

[Applause]

Ridgeway: We have some time for questions. We have three microphones on the aisles. We have probably about 10, 15 minutes for questions, so line up.

Marjorie Zatz: Hi, I'm Marjorie Zatz. I'm a Program Officer at the National Science Foundation and professor at Arizona State University. Thank you, this was really wonderful. I appreciate it very much, it's really important work, and congratulations to the team for doing such good work. I have a question going back to an earlier part of the presentation where you didn't find any race effects. I just wanted to understand that a little bit better, so it's a two-part question. One is, how much variation was there by race, and did you have data on the victim/defendant racial dyad because that emerges in death penalty work as so critical?

Gould: Right. So, I do not have with me right now the breakdown, but you're correct — there was not tremendous variation. It was substantially minority population. And we did have data on the race of the defendant and the victim, and there was no effect there in terms of explaining why a case went wrongful conviction or near miss.

Dave McClure: Hi, Dave McClure, Urban Institute. First, I wanted to say I think it's really great that you're doing this research, because I know how challenging it is and this is definitely a huge step forward. But I was hoping that you might be able to talk about something. You addressed it a little bit in your case selection criteria, but the processes you went through in the near misses and how you had this convincing evidence among the researchers that helped you distinguish between cases that were innocent, factually innocent people who were almost convicted versus potentially guilty people who managed to not be convicted.

Gould: So the near misses — so there's the two pieces, the one is the official statement of innocence, and then there is the facts that would convince us that someone was innocent. In terms of the statement, it was typically either a judge saying as the person's being dismissed, “I'm sorry that you had to go through this; it is clear that the facts do not show that you did this crime.” Or the person was dismissed and then the prosecutor in explaining why he's not going to refile charges says, essentially, “We have the wrong person.” In terms of the facts that would convince us in addition to that, that the person — well actually it wasn't even in addition, the facts generally that would lead to the official statement would also be the facts that would convince us. So things such as, well, a DNA test comes back and shows that he didn't do it, there are multiple witnesses who say this isn't the guy, including sometimes — we had one case where the victim kept telling the police department, “This isn't the guy. This is not the guy who raped me.” And finally the victim went to a newspaper and told the reporters, “This guy's being prosecuted or the case is going forward and I'm telling them over and over again it's not the guy.” So things like that were the kinds of evidence that would convince us, just as it convinced prosecutors or judges or jurors that this was not the right person.

Ted Collins: Hi, Ted Collins from the Center of Global Development. Thank you for your work, it's fascinating and I enjoyed it a lot. I just had a quick question. I work with data a lot, often because I'm a research assistant, so I had a question about the finding of the three factors, the three independent variables that worked in tandem, the withholding exculpatory evidence, poor defense, poor evidence on the part of the prosecution. I was wondering if you went back and tested that hypothesis quantitatively?

Gould: No, we didn't. So the way I think we understand this is each of these 10 factors on its own predicts wrongful conviction versus near miss. The more of these that are present in any one case, the more likely it is the innocent person is going to be wrongly convicted. When I talk about we then focused on those three, that came out of the qualitative analysis, but you're correct, we did not go back and then retest it that way. So if I am being absolutely precise, I say that each of these 10 variables had an effect like that on the likelihood of a wrongful conviction, but that if you understand these in terms of — if you interpret these in the way they are understood by police, prosecutors, forensic scientists, and the like who are involved in these cases, they would seem them as certain ones of these are more powerful than others in terms of their everyday actions.

Shawn Armbrust: Hi, I'm Shawn Armbrust with the Mid-Atlantic Innocence Project, and my question relates to kind of forensic errors versus misconduct and prosecution and police errors versus misconduct. I'm wondering how you distinguished between error and misconduct with respect to those areas because it can be kind of fuzzy.

Gould: We used the definition of intentionality. So when someone was intentionally trying to fabricate evidence, that was misconduct as opposed to simply error.

Armbrust: Can I ask a quick follow-up? So, with respect to forensics, one of the things you mentioned specifically was kind of over-testifying and there have been some cases — I know the Michael Malone cases for example, out of D.C. — where there's a case to be made that he knew he was over-testifying. So where did you draw that line?

Gould: That would have been an error, not a misconduct case. But you're right, reasonable people could disagree as to how intentional that was, but we had to draw the line somewhere, and so that was an error, not a misconduct.

Brent Berkley: Good morning, my name is Brent Berkley. I'm a Program Director with the National District Attorneys Association here. I don't know if I'm the only prosecutor in the room, but probably one of the few, and I know I've got several of my law enforcement brothers and sisters here. I want to thank them for all of their work. One of the questions that I have on this and all of the research that's done on this are, how do you look at sort of the proportionality of the wrongful or erroneous convictions that you talk about? Because if you take your 260 number, even if you double it, triple it, quadruple it, over the course of the 10 million felonies that are filed every year or half to one million violent felonies that are filed each year and you look at a percentage of the cases — and I don't want to in any way say that one innocent person is okay and we're looking at percentages here, but you're looking at 99 point, you know, six or seven decimal points of accuracy. And in a human system, it's hard to get a human system that works to that level of accuracy, and not in any way to discount the horrendous impact of a wrongful conviction, but I hope in some way your research is indicating that there is, we get it right so often because it goes through the responding officer, their supervisor, the sheriff, the detectives, the jury, the judges and everything else, there's so many different places where it's questioned before we get to that conviction rate. But in addition, if you could respond to that if that's included anywhere in your study. The question I have of the near miss, how are you identifying or defining “near miss” as opposed to the system working as it should? You get a prosecutor dismissing a case as it gets forward and realized, “Wait a minute — we have the wrong person.” I, as a prosecutor, did that multiple times because no prosecutor that I have ever met wants an innocent person in prison. And likewise, a jury finding somebody not guilty, that to me sounds less like a near miss and more like the system working as it should. If you could just respond to that.

Gould: Sure, let's do your second question first. I think this to some extent speaks to what John was talking about as well. There are any number of points in the system where we could have done a cut point and said, “How is it that police officers figure out when to arrest someone or not? How is it that prosecutors decide when to indict or not?” And I agree with you that police and prosecutors are those most likely every single day to screen out the innocent so that they don't get arrested and that they don't get convicted. So I certainly do not want to have this research seen as pointing fingers at some group as being bad versus good.

Your other question, which to some extent goes to the issue of how prevalent are erroneous convictions, and there has been, as you know, a great deal of debate about how often these occur. Our study did not get into that, but I will say that there have been a number of estimates from as little as less than a percent to as high, nationally, of 5 percent. The National Institute of Justice sponsored research by the Urban Institute that looked at Virginia cases and that found that in sexual assault cases, that the rate of mistake was as high as 8 percent, around that. So there's certainly enough research out there to suggest that these cases are not simply such a rarity that they do not warrant our attention. The way I would answer you is twofold: You're right on one fact, which is that every day, police, prosecutors and defense lawyers get this right much more often than they don't, but at the same time, airlines get planes “right,” if you will, many, many, many times a day. Planes don't crash usually, commercial aircraft. That's why when one does we spend so much time and attention looking at it. But if the error rate there is tiny and we have an entire agency, the NTSB, that's job is to try to figure out what went wrong and prevent those future errors, why in the criminal justice system, where we know the error rate is higher, do we not spend the kind of attention that the IACP is now doing to look at these kinds of cases?

Firman: Yeah, that's a great question, by the way. You just raised the question that is most of concern to the IACP as well. That is this ratio, there's a ratio of rightful convictions, rightful arrests to wrongful, and if we ran the numbers and we take the thousands and thousands and millions of cases against the numbers that we know we've documented, the ratio would be zero point zero something one something, I don't know what it would be, but it would be so low that as a Director of Research at IACP, what I really should do is say, “Well, we can't waste time on that because we got to go with something bigger.” But what I'd like to call your attention to is two things: police use of force. A decade ago at NIJ we studied police use of force and we looked at the ratio, and it was a critical study, an important study because of the public and media concerns. We found that the rate of police use of force, over 10,000 calls for service, the rate of use of force was four — that's zero point zero something-or-other percent. But the bottom line was, any use of force that would catch the attention of the public and be of concern, it was worthy of looking at. Police officer–involved domestic violence. We've been for years working with the Office of Violence Against Women, the send-out model policies and programs on police officer–involved domestic violence, when a cop goes home and commits a crime. The ratio, right, the ratio is miniscule. So the answer sounds a little flip, but we really don't care in many senses. The IACP takes this very seriously because the bottom line here is your goal is zero tolerance. And police officer–involved domestic violence, inappropriate use of force, or a wrongful conviction, the goal is a zero-tolerance goal. Never going to be achieved, realistically, but that's the goal. So I think that's the answer, that you're right, that in context, numbers are small, but in terms of damage and issue, the number is large.

Lyndon Diaz: My name is Lyndon Diaz with the Bureau of Justice Statistics. Did you look at type of defense counsel, public defenders versus dream team or pro se — that's one question — and did you also look at the type of offense?

Gould: Yes. So the answer to both of those is yes. And what's interesting — and actually, how appropriate to be talking it about a week after the 50th anniversary of Gideon v. Wainwright — is that when we talk about weak defense work here, it does not turn on the type of lawyer. So whether someone is represented by a public defender or by private counsel, that does not affect whether an innocent defendant is going to be wrongly convicted. Similarly, it didn't turn on type of crime. Again, remember, we're looking only at violent felonies here, and there is a strong argument to be made that there are a whole other class of cases out there that are worthy of attention. Why violent felonies? Because those are the ones where it's easiest to get the case facts to be able to do the analysis. But no, these cases did not turn on whether it was a murder or a rape or a kidnapping or a robbery.

[Applause]