Law enforcement

Ellen Scrivner: I'm Ellen Scrivner, Deputy Director of NIJ, and I welcome you to this session on behalf of our Assistant Attorney General, Laurie Robinson, and also our acting director at NIJ, Kris Rose.

This session, “State of the Field: A New Professionalism,” I think you're going to find thought-provoking, with lots of information to really start a good discussion.

You're going to have the opportunity to hear about two papers that will soon be presented to the field as outcomes of the second Executive Session on Policing that is being conducted by the JFK School at Harvard and funded by NIJ.

Today's presentations will be delivered by two well-respected academics, and they will be followed by comments from a highly regarded and progressive police chief and your moderator, all of whom have participated in the Executive Session.

I think you're going to find it interesting that through our panel today, you're going to hear a resurgence of what some see as a rather old and discounted term, the concept of “professionalism.” Just to clarify, these papers are not addressing professionalization or professional crime-fighting concepts that were developed in the '60s and '70s in response to departments that were rife with political patronage and corruption. Rather, they're responding to a larger issue, and that is, what is the next big idea in policing.

Our presenters are not looking, however, for just the right buzz word. Rather, they are taking an in-depth look at all that has gone before, and they're going to capitalize on those traditions to create more of a new frontier.

I think the question, then, could be asked, well, just what are we talking about. Are we talking about models, paradigms or yet another reform, such as problem solving, community policing, intelligence-led policing, evidence-based policing, predictive policing, smart policing? Are we talking about any of that?

All in this room have seen reforms come and go, and they have been labeled in numerous ways, but always they were presented as quite distant from the professionalized crime-fighting approaches of yesteryear. There were strong attempts to make sure that there was no association with that.

With each reform it has been tempting to believe that this is the answer, kind of the “aha” moment; however, realistically, it probably cannot find an answer that will not change in the next several years as the world changes around us. And as policing is influenced by those changes, our approaches and our strategies begin to change as well. So it's not a case of people just casting around really for the next big idea. It's looking for what is the idea that builds on all of the things that have gone before us.

So, in terms of today's paper, I'm not sure that we're just talking about a new response. After hearing the presentations, I believe that you will agree that these are exciting and cutting-edge papers that present some fundamental changes in terms of a different conceptual framework that links the two papers that are going to be presented, and that conceptual framework is professionalism. They represent a different way of thinking about where policing has been, where it is going and the organizational focus that will be needed to sustain this framework. I think you'll also agree that both papers very much represent where the field is right now.

Following the presentations, Chief Ron Davis from East Palo Alto will present his reflections on how these ideas will play out on the ground. So, in the tradition of NIJ, we are combining the research and practice orientation in this particular presentation.

After you hear the presentations and Ron Davis' comments, we then will take questions from the audience, and I will ask you to hold your questions until then.

So, at this point, I would like to introduce our panel. Our first presenter will be Professor Chris Stone. Christopher Stone is the Guggenheim Professor of the Practice of Criminal Justice, and he is the faculty chair of the program in Criminal Justice Policy and Management at Harvard University's Kennedy School of Government. From 1994 to 2004, Chris served as director of the Vera Institute of Justice, having joined the Institute in 1986 as head of its London office. In 2006, Chris was awarded an Honorary Order of the British Empire. I believe they call that the “OBE,” and not too many people in this country have that. He was awarded that for his contributions to criminal justice reform in the United Kingdom. He received his bachelor's degree from Harvard, a master's degree in criminology from the University of Cambridge, and his juris doctorate from the Yale Law School.

Following Chris, you're going to hear from Professor David Sklansky who joined the Boalt faculty at Berkeley in 2005, following a decade at UCLA School of Law where he won the campus-wide Distinguished Teaching Award and was twice voted the law school's Professor of the Year. He teaches courses on criminal law, criminal procedure and evidence. And David serves as faculty chair of the Berkeley Center for Criminal Justice. After graduating from Harvard Law School in 1984, David clerked for Judge Abner Mikva of the U.S. Court of Appeals for the D.C. Circuit and for U.S. Supreme Court Justice Harry Blackmun. He also briefly practiced labor law at Washington, D.C.'s firm Bredhoff & Kaiser from 1987 to 1994, and he served as an Assistant U.S. Attorney in Los Angeles where he specialized in white collar fraud prosecutions. While at UCLA, he served as special counsel to the independent review panel appointed to investigate the Los Angeles Police Department's Rampart Division scandal. David is the author of Democracy and the Police,Stanford University Press, 2008, and it's a well-regarded evidence casebook, Evidence: Cases, Commentary, and Problems. He has written extensively about criminal procedure and policing.

And our third panelist who will be the commentator on the papers that Chris and David are going to present is Ron Davis, who was appointed chief of police for the City of East Palo Alto on May 31, 2005. Prior to his appointment, Chief Davis spent 19 years with the Oakland Police Department where he served in assignments, including the police academy director, criminal investigations commander, patrol commander and Inspector General for the department. Chief Davis is a former member of two federal monitoring teams responsible for oversight of police reform agreements between the U.S. Department of Justice in Washington, D.C., as well as the Detroit police departments. He serves as a police reform expert for DOJ and has testified at the United States Senate Judiciary Committee hearings on racial profiling and the United States Congressional Black Caucus hearings on police misconduct. Ron is the author of the article, Bias-Based Policing and co author of the recent U.S. Department of Justice publication, How to Correctly Collect and Analyze Racial Profiling Data: Your Reputation Depends on It!, and he is a contributing author to the recent Police Executive Research Forum publication, Chief Concerns: The Use of Force. Chief Davis serves as a member of the prestigious Harvard University Executive Session on Policing, sponsored by NIJ, and he possesses a bachelor's degree from SIU in Illinois and is a graduate of the Senior Executive Program at Harvard University's JFK School of Government.

So join me in welcoming our panel because I think we are going to have a —

[Applause.]

Scrivner: And with that, I will ask Chris Stone to take the lead on our first paper presentation.

Christopher Stone: It's a pleasure to be here with all of you and my friends on the panel.

What I'd like to do is take a few minutes to share with you the outlines, the basic thrust of a paper that Jeremy Travis and I have been writing for the Executive Session on Policing and really invite your comments, your questions but, most importantly, your arguments with it. It's a set of concepts we are actively working on and hope will be useful.

We offer the new professionalism as a framework that we hope can help police chiefs, front line officers, communities understand what's going on in policing and as a way of keeping organizations focused or their own individual work focused or their relations with the police from the community focused on what's going on, on how it can be better, on what it's trying to achieve.

The use of frameworks is familiar in lots of fields, and for a long time in recent years, a general framework around community policing has been very useful in the police field, but that has, as Ellen suggested in the opening, been so added to over the years. We have so many concepts, ideas, priorities in policing that the cacophony is sometimes hard to deal with. And, as Jeremy and I were listening to conversations over the last several years among police chiefs, watching what front line officers do, working with a handful of departments on trying to improve work, we think we hear a few consistent themes that we have brought together under this organizing framework of the new professionalism. Those themes are accountability, legitimacy, innovation and coherence, national coherence.

So we think that we hear a movement in policing, defined by these four elements; that a commitment to accountability on behalf of police departments, a concern with legitimacy — and we'll talk a little bit more about what we mean by that — a commitment to innovation, going beyond the kinds of innovations we've seen in the last several — or several decades ago, and a new exploration and experimentation with and perhaps even a commitment to greater national coherence in the policing field.

Now, none of these is new. All of these are ideas that have been percolating, developing, gathering momentum over the last several years, maybe a decade or more. Indeed, we think all of these have some — can trace roots back to the beginning of community policing in the late 1970s, early 1980s, but we do think they're coming together in a new way, and I'll talk a little bit about that.

Let me say a little bit about why we think this is a new development. It's new. We call this the “new professionalism” for two different reasons. First, we want to contrast it with the professional model of policing that Ellen mentioned in her introductory remarks; that is, for a series of reasons that we'll talk about, but I think is unfortunate, the academic field, to some extent those who train and write about police performance in the United States, labeled the reforms of the 1950s and 1960s in the United States as the “professional model of policing.” And when community policing was first being discussed and debated in the 1980s and early 1990s, it was frequently contrasted with the professional model, and, indeed, you'll still hear very prominent chief executives in law enforcement today talk about how we still have to complete the work of getting away from the professional model and more into community policing.

Well, the idea that police should be leaving a professional model of policing is a curious — it's a curious framework. It's an odd language to use, and, in fact, many chiefs have told us that it leads to a lot of confusion. We want to be professionals. Why should we be leaving a professional model of policing? So the new professionalism is a way of trying to reclaim what's good in professionalism and try and help organize that thought but not confuse it with the professional model of policing at mid century.

It's also new in a second sense; that is, we think that each of these elements — the accountability, the support for the pursuit of public legitimacy, the kinds of innovation and the kinds of coherence we're seeing now — we think are taking on new forms. We think they're developing in new ways, and we think it's important to mark that development. To the extent that that new professionalism can be achieved in police departments, we think it would be a substantial achievement and a turning point in many ways in the history of American policing.

So let me start with the first of the — so the contrast, it's new as opposed to the contrast with the professional model of policing. Maybe, I hope, a few of you are at least as old as I am so that you can recognize “Adam 12,” a television show that came right in the center of the ambition of the professional model of policing, as it was called. And although it was described as professional — and we think it was probably — we think the term “professional model” was applied to the reforms of the 1950s and 1960s in policing because, in one sense, police — the conceptualizers, the people writing about this and talking about it, like George Kelling and Mark Moore and others, saw in that model both a good thing, that is, an effort to protect the police, distance the police from the manipulations of political machines, of the corruptions that could come from too much political control or just the informality of policing in the early decades of the 20th century. The achievement of more rigorous recruitment, more rigorous training, tighter supervision were all designed to make real improvements on the kind of policing that the Wickersham Commission and others had seen and criticized in the early decades of the century.

But it had a series of flaws, and Kelling and Moore and others wrote about those flaws and about the flaws of the professional model. What did they talk about? They talked about how the professional model was too removed from communities. The cocoon of the patrol car became an image of the distance that police began to take from their communities. It had a very limited array of tactics, basically a randomized motor — random patrol, rapid response and the retrospective investigation of crimes. It was a command-and-control model, a top-down disciplined notion of how police ought to work. You were in service, if not to your sergeant and your captain, at least to your dispatcher, but you were taking commands from above.

And perhaps the most telling criticism of the professional model came about the ability of this so-called “professionalism” to sit right alongside continued racial bias in policing. I think that in 1990 a really important paper as part of the first Executive Session of Policing was co authored by the two — to this day, the two leaders of the Police Foundation. There was actually a one-year leader, who will remain nameless, for the first year for the Police Foundation, but then Patrick Murphy led Police Foundation for over a decade in very important years, and Hubie Williams then took it over and continues to lead it today. Those two leaders of that foundation together wrote a paper about criticizing the professional model of policing because of its connection to racial bias.

I'm just going to quote one passage from their paper written in 1990. “The fact that the legal order” — and they're describing this 1950s, 1960s model of professional policing. “The fact that the legal order not only countenanced but sustained slavery, segregation and discrimination for most of our nation's history and the fact that the police were bound to uphold that order set a pattern for police behavior and attitudes towards minority communities that has persisted until the present day. That pattern includes the idea that minorities have fewer civil rights, that the task of the police is to keep them under control and that the police have little responsibility for protecting them from crime within their communities.”

Williams and Murphy went on in that paper to point out that into the 1960s, a third of American police departments still restricted the rights of black officers to make felony arrests. It talked about the disappearance of black officers from most major police departments from the time of 1900 to the 1950s and other facts that actually made them feel that calling the police in the 1950s and 1960s “professional” was to miss the falsity of that profession, at least as experienced by African Americans.

So it is clear that the community policing that was championed as an improvement on this professional model was, indeed, and I think continues to be an improvement on all these features. There's nothing, I think, wrong with the critique of the professional model, except that it called what it was replacing “professional,” and the new professionalism is in a sense trying to redefine what police mean and should mean by professionalism, not connected to these features. So it's new in a second sense as well.

So let me just try and elaborate a little bit on what we see as new in these features. So police have always been accountable since probably at least the 1930s and '40s on. Internal accountability in policing is part of that command discipline we talked about. But the accountability that we see as new is a willingness to talk externally, talk in the language of accountability and actually hold departments and hold themselves — chiefs and others holding themselves accountable in a public way.

When I started working on police accountability with police agencies and writing about it in the late 1980s, early 1990s, several academics, several of my friends, told me that was a great idea, but I should not use the word “accountability” because the police would leave the room if I talked about it as accountability. That was code for dealing with misconduct. They didn't want to talk about it, and you should use the word “integrity.” You should talk about lots of things, but do not talk about accountability.

Today in Chicago, there's a deputy superintendent for accountability. That's the title? Accountability is championed by police executives in speeches all over the world when Americans go abroad, American chiefs go abroad and train on modern democratic policing. They almost always talk about accountability as one of the features that marks a professional agency. Now part of that is simply because accountability as an element of good government has become much more common in the last 10, 20 years. It's now embraced much more widely.

But part of it, also, is a change in policing. Accountability is not just about conduct anymore. In fact, the three C's here refer to what we see when police agencies, when police leaders embrace accountability. The best of them talk about accountability for the three C's: for crime, accountability for crime and the problems associated with crime, accountability for cost and accountability for conduct.

We can talk about examples of how police departments are accounting for their costs in new ways in this last decade. It's pretty obvious, I think, the way they used to think of CompStat innovations across the country or other features in which departments are holding themselves and holding themselves publicly and internally accountable for crime. You see there are lots of examples of that.

But on conduct, I want to also emphasize that accountability for conduct is not just accountability for misconduct. There have been new advances there. You think of Bill Bratton's public confession of error just days after the MacArthur Park disturbances, the breakup, police breakup of immigrant rights rally in MacArthur Park in 2007, when he comes back and actually holds a press conference and says we were wrong, says there was excessive force used and takes strict discipline against the commanding officer on the scene. There are more of these public apologies for misconduct, but there's also, I think, a revolution in the accountability for controversial conduct that chiefs think is legal and legitimate.

Take a look, if you haven't seen it, at the annual firearms discharge report issued by the NYPD every year. It is a remarkable document in which the department outlines every time a gun was fired by a police officer, talks about why they think the patterns are justified, even when they are highly disproportionate in terms of the race and ethnicity of those fired upon, and they defend that through a series of statistical and factual arguments. But they also describe when there has been discipline taken, the officers removed from the force, even ones that didn't make the press at all. So I think the public accountability for conduct has reached a new level as well.

We talk about public legitimacy. When Jeremy and I were putting this paper together, I was scared of using the word “legitimacy.” It sounds very academic. It sounds a very obscure, abstract term. But I've been surprised, actually, at how easily police executives I've been talking to have picked up on this idea.

I mean, let me be clear. Legitimacy is important for any authority. Any element of the state or other authority needs to attend to its own legitimacy, needs to understand the sources of legitimate authority as opposed to illegitimate authority, and there are many of those sources. The professional model of policing in the 1950s and 1960s emphasized law and sort of professional expertise as the legitimacy of that police force, of the police service.

Community policing focused more on the consent and the connection with communities, often in community meetings, through community surveys, through engagement of active community participation in the work of community policing.

Jeremy and I argue that a new professionalism needs to be ecumenical about this. These are all important sources of legitimacy — the law, democratic politics, professional integrity and the public. But we focus particularly on public legitimacy and particularly within public legitimacy on the confidence of those most disaffected from society, the marginalized communities, whether those are marginalized and disaffected just because there are kids and they are in their teens and they're just disaffected as a general matter or if they're members of ethnic or racial minorities who are disaffected, perhaps from the society for other reasons. It is precisely in those communities that support for the police and confidence in the police is weakest.

The latest Gallup survey from 2009 shows that when they ask people about confidence in the police as an institution nationally, about 63 percent of white adults say they have confidence in the police as an institution, about 38 percent of black adults answer the question that way, and those patterns focus police departments on the legitimacy of the public.

We talk about innovation, not just doing a few new things. There's always been innovation in policing, but we think we see innovation going on not just occasionally, not just at the front lines as was often true with the problem-solving of the 1990s and early part of this century, this decade, but, indeed, whole, as Herman Goldstein hoped would happen, with Problem-Oriented Policing, whole management teams, senior-level executives innovating themselves at a department-wide level and, indeed, the pursuit of continuous learning in organizations.

Finally, we think we see trends in national coherence. Our country is particularly fractured in its policing. Nobody in the world would ever dream of creating a system where you had an uncountable number somewhere in the neighborhood of 20,000 separate police organizations and how we achieve national coherence, although there probably — I would say probably has to be some consolidation. We have to do something about the 80 percent of the departments that have 25 members or less.

We are not going to achieve coherence in this country through consolidating or the creation of a national police organization. We're only going to do it, if we do it at all, through professionalism and through developing a common set of practices, expertise, understandings that moves across departments, we hope with increased mobility.

Let me just say, finally, to set up the debate that Jeremy and I are having with David Sklansky about this in this session, let me just say a few closing words about the connection between this new professionalism and community policing because we do think that, while community policing is a part and has always been a part of the rise of this new professionalism, we think that it's important to see community policing as embraced by the new professionalism, but not the same as it, and that the new professionalism goes well beyond, I hope, what community policing is doing.

So let me just recall a debate that occurred during the first Executive Session. Ed Meese, then Attorney General, was a part of that session, and Mark Moore, my predecessor as the Guggenheim chair at Harvard, both in that session. And Mark had proposed this idea that the professional model had been fine for its time but was really in trouble now and community policing should really replace the professional model and, indeed, he said he thought was happening with the community model to find a whole new era. Ed Meese said that was a bit grandiose. He thought, yeah, there's this nice thing going on with community policing that's important, but it's really just a small part of what he called a “strategic policing.” That was really, he thought, what would mark the 1990s, and he said, you know, I'm happy to support this community thing as long as you just say it's an important contribution, but a lot of other things are going on, too, and you don't want to exaggerate the importance of the community part of it.

What's interesting is that Mark Moore did not argue with that. He agreed all those other things were going on, problem solving, strategies, work on crime and robbery, but he said, you know, if we call it “strategic policing,” the police won't do the hardest part of all, which is actually building strong relations with communities. So he wanted to call the new movement “community policing,” not because he thought it embraced all its elements but because he thought it would focus police agencies on the hardest bit, the bit that they would probably ignore if left to their own devices.

So that's what we got. We got community policing, and it worked, as a rhetorical matter. I think Mark was right, and it took off with a big help from Bill Clinton and the COPS office and a lot of money. “Community policing” became recognizable as a phrase to most Americans. It became recognizable as a federal funding stream, but it is not clear that you actually got the kind of community engagement that we hoped for.

I will just race through because I'm out of time here.

Some of things that the community policing did not turn out to actually have an answer to, it was not expected community policing would be accompanied by a huge increase in the volume of Americans being arrested by the police. The idea was that there'd be other tactics. In fact, we have seen a tripling of arrest rates per 100,000 at the same levels of crime. It did not anticipate a huge surge in stops as a stop and search. Even today, police chiefs who were the leaders of community policing turn to one another and say, “Oh. And, Chief, could we talk about what the role the community is supposed to be? Because I'm not sure anybody knows what the community is supposed to be doing in community policing.”

It's a bit late. It does not tell us much about how to police political dissent, whether you think of the recent political conventions of the last decade or other elements of it. It does not talk about how to balance privacy and the huge expansion of covert surveillance available to police agencies today. It does not talk about how to manage the difference between disrupting criminal operations or actually arresting and prosecuting criminal operations.

These are recognizable problems, I hope you understand, and community policing is a valuable thing, but the new professionalism, we hope is a way of addressing all of these and more. Community policing, at least in our view, a valuable, important thrust, but it has simply become too vague today to really guide operational decisions and answer the concerns that departments have.

In the end, I think Jeremy and I believe that community policing is a hugely important part of what we mean and we see in the new professionalism. But police need frameworks today that help them understand how they do use force, how they are making arrests, the appropriate use of stops, how to use this new surveillance technology, how to deal with new kinds of expectations of privacy. These are fundamental issues of our democracy, and the role of the police in this democracy is really at stake right now. Police agencies need to understand how, even when they are controlling sometimes forcefully elements of political dissent, they are still encouraging, not suppressing, our democratic traditions.

We believe that the new professionalism that we see is a way of engaging those questions, a way of guiding the answers. In short, we think the professional policing enhances democratic progress when it accounts for what it does, achieves public support, learns through innovation and transcends parochialism. That's what the four elements of the new professionalism are, and we think we see them in American policing today. We want to applaud them, and we want to see them continue.

Thanks very much for your attention.

[Applause.]

David Alan Sklansky: Thank you all for coming. I want to thank NIJ and Ellen Scrivner for organizing this panel and for including me in on it. I also want to thank NIJ and the Harvard Kennedy School for sponsoring the “Executive Session on Policing: A Public Safety,” out of which the paper that Chris Stone just presented has emerged.

This has been a really phenomenal set of meetings that I feel very privileged to have been a part of that's brought together leading police executives from around the country, including Chief Davis, with a few lucky academics who've got to sit in the room and learn and then mouth off.

And I want to thank Chris Stone and Jeremy Travis for writing this terrifically interesting and important paper that Chris just presented to you and for sharpening and helping to spur debate on some really critical issues.

This is going to be an odd kind of debate because I agree with 95 percent of what Chris and Jeremy say. I think that their focus on accountability, legitimacy and innovation is exactly correct. I think they're right to say that these are emphases that are emerging right now in progressive police organizations around the world and that deserve to be encouraged. And I think that the way in which Chris and Jeremy discuss accountability, legitimacy and innovation is perceptive and largely correct. I agree, for example, with the three fold way that they describe accountability in terms of crime, cost and conduct.

I'm not sure how I feel about the national coherence piece of the agenda, partly because I'm not sure I understand exactly what national coherence means. But, in any event, I don't think that national coherence is as important to what they say descriptively and prescriptively as accountability, legitimacy and innovation. And with all of that, I mean, all I have to contribute is applause.

I'd say, moreover, that I agree with Chris and Jeremy that there are aspects to the thinking about policing in the 1950s and 1960s that needs to be recovered to some extent, that has been given shorter shrift than it deserves. I agree, for example, that the idea of law as one way to get police legitimacy is something that was overemphasized, maybe in the '50s and '60s, but maybe hasn't been given as much attention as it should in the last couple of decades, and it'd be good to give it a little more attention.

So my point of departure is really all about what might be thought a matter of packaging, which is calling this agenda a “new professionalism.” It may just be a matter of packaging. I'm not sure about that, but if it's a matter of packaging, I think the packaging is important, not as important as what's inside the packaging, but important. So why don't I like his packaging of professionalism? Why don't I think it's a good idea to call these disparate ways in which reform police chiefs around the world are converging a new professionalism?

In large part because of the baggage; the concept of professionalism in policing has two kinds of baggage. One is historical, and the other is present day. So the historical baggage is the connection with policing in the 1950s and the 1960s. That kind of policing was called “professionalism” not in retrospect by people who were pushing community policing in the 1980s, and it's true that they used the term, but they didn't come up with it.

The term “professionalism” was the term that progressive police leaders used for most of the 20th century to describe what they were doing but particularly in the 1950s and the 1960s. In fact, if you looked around at reform efforts in policing in the 1960s, what you would see is that every police department in the country claimed to be doing professional policing, just like every police department in the country in the late 1980s and early 1990s claimed to be doing community policing. In the 1950s and 1960s into the 1970s, that professional policing is what police said they wanted to be. It's what they claimed to be. And because it was a term that was slapped on virtually anything that police departments around the country were proud of, it's a little hard to isolate exactly what was the core of the kind of policing in the 1950s and the 1960s that later fell into disrepute, but I would say that it had three elements.

One was a laser-like focus on crime suppression as the be-all and end-all of the police mission. The second was a strong emphasis on detached, objective scientific analysis of how the police should respond to crime. And the third was hierarchy, a top-down managerial style that stressed the advantages of centralization. And all of that fell into disrepute, as Chris says, in the early 1980s. It was blamed for making police departments insular, arrogant, resistant to outside criticism and almost clueless in responding to the social ferment of the late 1960s and early 1970s.

It was supplanted as the reigning orthodoxy of police reform by this term “community policing,” and just as in the 1950s and 1960s, everything that police departments were proud of was called “professionalism” and every police department claimed to be doing professional policing, in the late 1980s and early 1990s, every police department claimed to be doing community policing, so it's a little hard to put your finger on exactly what it is that community policing was supposed to be all about. But in large part, I would say it was defined against the parts of the old model that seemed to be causing problems. So that is to say that community policing self-consciously widened the scope of what the police were doing, so that it wasn't all about crime suppression. It very consciously moved away from an emphasis on detached objectives, scientific apolitical approaches to police policy, and emphasized instead that the selection and implementation of police policies should be guided by some participation with, partnership with and consultation with communities.

Nobody called the new kind of policing “anti-professional policing” because everybody understood that the word “professionalism” has some valences which are positive. Nonetheless, a lot of what community policing was about was moving away from parts of policing in the 1960s that I think it was a good thing to move away with. So that's one piece of the baggage that the term “professionalism” has.

The other piece of baggage it has is present day because it is true that community policing no longer has the cache as a buzz phrase that it did in the late 1980s, early 1990s and even the first part of the 21st century. The newest buzz words in police reform circles, the buzz words that we've heard a lot during the Executive Session have to do with information-led policing and predictive policing. In fact, two of the panels that are running concurrently with this one have to do with information — one of them features Jerry Ratcliffe who is a major proponent of information-led policing. Another one is explicitly focused on predictive policing.

And information-led policing and predictive policing differ in some ways, but they're both efforts to focus police policymaking on data, on objective analysis of data, on objective analysis of data that's done in a centralized manner and objective analysis of data that focuses overwhelmingly on crime suppression as the goal of the police. So that is to say that both information-led policing and predictive policing return in significant ways to each of the three central components of the professional model of policing in the '50s and '60s, crime suppression focus, centralization and detached objective, apolitical analysis.

There are really important differences between what people who talk about predictive policing and people who talk about information-led policing, on one hand, and people in the 1960s and the 1950s were talking about, on the other hand. It would be wrong to say that people who — that information-led policing backers or predictive policing backers want to return to the world of “Adam 12.” For one thing, “Adam 12” was all about random patrol. That's what the guys in the show did every day, and the one tactic that everybody agrees we don't want to return to is random patrol.

More broadly and more importantly, a big piece of predictive policing and information-led policing has to do with the innovation that Jeremy and Chris talk about. That is to say, that a big piece of what information-led policing and predictive policing is supposed to do is look at objective measures at how a successful a police department is being in fighting crime and then use those objective measures to learn and continually to improve the performance of the police. That's new. That wasn't part of the professional model of policing in the 1950s and 1960s.

Nonetheless, I myself am skeptical of ILP and PP as banners under which progressive policing should march, not because I think technology and data analysis aren't important, but because I think there is a strong tendency for them to be overhyped and for technology to drive out dimensions of policing that are equally if not more important. In fact, I'd say that's a big part of what led to the discrediting of the professional police model in the 1950s and 1960s.

I would add to — I remember Chris had a slide that overlaid the two cops in “Adam 12” with all the things that turned out to be bad about that form of policing, and I think that was a good list of all the criticisms that got leveled against predictive policing, except that it leaves out over reliance on technology, which especially in Los Angeles was an important part of the critique of the LAPD from the late 1960s on. So, I mean, and I think there is a lesson to be learned there.

I'm not sure, but I think Jeremy and Chris may agree with me about a lot of this, and so it's possible that a lot of this really does come down to labeling, but, as I have said, I think labeling can be important. So, to me, the question is why would you want to call where policing is moving a “new professionalism.” What does the term “professionalism” get you?

Well, the term “professionalism” can mean a couple different things. Sometimes we can talk about somebody being professional when we mean they work full time at something; they make a career of it. We may disparage a politician, for example, by saying that she's a professional politician. That means that that's her only job and her only career.

Well, nobody thinks that policing in the United States suffers from a deficit of that kind of professionalism. Nobody thinks, that is to say, that today police officers don't make policing enough of — they don't have enough of a commitment to this. When I say nobody, that's not quite true. If you talk to older cops, they'll sometimes complain that younger officers don't have the same attachment to the job that they did when they were rookies, but outside of the ranks of older officers, I don't think there's much support for that view, and I don't think that — that's not what Chris and Jeremy are talking about when they're calling for a new professionalism.

So another way in which the word “professionalism” sometimes is used is to just refer to a high level of competence. Like a photocopy shop might say, “We're really professional,” meaning we do really good photocopying, but, I mean, if that's all professionalism means, being really good at something, then it doesn't really add anything. I think it's clear that that's not what Jeremy and Chris are driving at either.

I think that the only other strong connotation of the word “professionalism” that I can think of — and I might be missing something — is an appeal to a notion of a self-regulating, autonomous, occupational body with a specialized core of knowledge and expertise, like lawyers or doctors or engineers. And that version of professionalism, it seems to me, is inherently linked to a certain push towards detachment and certain emphasis on objectivity, a certain de-emphasis of politics and community engagement, and I think that those are things we should be worried about.

Community policing, it's true, has all kinds of problems as a concept. It was never fully realized. It's a radically incomplete set of ideas of about policing. It does raise all the questions that Chris raised towards the end of his remarks, what does it mean to partner with a community, what do we mean by community anyway, what do you do when different people in the community disagree, as they always do, about how the police should act, what do you do when, as always the case, there are lots of people in the community who don't seem to have any clear ideas about what the police should do or have conflicting ideas about what police should do, and what are the respective roles of different parts of a police force in engaging with a community, how do you use middle managers in community policing — that's something that advocates of community policing have always been really weak on — and how do you use a community policing approach to try to think about problems like policing of political dissent or policing of covert surveillance. So those are incredibly hard problems.

It seems to me that those are precisely the problems that police executives, police officers and people who care about the police need to be thinking and worrying about. That is to say, all the problems that the framework of community policing raises are the problems that I think we should want everybody who cares about and thinks about the police to be focused on.

The problems that the frame of professionalism raise seem, to me, less important. I mean, it is true that if you call for a new professionalism, then the very rhetoric raises some questions that could be useful. So one thing that the rhetoric of new professionalism raises is what do we mean by professionalism. What does it mean to be a professional? Another question that it raises is what's the relationship between policing and some other things that we think about as professions, like medicine or the law or engineering. Those are interesting, important questions, I think, but I don't think they're anywhere near as interesting or as important as the questions that are raised by the framework of community policing.

So that's why — I mean, my own view is that if you want a framework for policing, community policing is a pretty good one. If you want to emphasize that we need to push beyond what community policing advocates have already accomplished, which we definitely do need to do, call it “advanced community policing” or something like that.

But my own sense is that we're better off keeping the focus on precisely the questions that the community policing framework raises because I think that in that respect, we're not that different, we're not in that different a space or time than Ed Meese and Mark Moore were when they had their exchange, which is to say that I think today, as then, police don't need to be encouraged to incorporate new technology. What they do need to be encouraged to do and helped to do is to think about the very difficult questions about what it really does mean in a democracy for the police to be part of a community, to work with a community.

So, having said all that, I want to say that I do think that it's possible I'm wrong. I mean, it's possible that the rhetoric of community policing itself contains dangers that are larger than its promise. It's possible that just using the term “community policing” is stupefying because it carries with it, inherently and necessarily, this idea that communities are easily understood things and that it's obvious how you engage with a community.

I don't see that, myself. It doesn't seem to me that the language works that way, but I throw it out there as a risk. It seems to me that that risk is smaller than the opposite risk, the risk that by reemphasizing professionalism, deemphasizing community, we are at risk of repeating the mistakes that we made during the 1950s and 1960s in terms of over reliance on technology, under-appreciation of the necessity to engage constructively with a community.

So Chief Gil Kerlikowske said in a speech about 5 years ago that there's a possibility that the police, like other professions and other occupations, have a 20 year learning cycle; after 20 years, we forget the lessons we've learned and move on to the next new thing in policing. So my concern is that that may be what's happening at this moment, that we are tired of the lessons of community policing because they seem familiar, even though the community policing agenda is as important as it ever was and remains so largely unfulfilled.

Scrivner: Thank you very much, David.

[Applause.]

Ronald Davis: Good afternoon. I don't know where to start. I thought of the — anyone see the commercial? I'm probably dating myself a little bit, but I think it came out recently where you have the peanut butter and then you have the chocolate, and somehow they kind of bump into each other and, therefore, you have the Reese's. This debate made me think about it. You have two different thought processes, and we're wondering at which time they will collide together and they'll come up with something new that we now know as to be a very good treat and probably most of us eat and we shouldn't.

And so I thought of that when I listened to the debate and I read the papers of — I agree with both that they're so close together. But I'm going to try to add a view now as a police chief that may have to enact or make some of these thoughts work.

And I think about what is the role of a police chief, and I think part of the debate is maybe starting with some questions because, you know, the basic Covey principles start with an end in mind, and it seems to me that we quite often have not defined what that end is. And so there still seems to be a debate as to what is the role of the police department, what are the police there for. And the answer to that question would determine in many ways what we should be engaging in.

And so I think the real challenge — and I do agree with both and the many points that they made. I think the challenge right now as I look at community policing in my 25 years, is that, intuitively, I think everyone is feeling, definitely at the Executive Session, myself and in the industry, that we've hit this ceiling, that something has happened, that we now need to go in a different direction because we've taken this concept as far as it appears to have gone. So, when I look at community policing, I think back in the '80s when it was debated at the Executive Session, it was a radical thought. 2010, it's an easy program. Something happened, and part of it is probably success.

So the question is what is the next step in community policing. Is it advanced community policing, as David mentioned? Is it a new professionalism that Chris mentioned?

In looking at that, I have to look at the balancing act that I have as a chief of police. So, one, I have a community that whatever you call it, I still have to serve. Two, I have a political environment in which I have to survive; otherwise, there is no policing for me. And that's a reality of all the chiefs. I have external constituencies. I have to worry about inside the organization and some new dynamics that have changed policing in one of the three C's, and that is cost. I have to operate within very tight fiscal constraints, further reducing budgets and the ability to do easy programs and start really identifying what are the core missions or functions of the police department.

So the question for me going forward is how do I still provide a high level of service while still embracing the concepts of community policing because, if I heard this correct, I think both Chris and David are not saying abandon the idea of community policing, but I would have to lean towards that we do need something different in that community policing has to go further than I think the profession has taken it.

I think the true success of community policing is the evolution from community policing to community-based governance, and as long as we keep talking about community next to the word “policing,” I think we're going to keep running into this ceiling because now if we're going to start dealing with the kind of problem-solving that community policing suggests, it has to go beyond the police; that community policing, to be effective, in my opinion, should be a subset of community-based governance. And so I would probably put a little challenge on both that if you look at this concept of new professionalism, starting with the three C's, that it will probably actually be six C's.

Chris, I'll throw that first thought out there.

And so you have crime, cost, and conduct, but under community-based governance, you would have city leadership and the idea that all of city government is involved in making our community safe. That has to be led by the chief executive officer, whether it's a mayor or city manager. It involves public works, redevelopment. It involves planning. It involves from — every city department has something to contribute.

Why that's important with community-based governance is because then they should be measured by their ability to actually contribute to that ultimate goal. Right now it still seems to be solely focused on the police, so we had this debate about community policing versus a new professionalism.

The other “C” I would add would be collaboration with outside agencies and the community. The more we have to reduce our budgets, the more we have to collaborate.

The other “C” that's not in there is the community. And I think if you add those three C's to the existing three C's, then the issue of community and why we want community policing may exist, but it may actually be at a higher level, so just not community policing. It's community-based governance. It's the idea that the community has a lot more control than just that of the police department. It has control of the people they elect to run the police department or the city. It has control or input and control of all departments that contribute to public safety.

If we were to achieve that, then I think you can then go from community policing to a community-based governance to this idea of a new professionalism, which to me the ultimate goal would be something that Gil Kerlikowske mentioned to just getting into public safety.

And if we ultimately one day just accept that the role of the police maybe even will change to public safety, inclusive of public safety are everything you can think of. It would be crime, which is one component. It would be emergency preparedness, which is another component. It would be terrorism as a component. It would be race, racial turmoil as a component. It encompasses so much that you're not focused solely on crime control. It may actually even change CompStat.

I am a proponent of the idea of a new professionalism, as I read both papers, because I think we do have to take community policing to another level. I am concerned, however, with David — is how do I sell, quite frankly, a new professionalism to my elected officials that have put so much into community policing. And I think to get to this new professionalism, there may have to be that stop along the way. It may have to be not immediate, but the first stop may be, once again, the transition from community policing to community-based governance, then to a new professionalism.

I think the police profession has to step it up with regards to professionalism and identify a question that Patrick Murphy asked some years ago: Are we a vocation or a profession? And why that's important, I think it goes to the issue of the national coherence. If we're a profession, that means we're going to live by and adhere to a body of standards, of core values, of regulations that hold us accountable. And that policing as a community member, as an American, you have a right to have a certain levels of standard, regardless of where you go in the United States. It should not change because you cross some jurisdictional boundary. So there should be a body of standard that's consistently followed, especially with regards to the accountability.

And I think we have to get to the issue of professionalism. That was suggested. Then there's criteria to get hired. There's criteria to get fired. There's criteria as far as education, as far as training, and there is a level of accountability that goes with having established professionalism.

Unfortunately, we have made the word “professional” bad, and it would seem to be going backwards. So I think David may have struck something that it may be the packaging.

But I would offer that it may be time for a new package with the concept that as not abandoning community policing but actually advancing it to the next level. And that if we were to package it as a recommendation, in such a manner that we're looking at community policing now evolving to a larger context of community-based governance, that we're looking at taking police to the next level of professionalism which looks at the core areas that we talked about with regards to police legitimacy, the three C's and now we would say the six C's, the issues of innovation and creativity, I think all of that then fits and meshes perfectly and goes back to my initial comment about the peanut butter and the chocolate coming together to come up with a very good treat.

I think it's going to be a lot of work, and we may have just defined the last 20 years. I don't have the answer. I'm just thinking from the perspective of a police chief, how do I make all of this work, how do I do it within very tight fiscal constraints, how do I do it so my officers understand what the heck I'm talking about, how do I do it so that the politicians and elected officials don't look at me like I've lost my mind and am trying to abandon something that they've passed through resolutions, and through even may have even got elected on, how do we do all of this. And I think what I'm feeling intuitively is that I don't know the answer to that. I just know we have to do something.

My concern now is community policing, as I mentioned earlier, is went from that radical thought to just an easy program. We can define what is a community policing officer, which I don't understand what that is. If community policing is a philosophy, then all officers should be community policing. We have community policing grants, which are separate from technology grants, which are separate from policing grants and enforcement grants. So we are constantly dividing this concept up to make it more and more programmed, and a threat to community policing is that it's going to be specialized. And, as a good professor once told me, to specialize in some cases is to minimize.

So the further we go with embracing a concept without advancing it, whether it's advanced community policing or a new professionalism, the risk we take is to minimize its actual impact. And at some point, community policing is going to be some old school program that we had 10 years ago that will be on the chopping blocks first when we start reducing budgets.

The idea of professionalism, I think is not tied to cost, in a sense, other than you are to manage those costs most efficiently and effectively as possible.

So somewhere in between, I think the two brilliant minds have come up with something that I believe the industry has to tackle with and grapple with, and I hope comes out of the Executive Session is they'll answer to that, will set us on a course, I believe, for the next 10, 15 or 20 years because I just know as a chief, I'm feeling that I'm hitting a ceiling, that I know that I'm engaging with my community and I'm having the meetings and I'm trying to improve the relationship and we're providing cultural competency training and we got the officers really understanding the role or at least the importance of community involvement. We're still very vague with the role of it. I think someone said that earlier, and I agree, but we now know that the problems are deeper than that.

So the question I ask my community is, now that we've had the meeting, what next? What does that mean? So we've talked, and it took 10 years, 20 years, quite frankly, for the profession to even acknowledge that the community was the recipient of services that provided input, and that's a major advancement, but where do we take it beyond the meeting?

If I say you have a role, what really is that role? Is it a role to set priorities? Am I misleading you to think that you can set a priority of traffic enforcement when you have 20 shootings going on in the neighborhood? Is that realistic, or is it helping to identify the priorities on some of your non essential areas? Is it defining what the response is going to be? Because some communities, even those that are most likely to be accused of being abused, will give you abusive tactics in response to things that bother them.

So how far does this thing go, or are we misleading people to say that you identify the priorities, you identify how we respond, knowing that as soon as you get back to headquarters, it doesn't mean anything because I'm still going to put the officers out there the way I want to, and they've still got to operate within a body of standards regardless of what someone said at a community meeting.

So we have to take a look at what does this mean now, where do we go to the next step, so that I have realistic expectations for the community, this is what your role should be, this is what we can do together. But, once we identify it, then I do want to, as an executive, embrace a body of standards. I want to embrace science and technology to do it in the best and efficient manner that we can do. I have that obligation.

I don't think the police department should run away from technology, and I don't think technology should run away with the department. And I think somewhere in balancing it, that is to use it to truly get to the most efficient and effective police department one can have, but those balancing acts, once again, goes to trying to merge the two concepts.

Like I said, I don't know what the ultimate title will be, but it has to be something that captivates the officer and the staff. It has to be something that provides political support or obtains regardless of political support, and it has to be something that's tenable within the resources, the limited resources that we all know we have.

So somewhere in there — and I think Ellen is going to get into this, too — is I think, intuitively, they have struck upon something that many of the chiefs are feeling, and that is we have hit this ceiling and we need to break through it. We have done a great job, and the crime reduction is at its highest. I think police and community relations is probably at its highest. We still have a lot of question of legitimacy, especially in communities of color, but to compare that to 20 or 30 years ago, undoubtedly is better. But we also know now that new problems of policing require different responses.

Excuse the phrase — and I'm one that struggles with this quite often — if we were to diet as a country, we lost that first 10, 15 pounds. And that was easy, it was good, it was water weight and good for us. Now it's getting down to the real hard part, and that is how do we actually go further, how do we sustain it and not keep going up on these up and down increases and decreases of what we called as “community policing.”

And I think the way to do that is to quit making — not to minimize it, not to specialize it, but to embrace it as a larger capacity, a larger context for city government, so that whatever the police department engages in, it has to fit into community-based governance. Otherwise, that chief executive would find him or herself operating outside the direction of the city.

So I think I'll end there and just open it up, but I just think as a chief, I do appreciate the thought. I do appreciate the ultimate goal. But I think the challenge is going to be how to interpret this, and to think about it at some point if the great minds in here, with all the acronyms, are struggling with it, then how do you translate it to people like me, the chiefs that are running the organizations and to the hard working men and women that have their hit in the field, and you want to translate that into some specific goals.

And I think the ultimate goal, I would envision a police department that has public safety officers that are not necessarily over-reliant on crime, unless that's the issue of the day, but will look at a wide range of everything. And if you think about a public safety department, then all of its members are public safety agents and officers, including civilian or non-sworn at every level. Whether it's a community service officer or code enforcement officer or secretary, they're all contributing to public safety. So that may be also something to think about, how do we evolve to that point, which I think Gil was eluding to as just how about we just police. I would maybe change that to we just do public safety and recognizing that community engagement of policing is the philosophy in which we operate as a government, not just as a police department.

Thank you.

Part 1: Collective Efficacy

Jolene Hernon: This is a two-part podcast. Today, we are talking about collective efficacy, a very early and important finding from the “Project on Human Development in Chicago Neighborhoods.” In the companion interview, Dr. Liberman will talk about violence, race, ethnicity and exposure to firearms. Akiva, tell us about collective efficacy.

Dr. Akiva Liberman: Well, the question they were interested in was why some neighborhoods are able to better control deviant behavior and disorder than other neighborhoods. And their theory, collective efficacy theory if you will, is that the primary ingredients for that are a shared sense of values and trust of your neighbors and shared expectations about intervention in deviant behavior, to control disorder in deviant behavior.

Jolene Hernon: So that sounds a little abstract. Can you bring that into the real world for us?

Akiva Liberman: Sure, let me give you an anecdote. When I take my kids to the playground in my neighborhood, and there is a kid I don't know who's doing something I think is inappropriate, the question is, “Am I going to intervene?” If I think that my neighbors and I share standards of behavior, that they trust me and they expect me to intervene, then I'll intervene. But if I think that the parents of that kid have different standards of behavior then I do, or don't trust me and don't want to me intervene, then I'm likely to just let it go. That's the same concept here for collective efficacy, that you need to have shared standards of behavior and values and expectations that your neighbors are going to intervene on your behalf and vice versa.

Jolene Hernon: Okay, why is that such a big deal?

Akiva Liberman: It is a big deal for a couple of reasons. First, they show that the collective efficacy in a neighborhood was related to the amount of violence and victimization in that neighborhood. So that seemed to show that this is a process related to people being able to control disorder and deviant behavior in their neighborhoods. The other thing is that it's long been known that some neighborhoods have more violence than others and it's been shown that, for example, levels of poverty and disadvantage are related to level of violence in a neighborhood. But nobody really thinks that it's a direct connection. Being poor doesn't make you violent.

Jolene Hernon: Yes.

Akiva Liberman: So, why is that? And prior research didn't really satisfactorily get at the processes that might be related to the poverty on the one hand and violence and other negative outcomes on the other hand.

Jolene Hernon: The more collective efficacy a neighborhood has, the less violence. And the less collective efficacy, the more violence. Is that right?

Akiva Liberman: Yes, that is right.

Jolene Hernon: So what kinds of characteristics in a neighborhood cause a neighborhood to have more collective efficacy? Or maybe I shouldn't say cause, but what factors in a neighborhood contribute to collective efficacy?

Akiva Liberman: Neighborhoods have been found to have more collective efficacy when they have less poverty and disadvantage, more residential stability and also when they have more Latin American immigrants, who are most of the recent immigrants in Chicago.

Jolene Hernon: So it sounds like poverty is the big issue here, the big factor?

Akiva Liberman: Well, not precisely. Poverty, they would say, is a condition under which shared values, trust of your neighbors and shared expectations for intervention are undermined. Poverty and disadvantage.

Jolene Hernon: Okay.

Akiva Liberman:  But again, poverty and disadvantage don't seem to be . . . it seems hard to explain how poverty and disadvantage would directly explain levels of violence. The other thing to say is that when two neighborhoods have equivalent levels of poverty, but one has more collective efficacy, then the one with more collective efficacy has better outcomes.

Jolene Hernon: Okay.

Akiva Liberman: Another important thing about collective efficacy is that it's not a characteristic of individuals; no matter what my own resources are, how wealthy I am, what my own values are, I can't generate collective efficacy by myself. It's something about . . . I don't want to say relationship because I don't have to necessarily know my neighbor to share values with my neighbor, but it's a sense of the social fabric and the collective understanding about what we should do.

Jolene Hernon: Thank you very much, Akiva Liberman, social scientist analyst here at the National Institute of Justice.

Part 2: Race, Violence and Exposure to Guns

Jolene Hernon: I am here with Dr. Akiva Liberman, who's written a Research in Brief for NIJ on adolescents, neighborhoods and violence. Welcome Akiva.

Akiva Liberman: Thank you, Jolene.

Jolene Hernon: This is a two-part podcast and today we're talking about two studies out of the “Project on Human Development in Chicago Neighborhoods.” One study on race and violence and the other study on exposure to firearms violence. Akiva, tell us a little bit about the RIB first.

Akiva Liberman: This Research in Brief summarizes the findings from four published articles coming out of the “Project on Human Development in Chicago Neighborhoods.” Today, we are going to talk about two of them. One of them concerns race, ethnicity and violence and the other concerns the effects of the exposure to gun violence. I should say that I didn't conduct any of this research myself; I'm summarizing it. The paper on race, ethnicity and violence was written by Robert Sampson, Jeff Morenoff and Steve Raudenbush and the paper on the effects on the exposure to gun violence was written by Jeff Bingenheimer, Robert Brennan and Felton Earls. I've just summarized them for our audience.

Jolene Hernon: Okay, the ethnicity and the race article that you summarized in this Research in Brief. Tell us about that one.

Akiva Liberman: Okay, that article was published in 2005 in the American Journal of Public Health. And it attempted to answer the question, “Why do adolescents of different, from different racial and ethnic groups, seem to commit violence at different rates?”

Jolene Hernon: Seem to or do?

Akiva Liberman: Well, they actually do. I mean, the first result of the study is that these participants who are asked about how much violence they themselves have committed differ in their reports as a function of race ethnicity. The black kids report committing more violence. The Mexican-American kids report committing the least violence, a bit less than the whites. And everyone else is somewhere in the middle.

And so, that is a long standing kind of finding, although sometimes it comes out of arrest reports. But here it is from self-reported violence. And they sought to see whether they could explain why that would be.

Jolene Hernon: And what did they find? Could they explain the differences?

Akiva Liberman: They found that they were able to explain about 60 percent of the black/white gap and all of the Mexican-American/white gap by three basic factors, those being whether the kids' parents are married, their immigrant generation and features of their neighborhoods.

Jolene Hernon: And was there one that was the largest influence?

Akiva Liberman: Of those three, the neighborhoods had the largest influence and that explained about a third of the black/white gap.

Jolene Hernon: Okay, so are we talking about collective efficacy in the neighborhood? Then, is that what they found?

Akiva Liberman: Actually, they didn't. In this study, again, this is from the longitudinal study, they found that the presence of lots of first-generation immigrants in the neighborhood and the presence of professionals living in the neighborhood are both protective factors. And neighborhood residents who are cynical about the law is a risk factor for those adolescents committing violence.

Jolene Hernon: Did I hear you correctly? That being an immigrant is a protective factor? How does that work?

Akiva Liberman: Well, first of all, it's worth pointing out that findings about immigration were consistent at both the individual level and at the neighborhood level, so individual kids who are first-generation immigrants commit less violence than second generation who commit less violence than the third generation. That is at the individual level. At the neighborhood level, kids living in neighborhoods with lots of first-generation immigrants commit less violence too.

Now it's important to just give this a little context. We are talking mostly about immigration from Latin America, primarily Mexico, in Chicago at the period being studied here. And contrary to some stereotypes, perhaps, yes, the first-generation immigrants have much better outcomes on a whole variety of things. Also, some health outcomes compared to whites of comparable means.

Jolene Hernon: But I thought immigrants tend to be poorer and more disadvantaged than whites?

Akiva Liberman: Right.

Jolene Hernon: So how does the poverty and disadvantage play into this?

Akiva Liberman: Well, we try to distinguish these things by comparing implicitly in statistical models by comparing immigrants and nonimmigrants at similar levels of poverty so we can disentangle those. And it turns out that immigrants do much better for the level of wealth or poverty that they have compared to whites and also compared to later generations of immigrants.

Jolene Hernon: Compared to later generations of immigrants? I think I have heard you talk a little bit about this, the “acculturation effect.” Can you explain to us what that is?

Akiva Liberman: Yea, well, there are two different hypotheses that haven't really been completely settled I think, but both might have an element of truth to them. One is that the first generation of immigrants who come over are especially law abiding, motivated, hard working. There's also health outcomes. Maybe they also are particularly healthy and hardy and so on because it is hard to get here.

Jolene Hernon: Yes.

Akiva Liberman: And then, their kids don't inherit that completely and then their grandkids, even less so. So they come with some kind of inborn advantage with regard to the kind of things that we are talking about that wears off with generations. That is one possibility.

The other possibility is that something negative is happening to their kids and grandkids as a function of being here. That's called the “acculturation paradox.”

We are talking here about immigrants from Latin America, primarily. And they come here and they are striving as best they can to fulfill kind of mainstream American values. They're striving to achieve mainstream American goals. Your kids and grandkids notice some rift between the Latino community and the mainstream white community and become somewhat alienated as a function of that, and, over time, they then start identifying with some kind of counter cultural tendencies. So, these are two possibilities, they both might be true to some degree.

Jolene Hernon: Akiva, can we move now to the exposure to gun violence findings?

Akiva Liberman: Yes, those came out also in 2005 in Science magazine and the first author is Jeffrey Bingenheimer. And that study explored, “What are the effects of being exposed to gun violence; either by witnessing it, being a victim yourself or having a family member who was victimized.”

Jolene Hernon: What were the findings?

Akiva Liberman: The findings were that the kids exposed to gun violence were twice as likely to commit violence themselves as kids who were not exposed to gun violence.

Jolene Hernon: But maybe the kids who committed violence had a propensity for being violent anyway. I believe in your RIB you mentioned many of those kids were more impulsive and aggressive. Their parents used corporal punishment more. And so, how did the researchers tease out the differences between kids who were more likely to be violent from the kids who were exposed to violence?

Akiva Liberman: This is very important issue you are raising. It's the issue of trying to reach causal conclusions from non-experimental data.

Jolene Hernon: Yes.

Akiva Liberman: And there are a couple of things to say about it. First of all, they took advantage of the longitudinal design to at least know what came first and what came second. So, in the second project interview, you will remember there were three interviews with each kid, in the second interview kids were asked about whether or not they had witnessed or been victims of gun violence. At the third project interview, which is about two and a half years later, they were asked about any violence they had committed over the last 12 months.

So the first thing we know that it is not a chicken/egg problem. But, even so, as you mentioned, there are lots of differences between kids exposed to gun violence and kids who aren't. And the main challenge for this paper was to disentangle that. And the goal is basically to be able to compare kids who are similar on all those things but some were exposed to gun violence and some aren't. So, instead of just comparing the group of all kids who were exposed to gun violence simply to all kids who weren't, what you try and do is get groups who are very similar and do comparisons within those groups.

In this case, they used a method called “propensity score stratification.” They divided the kids up into 12 strata, 12 groups. And the difference between them is how much they were at risk of exposure to gun violence. And that takes into account all those other things that you were talking about; neighborhood characteristics, their individual psychological background, stuff about their family, peers, all that, is taken into account.

And then, kids who are at high-risk of exposure to gun violence because of all that stuff are looked at. And those, among that group with high-risk, those who were actually exposed are compared in outcomes to those who were not exposed. And similarly, kids who are at low-risk of exposure to gun violence but some of them were exposed are compared to similarly low-risk kids who were not exposed. And this way, most of that possible bias is eliminated.

Jolene Hernon: So now, let's ask the question that all the chiefs of police want to know, about how they can make their neighborhoods safer. Based on what the research shows, what are the policy implications?

Akiva Liberman: Well, let's work backwards. Starting with the paper on exposure to gun violence, this paper, I think, is the strongest methodological study to date about what the effects of witnessing gun violence are. You can't do an experiment on this question. And so any results . . .

Jolene Hernon: You can't do experiment on this question because?

Akiva Liberman: I can't experimentally decide I am going to have some kids get shot at or witness somebody get shot and see what the effect is.

Jolene Hernon: Right, Okay.

Akiva Liberman: So we are left to resort to what are called observational or correlation studies. And it's very difficult to be confident that you have gotten cause and effect relationships. This study is probably as strong, I would guess, as you are going to get at addressing this and it finds a pretty dramatic effect of exposure to gun violence that seems to confirm the pretty common sense notion that there is a kind of a cycle of violence. People who witness and are exposed to violence are more likely to themselves commit more. And then, of course, people who witness them committing it are more likely to commit more themselves. So this suggests you want to interrupt that cycle of violence and that it will have long-lasting effects. That is the first kind of policy implication. And particulars about how you control gun violence and how you limit access to guns and so on I will leave to others to discuss.

Jolene Hernon: Okay, okay. How about this question then: “How do you make your neighborhoods have more collective efficacy?” Because the more collective efficacy in my neighborhood, the safer my neighborhood is.

Akiva Liberman: Right, that's a great question and that is what everybody asks the researchers, and they always defer the question. And the reason is that they didn't do a study trying to change the level of collective efficacy. They just did a study finding that the level of collective efficacy is related to all kinds of good and bad things that happen. To find out what changes collective efficacy requires a whole new research project, intervention studies, which are difficult to do. And I would say that they've identified the target for the intervention. Now, we have to do the hard work to develop interventions that might actually effect collective efficacy.

Jolene Hernon: What about race? What implications are there for what we know about race and violence?

Akiva Liberman: Well, the most obvious implication is that it's not race per say that's creating differences in violence. And that might lead one to be somewhat more optimistic about the future. Let me conclude with Sampson, Morenoff and Raudenbush's own words. They conclude by writing, "We conclude that the large racial and ethnic disparities in violence found in American cities are not immutable. Indeed, they are largely social in nature and therefore amenable to change."

Jolene Hernon: Okay, that's a good positive note to end on. Thank you very much. Akiva Liberman, social science analyst here at the National Institute of Justice.

Akiva Liberman: My pleasure to be here.

Charles Wellford: It's clear that, you know, law enforcement has many, many issues that it needs to address and can be addressing.

So we continue this today with a panel that is addressing this same topic, what does law enforcement want researchers to focus on, what are the key research issues, and we have an excellent group of people here. Stephanie Stoiloff who you've seen in the materials is the commander of the Forensics Division in the Miami Dade department, covers the technology, forensics side. Bernie Melekian is here with a long background, a long history of work in policing in San Diego and now more recently being appointed by President Obama as the director of the Community Oriented Policing office. And Mike Ward, chief of police in Alexandria, to bring the perspective of smaller agencies to this discussion, as you know, 20,000 agencies in the United States, most of them small by any definition of that, and too often our research programs tend to be focused on issues of importance to larger agencies without the consideration of how to translate that into the smaller agency perspective, so we've asked Mike to address that.

Stephanie Stoiloff: OK. Well, I have to tell you that I am commander of Forensic Services Bureau for Miami Dade Police Department. I am a member of the IACP Forensic Committee, but when asked to present some of the items in the research topics, you know, it's a little daunting when my world is all about DNA and forensics and crime scene, but there are topics that are relevant to everyone, you know, whether it's in the Forensics Services Bureau or whether it's in a small town police department, whether it's in a large police agency. So I thought, well, what are the issues that really are affecting.

I was trying to think of what would be relevant to you, and I thought, well, one thing I have to do is present something that isn't too technological because I'm going to lose everybody in the first three seconds; your eyes glaze over — it's the scientist in me — or I'll also give you an option of something high tech that we've done that's been very successful.

So what are we all responsible for? How to protect our officers, how to provide the best service and investigation, how to utilize either the latest research or technology to improve the operational efficiency. And we all know that not everything translates down to every agency. So some of these things are a little bit difficult.

So there's two types of solutions. One would be not based on technology but based on something you've seen somewhere else and you want to bring to your agency;, how do you do that. Two, technology based; everybody wants to run out and get the latest technology. Those of you who are computer savvy know you don't run out and get Version 1.0. You wait for like 2.0 to come out, so somebody else can work out the kinks.

So the first problem — and this is huge in Miami. Mike and I were talking earlier that guns aren't so much of a big deal for a small town, but it's huge for Miami. So we have a problem if there's guns on the street. This is my low- tech solution example.

We do something called “Gun Bounty.” Any of you familiar with that? This is not a gun buyback. OK. So I have bought a few things with me today to show you, but this is what we do. May 24, 2007, the mayor authorized a partnership with the department, our department, and with Crime Stoppers to develop the Gun Bounty Program. They said, well, what, you know, really how successful this is going to be. There was other models, and this is a situation where we based it on something else that had already been done, and we said we think we can make it better. A thousand-dollar reward for turning in someone with an illegal gun, meaning you're going to make the arrest. The goal here is to remove those weapons from the street.

So there's information they give out, the Crime Stopper number. Once the person is given the information that leads to a successful arrest, they're eligible for a thousand-dollar reward. All calls remain confidential, and it's completely anonymous if you wish to remain so.

So let's talk about this. What's the difference between this and a buyback? Well, with a buyback, there's no anonymity. It's certainly not guaranteed. You're trading in a gun for money, and there's no anonymity involved there.

There's no requirement of a reward, but imagine the success as you're going to see that we've had once you ensure complete anonymity and once you offer $1,000.

June 2007 through May 2010, 264 arrests have been made, 432 guns. Now, I don't know how you feel about this, but we have — I get about — in the bureau, we get about 3,300 gun cases per year, 80 high-powered rifles like the AKs we obtained from these. All of these guns then go to the lab and are test-fired for comparison to evidence that's collected on the street. Three hundred and twelve semiautos and revolvers, 40 shotguns, I mean, this is just by offering — we have something, and this is what it looks like. It looks like a little — like a stack. So you have a bill like this. This is how they do their advertisement, $1,000, and they get the arrest.

Crimes solved. I know in the back, you probably can't see this, but it's three homicides; 11 robberies; two home invasions that they foiled; 34 commercial burglaries, one residential; 114 additional arrests for other offenses; 59 stolen firearms recovered in addition to the arrest made for the Gun Bounty; one immigration hold; two dealings selling without a license; 10 arrested for outstanding warrants. That's just the crimes solved.

Then you have property seized. We all know how good that is because the property seized and that money goes back into general fund.

Narcotics valued at over $2.2 million. Currency, well, you can read the rest.

The best part is 10 Gun Bounty cases have been federally prosecuted; seven are closed; all subjects have been sentenced to an average of 14 years in federal prison, all because of a thousand-dollar offer.

So how do you market this? I mean, again, this is as low tech as it gets. Gun Bounty information is distributed in neighborhoods. It's to businesses on a daily basis. They distribute monthly at community events. They distribute within 24 hours in areas where a shooting has occurred, and, well, in Miami, there's a lot of those. The material is dispersed through media, everything you can think of. Every inmate who is released gets one of these, one of these “how to get a stack.”

That's the low tech, but look how far that's gone. Now, other agencies within the United States have actually done the Gun Bounty Program, but nobody has had the success that we've had. We're not really sure why ours has taken off and why ours is so much different, but we've made great, great strides and been recognized for this across the country.

Okay. So now we're going to go to the high-tech solution. What's the other problem? A big problem that you have, whether big or small agency, how do you make a positive ID? Because, well, most people are very fond of giving aka's you have no way to verify. So we need a way to do a positive ID on the street.

Our solution was mobile AFIS. Mobile, what it does is it gives them the ability in the field, on the street to make a positive ID. It provides the user with the ability to quickly establish the individual's identity. Why? You need to know if this guy's got a warrant. You need to know if it's who he says he is.

We had two recent occasions. One was an officer beating. We had an officer — he had pulled and made a traffic stop, pulled over two people. The woman was taken into custody; the male fled. Gave a BOLO, because there was somebody running from the scene that looks a lot like him. Unfortunately, it wasn't him, but they were able to distinguish that by the mobile AFIS. Unfortunately, the subject went into a nearby building — it's very close to on the street where they were — dropped a cinderblock from a second story on top of the officer's head, then jumped down off a second story, beat the living daylights out of him, and then ran him over with his own car. I am happy to say he's okay. I mean, it's a miracle that he's okay, but we were able to stop the direction of there, pursuing a false lead in order to refocus. Of course, I mean, in retrospect, the guy jumping down off the building, there were witnesses to that, that helped make that arrest, but it was a really bad situation. But it did get the other guy, the first guy that they stopped, they were able to do — he was similar to all the information they were giving out on the BOLO, and they were able to let him go.

So mobile AFIS saves time, and this is going to be whether you're in a large agency or small. This just shows — the graph on the left shows you how the population has grown. In 1830, there was a population of 83. 2010, it's about 2.4 million.

So we need to have the search results come back quickly. They need to be accurate, and you have to work with your IT people. So don't try to do this by yourself. You need the buy-in of them. You know, they need to set everything up so it works for you.

The device has to be easy. I have been told by many of the police officers on the road, you know, they glaze over when you talk science or you talk anything technical. So you want to give them something that is easy to do. You push a few buttons, and it works. And when it works, there's a lot of feedback just because of that, and, again, feedback from your people on the street.

So how does it work? The officer stops someone and needs their ID. They log into the device, capturing the two index fingers on the scanner. The mobile device sends those images, and what it does, it will take the one that gets captured, and then it actually enhances that. I mean, on the fly, it will enhance it, and you can pick which one you want to send. Sometimes the one they capture could be better, sometimes the enhanced one is better, but you have the option to send either one in.

Two-finger search is performed against the fingerprint database, and everything is done automatically. And no examiner is looking at these. It's just an automatic search against AFIS.

The nice thing is, is if the search yields a hit, it gives you a response back. It gives you the information with the SIN number. If it doesn't hit, it gives you a message back. So, either way, you're not sitting there waiting, going, “Well, did I get a hit, did I not get a hit, what's the response, is this thing working?” And now the officer knows this person's true identity. So now he can use the SIN number and check and see if there's an open warrant.

As I was telling you, the picture on the left is the example of the two prints that are taken. The one on the left is the capture. The one on the right is actually the enhanced. And from where you're sitting, they don't look any different, I know. And then it will give you the information back. The mobile AFIS comes right back to the inbox on the handheld unit. Unfortunately, I couldn't bring one with me. And it gives you the case details. On the right-hand side, what it does is tells you the score, how it ranked on the hit, the name, date printed, the ID, the sex. It gives you all the information about the offender.

Now, I know this looks funny, but I actually simplified this. I took about half the boxes out. On the left, on the top, it shows what happens with the mobile device. Basically, it's going through an AT&T server, and it goes all the way through firewall after firewall until it comes into our building, and that's our modern MDPD headquarters.

When it crosses into the switch in the AFIS room — we have a room just for the servers — it crosses over into another series of servers that produces the hit information that goes back to the device.

This is just an example. What this is, it's statistics broken down by device name. We have a whole bunch of these in department, and we have them — they are each assigned a different number, so that we can track. And what it shows you, if you can see, just for those of you in the back, device name is the first column, and the second column is entity, which means whether it's warrant, robbery, whoever is using the unit, the hits that they've had for the month, the no hits, the ones they've captured and sent and had no hits, the total they've sent, and the percentage. On the right hand side, number of searches conducted, so you can see these are well-used devices.

Monthly stats per entity, we capture how many transactions, how many hits, et cetera, and this is useful because you can definitely see that the number — you have 133 hits out of 232 transactions;, you're picking up a lot of people.

At the medical examiner's office, they use it to print the victims. They're not always good guys. So it's a way to do the ID, so you can know who's there and who's not, and you capture the same way.

So the successes of this? It saves time, it saves money, keeps the officers off the street. And one thing I should have added, too, is that it protects the officers, too. You're putting them out there on the street, you're keeping them from having to do other things and bring them in for fingerprinting and things, but you're also protecting them.

So these are some of the examples how we've used it. We've used it with ID of unknown patients, the John Does, John and Jane Does that go to the hospital, the deceased at the medical examiner. This is a huge, huge thing that we use in the Super Bowl. We've had the Super Bowl twice in the last three years, and for the command post, it was essential.

Stings, prostitution and robbery stings. Now, this is what I think is hilarious. My robbery guys nicknamed the device “Pinocchio” because they said subjects have become more inclined during a stop to provide their real name due to the mobile fingerprint device.

[Laughter.]

Stoiloff: So these are just — there's many, many, many types of handheld devices. You're not tied to one. If you research it, there's all different kinds, and they're coming out with bigger — I mean actually smaller, faster, better ones every year.

So questions, there is mobile — this is the contact for the mobile AFIS information and for the Gun Bounty.

J. Michael Ward II: Stephanie and I had a great conversation before we came in here today, and the Gun Bounty Program, I thought was absolutely tremendous. And I thought, well, you know, if we tried to do that in my area, it would be ye that has the mostest wins. So, you know, it would be the exact opposite. We'd have people showing up wanting to buy the guns, not give them up.

[Laughter.]

Ward: And, in fact, in a most recent domestic, we took over 50 or 60 firearms out of a house on a domestic, but that's typical of the state of Kentucky. We cherish our firearms. It's a right in our state, and it's something that we feel very dear about.

Obviously, in metropolitan areas, that is different. Including Louisville and in some areas of Lexington, they have issues that I'm sure they'd like to get rid of them, too.

There's a couple of things that I noticed this week, and I've spent my time going to a lot of the sessions, as many as I can get to and pop in and out.

From a small agency perspective, Stephanie commands about a hundred and — what did you say, 150?

Stoiloff: Ninety.

Ward: Ninety? Ninety people in her forensic group, and that's the sworn officers and the technicians that go out to crime scenes.

My area, I live in northern Kentucky. Alexandria is just south, about 14 miles south of Cincinnati. So our area is very suburban and rural. Most of the people that live in our area, they either work across the river in Ohio or in some of the industrial areas in Northern Kentucky.

But, in our county, we've got 10 agencies, 10 law enforcement agencies in my county. The largest is 35 officers. The smallest is one. So, in the three northern counties of Boone, Kenton and Campbell in northern Kentucky, we've got just roughly over 300 total officers. So, for a small agency, we have to become the jack of all trades, and that's very difficult for us.

As a matter of fact, next month I'm sending my — I've got 15 in my agency. I'm sending my third officer to get certified as a criminologist at the Criminalistics Academy down at the Department of Criminal Justice Training. That certifies him to do the evidence collection that we need because we don't have the resources regionally, even, to pull from. So we have to work together. Small agencies, if we have a large crime scene, we have to pull from each other, and that's where in the research and what I've seen and heard about forensics and the investigations this week, I would love to see small agencies get more involved in that aspect because there's so much that we have to do.

If you compare a patrol officer in a small agency and a patrol officer in a large one, the real difference is the large one answers a call, picks up the phone, and calls the technicians to come in and do the work. The small agency picks up the phone and calls a dispatcher and says, “I'm going to be here for a while,” because he or she has to do all that work.

So we are held to the same standard in court, regardless of the size of the law enforcement agency. The officer has to be able to do all that stuff. So it really creates a huge challenge for us in funding and resources. The biggest thing is in funding for training. We've got the officers that have the ability and the willingness to go learn. We don't necessarily have the funding to pay for the training. So there's one issue that I found this week.

Another one is in our labs. In Kentucky, we have one lab system. It is the state's, and they do a tremendous job, but in this economy and as in years past, even when the economy was good, when things needed to be cut, it seemed like the labs were cut. The funding for the labs has been a huge challenge for us. We don't submit forensic work to our labs for property crimes. We just don't do it because it takes too long for them to crank it out. It's an overload.

Now, that has its good and its bad points. If we don't submit the work, they can't request more personnel, and they can't say, “Hey, we've got all this backlog we need to clear.” But, when we go to court, we just have to hope and pray that the officers are able to articulate properly in court and get a conviction without having that forensic evidence. As an example, tool marks on a burglary could take anywhere from 18 months to two years to get back from the lab. You can't have a case sitting around that long. So that's some of the issues that we face in the rural area.

Kentucky is like in other states, and we're not unique in this perspective. The largest agency in the state is Louisville Metro. We've got 1,400. The next would be the state police with just a little over 800, and then I believe Lexington is right around 500. We have 460 law enforcement agencies, of which 380 are 10 men or less. So, in our state particularly, when we talk about research and whatnot, it's great that Louisville, Lexington and the state police get that look because they certainly have issues, but the majority of the state is small and rural, and we kind of feel at times that we just are an afterthought. And we've got the staffing. It's amazing the people that we're hiring today. We're very proud of the officers that we're putting on the street.

So I listened to a presentation on predictive policing, listened to the DDACTS, the Data-Driven Approaches to Crime and Traffic Safety, which is huge for us. We don't have analysts. That's my biggest problem. I don't have the people. I would almost have to hire a civilian to come in and to do that work for us. We have one criminal analyst assigned to the state police post in our area, but that person is responsible for 10 counties. So having one analyst try to do all those agencies and 10 counties is daunting. So we really need that support in that area because within five minutes — well, actually, from my office in 15 minutes, I can be in downtown Cincinnati. And our big issue in our area is heroin because we can go across the river into Cincinnati and buy heroin and be back across the river in literally 20 minutes, and we've had overdoses and whatnot. So that permeates the northern Kentucky area, and all those agencies, those jurisdictions that are crossed, if we had the analysts to work with us in that regards, that would be fantastic because those are hot spots and areas that we could target.

One of the things that we are very proud of as small agencies is our ability with computers. We've been looking at the mobile AFIS. Our problem with that is our pipe down state to the AFIS database is too small, and we can't do it.

But I was fortunate enough last year to be a part of the technology work group that IACP held in Indianapolis, and I sat on the mobile computing side. And right next to me was a commander of that unit for the L.A. County Sheriff's Office, a great guy. I was really ecstatic, I could sit and talk to somebody about what large agencies are doing, some of the technology that's out there, and it was reversed. We were doing more with our computers than what they were doing in L.A., and I was shocked. We are accessing databases on the Internet and doing so much more. I've got officers and it makes me laugh to think of the things that I did on third shift when I was bored, and we won't talk about that because maybe some of the statute of limitations aren't up.

[Laughter.]

Ward: But we've got guys that are data mining. They're going out and they're hitting these databases, and they're pulling down names and they're pulling down addresses and they're cross-referencing them. And instead of drinking coffee and putting on the golf course in the middle of the night, as some of us used to do, they're actually working on a computer and going and waking people up at three and four o'clock in the morning and putting them in jail because that is the level of staffing that we're hiring now. These — and I say “kids” — that we're hiring are fantastic. They're much better cops than I was at that age because of their ability to utilize and to implement technology. It is amazing what they can do, and I think we need to harness that and get some of those people more involved because the ideas that we have implemented have come literally from the patrol staff.

We have built interoperable data systems on the Web that agencies dump all their reports and whatnot in, so that we can share that locally. That was done from the car up. It was literally — we took a group of patrolmen, we sat them in a room, and we said, “What would you like to see? If all this information was available to you, how would you want to see it? How would it work?” And we built it from there, and we did it with just a little under $30,000 in grant money.

And so what can be done inexpensively is amazing when you put a bunch of heads together, and when you don't have the resources to draw on, you figure out ways to do it a little bit better. That's where small policing is. I'm kind of at awe with the research that I've seen this week, and the knowledge that's here in this building has just been tremendous for me, and I appreciate that. I just wish there was more representation from the practitioners, from the people that are using, and it's amazing to me that — you know, I've been in law enforcement in Kentucky for about 28 years, and this is really the first time I've had an opportunity to see what NIJ truly does, other than getting publications, e mails and occasionally perusing the website. So it's tremendous, and I wish we'd had an opportunity to get more involved with this earlier.

Bernard K. Melekian: Well, thank you, Chief, and thank you, Commander, and I will write today down in my diary. I don't know that I have ever before been introduced as the “grand old man.”

[Laughter.]

Melekian: And so here I stand. Thank you very much. It is a privilege to be here. My name is Bernie Melekian.

Until seven months ago, I was the police chief in Pasadena, California. I have 37 years in local law enforcement and 13 years as a police chief. And, actually, it's very appropriate if we're talking about stages of research. By size of department, Pasadena would fall right between these two agencies. We had 260 sworn officers. We had another 140 civilians. We were one of 45 police departments in Los Angeles County. There is a range just within Los Angeles County; Pasadena was the fourth largest department, if you count the sheriff's department, the largest being LAPD at about 10,000 and the smallest being a small town of Sierra Madre of 13 sworn officers. So there's a wide swing, and Pasadena was an interesting city.

We have — I think the image of the city nationally is based on the Rose Parade and the Rose Bowl game. I think there's this image that Pasadena is two square miles of mansions and once a year we have a parade.

[Laughter.]

Melekian: The reality is that it is the fourth biggest city in the county. It is ethnically and economically diverse. If you drive south from the police station for a mile, you're in an area where some of the wealthiest, most powerful people in Los Angeles live, and if you drive north for half a mile, you're in an area where 40 percent of the people live at or below the poverty line. It average — a city of 140,000 people, average 25 murders a year; when I arrived there, most of them gang related. So the issues of crime and research and how to marry those two and how to do policing in that complex environment without the massive resources of, say, a Los Angeles or a Los Angeles Sheriff's Department is significant.

In looking at — you know, I didn't know until 45 minutes ago I was even going to be standing up in front of you. So here's my presentation for the group, but I thought I would touch on three areas that I think are significant. I notice that Charles, I think rightly, commented that when we talk about research in the field, we tend to talk about technology and about computers and about those sort of kind of things, and we talk less about the human side of our business. We talk less about the importance of leadership and the importance of sort of how to connect with people.

I thought I would take advantage of maybe my tenure in this business to talk about three areas that I think warrant research, some of which is being done, and then talk very briefly about some things that the COPS office is looking at.

In no particular order, I think, the first I would list is the issue of credibility and public trust and connection, connection with our various communities. When we talk about community policing and we talk about that sort of connection piece, I would argue that it's very likely that the small departments and the rural departments of this country are doing community policing almost by default. They do it and they did it long before it had a name. It is in Los Angeles and Miami and Chicago and the other sort of large urban cores that you have to address this issue.

And, in a minute, I'm going to talk a little bit more about the concept of police legitimacy and sort of balancing the two, but there's some fascinating work being done by Professor Tom Tyler at New York University in the area called “procedural justice,” and he's written a book. He wrote it actually back in the early '90s, and it was reprinted a couple years ago, called Why People Obey the Law. And the notion was that most of us in this business and most of, I think, people perhaps in general believe that people obey the law because they worry about the consequences. They do kind of this cost- benefit analysis. He suggests that that's not the case, that his research and other research indicates that people obey the law because they ascribe legitimacy either to the law or to the lawmakers, and that it is in dealing with the police, that it is in the process of the interaction between the officer and the citizen, that it is the way that people are treated during that process that it may be more important than the outcome.

“Selling a ticket” was what we used to call it or “selling an arrest” in some ways is really nothing more than communicating this idea that, yes, I may be taking you to jail or, yes, I may be writing you a ticket or, yes, I may be responding to a problem that I can't fully solve, but if the person feels listened to, they may walk away from that with a greater sense of appreciation for what law enforcement does. And I think research in that area needs to continue and be focused.

Second is the issue of budget impacts, and we've all talked about in every conference that I've gone to, when I was still with Pasadena and since I've come to the COPS office, talks about budget, budget, budget and the decline of funding.

I believe — and this is not the Department of Justice talking. This is Bernie Melekian talking, and so you can put an asterisk by it or ignore it or whatever you want to do, but I firmly believe that most of us are convinced that recovery means that sometime in the short term that the economic condition of our various departments is going to return to where it was a couple of years ago, and I don't believe that that's the case. I really believe that going forward, the nature of this economy is going to fundamentally change the delivery of police services in a way that I don't even pretend to know what it is yet. But those of you that are out here studying this business, the way that we deliver our services is, in many ways, inefficient.

I've been in this business long enough to remember going on patrol without a walkie-talkie. I can still remember call boxes. They were on their way out when I got here, but they were there. They were still there. You still used them. You didn't get overtime. I know that's a shock. A lot of things have changed for the better, but one of the consequences of that change is that we have become, we the police profession, both sworn and civilian professional staff, have become very expensive. We eat up huge percentages of local budgets, and the reality is that the delivery of our services is going to have to be done differently than what it is.

I thought a long time ago when I discovered the concept of fiber-optic cable and thought I had had some kind of a technological epiphany that sooner or later, if you think about the reality of a citizen calling to report a crime, particularly a misdemeanor crime that is not life threatening, the idea of sending out one or two officers in a radio car at whatever the total cost of that is, is extraordinarily inefficient when in many cases, what people want to do, if you take Tyler's legitimacy piece, is they want to be able to talk to a police officer. Why not have a videoconferencing capability as more and more people get computers in their home and can Skype and do things? It may well be that one officer sitting in a police station will be able to link via videoconference and take reports. Now, that's probably archaic thinking by now. I mean, I'm sure that there are other and better ways that most of you know what to do, but you get the point of what I'm trying to make. We are going to need to deliver our services in some different way.

And the last point and perhaps because Pasadena was ethnically and economically diverse, a city of 130,000, 140,000 now, at school there are 23 languages spoken in that school district, the issues of race and diversity have become no less intense, but they have become far more complex than what they once were.

When I came in this business, talking about diversity, talking about community relations, talking about community outreach, I meant essentially relationships between the African American community and the white community, and today it is about differences in language, it is about differences in religion, it is about difference in culture, and the use of technology and the need to do language research and to do cultural training and language training to reach quickly across these barriers that change faster than we can anticipate, I think is something that we really need to look about.

I particularly like the emphasis that I've heard here in the brief time I was here about the need to link academics and to link research and to link practitioners in the field. All of us have watched programs or read about programs, you know, that are great, that are great at problem solving. Look, we've eliminated crime, we eliminated homicide, blah, blah, blah, blah, blah. It gets a nice press conference. It gets a nice ribbon cutting. It gets whatever it gets. And if you go back a year or two later to try to find out how the program is doing, it's gone. You know, either the chief left or the mayor left or the professor left or whoever it was that was doing it left, and so did the program.

I believe and one of the focuses that I've tried to bring to the COPS office since I got here is we need to fully partner, law enforcement and the academic community, in a way that does research to focus on how — what it is about successful programs that might make them sustainable, that might make them last beyond a change in leadership, and what would make them portable.

If it works in Boston — for example, I talked about we had 25 homicides a year. We looked at what Boston was doing. I went personally one year and visited with the Boston Police Department. The next three years, I sent my gang unit to Boston for two weeks at a time. We mirrored the Boston project, and we, too — Pasadena had 30 months of zero homicides, down from 25 a year. I mean, we looked at what Boston was doing, and we figured out how to replicate pieces of it in Pasadena. We have to do more of that, because, in every one of these successful programs, there are aspects of it that are unique to that location, that are unique maybe to the personalities involved, but there are also bigger chunks of that, that are easily replicable and easily could move from one jurisdiction to the next. And the National Network of Safe Communities through John Jay College and others are really trying to do that.

So I'll close with this idea that there needs to be more research in how we deliver our services and a focus on leadership and training and particularly at the supervisory level, and as we talk about predictive policing and predictive analytics and the use of data mining and the use of technology, I would suggest that if we're not careful, we will fall into this sort of binary trap that here, on one hand, is the use of technology, and here, on the other hand, is this human connection of community policing or values- based policing or whatever term you want to make that talks about the connection of one person to the next and asks us to make a choice, and I would suggest to you as law enforcement professionals, that's actually a false choice.

The choice, the goal is to follow both of those tracks to Tyler's notion of legitimacy, to Tyler's notion of how are we effective as a profession, and I would hope that going forward — and I know NIJ and the COPS office are working together in August to have a large national working conference on this issue of procedural justice, but I am particularly hopeful that we do continue to do research in those areas as well as the issue of technology.

So, with that, I don't know if there's any questions, but I certainly thank everyone for listening to me.

[Applause.]

Winnie Reed: My name is Winnie Reed. I'm with the National Institute of Justice, and I'm the moderator for this panel.

This afternoon, we're going to hear from three presenters talking about three evaluations of situational crime prevention efforts. Nancy La Vigne from the Urban Institute is going to speak first about the evaluations themselves and some about the projects. Nancy is the director of the Justice Policy Institute at the Urban Institute in Washington, D.C.

She will be followed by Captain Gary Wedge. He is with the Administrative Services Division of the Chula Vista Police Department in California. He's going to be speaking about some of the benefits and challenges involved in the implementation of the Target Safe City Initiative in San Diego.

Finally, Tara Wildes, chief of the Jails Division in the Jacksonville Sheriff's Office in Florida, will be talking about some of the benefits and challenges of the implementing a situational crime prevention initiative preventing sexual assault and inmate misconduct in a jail setting.

First, we'll hear from Nancy La Vigne.

Nancy G. La Vigne: Good afternoon, everybody. I am going to try to cover a lot of information in a relatively short period of time.

I'm not going to assume that everyone in this room knows what situational crime prevention is, so I'm going to begin by giving you a broad overview of that, so you have a context with which to interpret the remainder of this session.

Then I'm going to be focusing on three different NIJ funded evaluations that have both applied situational crime prevention concepts, and they're in various forms of evaluating the impact of those applications, trying to tie it all together, a lot of lessons learned both for practitioners and for researchers who undertake this kind of work. So we'll, hopefully, have some good discussion at the end.

And just real broad brush stroke, although this isn't rocket science and it always stuns me because I think it's just such common sense, it's really based on microeconomic theory. It assumes that most offenders aren't really motivated to commit crimes. They don't wake up in the morning and say, “I'm going to rob a bank.” Yes, those people do exist, but most of the people who commit crimes are committing crimes of opportunity. They walk by and see a situation where, you know, maybe it's a door that looks ajar or something like that. They take advantage of those opportunities.

They also weigh the costs and benefits of engaging in criminal behavior, and those costs include the risk of getting caught, right, the risk of apprehension, how much effort it takes to engage in the criminal act, and then the benefits or what they hope to gain from it. And it could be monetary gain, or it could be some other kind of gain. Like, you know, a lot of violent acts, the gain is sort of status among one's peers, for example, be it juveniles or in a correctional setting.

So situational crime prevention says given that theory, we can change aspects of the environment and aspects of potential criminal targets to alter that cost benefit ratio in a way that closes off criminal opportunities and discourages a lot of those would be criminals who didn't wake up in the morning dead set on committing a crime.

This chart on the right hand side just gives you a lot of examples of ways that you can increase the effort or risk and reduce the rewards and also remove excuses of some of those low level criminal activities, “Oh, I didn't know I was breaking the law.” Post the laws more prominently. So those are just some examples right there.

So, with that as the backdrop, I'm going to describe three different evaluations. The first one is looking at crimes occurring in the D.C. area Metro, the subway system — we call it “Metro” here — in the parking facilities.

And I want to acknowledge my project director, Samantha Lowry, who did the bulk of the analysis for this presentation. She's not here today, but she's here in spirit.

So why did we decide to look at this issue? Well, frankly, it stemmed from my doctoral dissertation when I was looking at crime occurring in subway systems, specifically looking at Metro as a case study because it had always been held up as this great example of this good design that closed off criminal activities, that it was managed very well, et cetera, but no one had set out to look to see whether that was true. And in the course of doing that research, what I learned was, yeah, crime is really low on the rail, but above ground, crime is relatively high. Most of the part one crimes are occurring in the parking facilities. They're mostly auto thefts or auto break-ins or sometimes other parking related crimes, and, you know, that's the bulk of the crime problem that Metro was facing.

And so we reached out to the Transit Police, and we said would you like to partner with us, an action research partnership which means that we were involved in the beginning to set out to understand what the factors were behind these crimes and how could we together develop the series of interventions that could close off criminal opportunities and then we would evaluate it along the way and see if there was an impact.

So that was our goal to do a lot of research looking at historical data, doing site observations of the parking facilities, trying to identify all of the environmental and management characteristics that seem to be related to crimes occurring there, identifying strategies.

And then this is a randomized control trial design in that there were 51 — actually 52 parking facilities. We ended up looking at just 50 and randomly assigning the intervention to 25 of those 50, and I'll tell you a little bit more about that later.

We used a lot of different data to try to understand the nature of this problem, and we really need to look at not just data that was maintained by the Transit Police but also the data for all the law enforcement jurisdictions that serve Metro. You know, it's a pretty big system, and so that ended up being seven different jurisdictions where we had to go and ask for crime data. So we looked at part one crimes within a mile radius of the Metro station. We looked at a lot of administrative data on sort of the nature of the parking facilities, what their capacity was, how full they were. There's a lot of different factors that I'll describe in a moment, including the site observations where we went and we made note systematically of all the different characteristics that we observed, whether there was fencing around the facilities, whether there's one exit or two, was there an attendant booth or not, was the attendant booth stationed, et cetera.

This is just some of the highlights from the site observations. Most of them had payment facilities where you had to pay upon exit. We saw that as a very good way to prevent auto theft. If you have to pay to exit, you need some kind of demonstration that you got the parking ticket that you received when you entered the system. There were some places that had multiple exits that made it more difficult to monitor the facility. We looked at the surroundings, and then, as I just showed you already, we looked at above ground crime rates in the surrounding jurisdictions. We looked at whether or not they had attendant booths and the degree to which there were actual humans in the attendant booths and many other things.

So, given all that, we came up with a long laundry list of interventions, and some of them were really creative. And one by one, they all got shot down. Either they weren't feasible practically or politically and sometimes both. So what we ended up doing is rather mundane. It was to install cameras at the exits of half of the randomly selected facilities. In addition to the cameras, we put prominent signs that said, you know, your car and your license plate could be captured on camera, and we said “could be” because, you know, we didn't want — there's liability issues. You have to be very careful with the language that you put on these signs because then you don't want people to assume that they're somehow protected and, if the camera malfunctions, they're not, but they were very prominently placed.

Even though a lot of these facilities had challenges with both auto thefts and car break-ins, mostly this intervention was aimed at auto thefts because most of these places, there's only one exit. We positioned cameras, so that they were at the exits. You know, you're going to leave. You're thinking that you're going to be captured on camera. You don't know whether they're going to capture the license plate or you behind the driver, in the driver's seat or both. You know, you'd think that that might make a difference, but we were also hoping that that would also sort of send a message that Metro cares, that they're looking in other ways, and it might even discourage some thefts from autos, some car break-ins, right?

The intent was to link. The cameras had an ability to record. The intent was to link that to some license plate recognition software that they already had in place; however, during the course of the intervention, the auto theft unit was disbanded. We have yet to locate a human being who can tell us whether or not they're using the cameras in any way. So this could be an evaluation of the bare minimum of like put the technology up and hope something, you know, makes a difference because it's probably not human interaction with that technology.

So this slide just tells you a little bit about how we did the random assignment. Since we only had 50 parking facilities, that's not a huge N to work with for random assignment purposes, and even if we were to flip a coin each time and decide for each 50, do you go in treatment or control, we couldn't ensure that those two groups would be equivalent. So what we did first was to create matched pairs on a lot of different factors. Once we created these pairs based on historical crime rate in the surrounding area, the capacity of the facilities, their payment policies and so forth, then for each pair, we assign them to treatment or control. And that method was designed to ensure more equivalency between the two groups.

So I wish I had more exciting findings to share with you, but this is one of those lessons learned that I think we'll talk about later on, and that is that all the things that can go wrong when you try to engage in this kind of evaluation work, in the real world, where staff changes and policies change, so we knew originally that when we did our analysis to understand the factors that were driving the auto thefts and other types of car crimes in the facilities, that the data was not maintained as well as we would like.

So how many of you have been on Metro, are familiar with the facilities? Some of you not. Well, yeah, fair share.

So, in some places, like Shady Grove, you've got multiple parking facilities. You've got parking lot A, parking lot B. You've got a multilevel garage, whatever. There's a lot of different options for parking, and when events occur, you sort of want to know where, right? We were counting each of these facilities as a separate entity.

So, within Shady Grove, of the three facilities, you know, two could be in treatment and one could be in control. So we'd really want to know the location of the events.

We made a lot of efforts to ensure that officers were recording that data very well at the onset of the project and then came to learn that they changed their MIS system. They changed staff, and the auto theft unit who we were relying on to make sure that they were recording this information well was disbanded. So now when we're looking at post intervention data, we have a lot of data where we don't know what facility within a larger station area experienced the crime. So we're like, “Oh, great, all this work.”

What we did so far was to look at five matched pairs where there was only one parking facility, so we knew exactly where it happened. Even with that very narrow analysis, it looks like we've seen a bit of an impact on crime. It's really too soon to tell. We really need to aggregate across all 25 treatment and control facilities, but it looks like it could have made an impact, could have made a difference.

And we're in the process of developing a strategy where we can estimate the location of those crimes in those situations where you have stations with multiple facilities.

So that is — see, I told you I have a lot to share. I've got two more studies to share with you, but we promised we'd do it all, didn't we, Winnie?

Winnie Reed: We did.

La Vigne: So hold those thoughts because I'm sure there's a lot of burning questions, and then I'm going to move on to our Safe City Evaluation, which is another application of situational crime prevention, but it didn't really start out that way.

It started out by the Target Corporation — everybody knows Target, right? — approaching us because they had this crime prevention program called Safe City. It had a lot of great packaging. I mean like glossy little folders that you give to communities. It was designed to encourage law enforcement to partner with retailers in mall environments and then, together, engage in crime prevention activities, but the Target's original focus was really on we're going to help you make these partnerships, then we're going to quickly leave, we don't want to control this crime prevention activity, but we will give you a big blown up check, you know, for the photo op and the press conference and so forth, so that you can spend money on something really cool, like CCTV technology. And so a lot of jurisdictions said, “Oh, this is cool. We can get money to buy cameras.”

And Target wanted us to evaluate it, and we said, “Well, what could you pay us?” And they couldn't pay us anywhere near what it cost to evaluate something. So we said, “Well, why don't we partner and go to NIJ and see if we can get funding?” and they said OK. And then we looked at their materials and we said, “There's nothing to evaluate because there's no real 'it' there. You know, there's like money for technology. What if we started anew and identified some sites and worked with you and developed more of a model that's based on theory, say situational crime prevention, and then together we help jurisdictions implement it?” And so that's what we ended up doing.

We selected four sites: Chula Vista, which you'll hear about in a moment; Cincinnati, Ohio; Tucson, Arizona; and Hyattsville, Maryland. All of them have their own unique stories, but just to give you some background about the way we were looking at this, we were looking at this as a retail crime issue.

As you can imagine, shopping malls, most of these were strip malls that were anchored by a Target store. They present a lot of opportunities for crime because they're designed to attract people, right? I mean, that's part of the business model is bring people in, make it attractive, make it accessible. You want to be on a transportation network, and all of those factors also make it attractive to would be offenders.

Malls are also places where shoppers can easily be distracted, so it's a good opportunity for pickpocketing. It's a place where people park their cars for, you know, an hour or more, so their cars are unattended for a significant period of time, which leaves them vulnerable to break-ins or even thefts, and all kinds of crimes end up occurring there.

So we wanted to document the process of retailers partnering with local law enforcement. To implement Safe City, we were viewing this again as action research where we weren't just these Ivy Tower evaluators who are over here waiting until the end and saying yes or no to the intervention work. Rather, we were there every step of the way. We were interviewing stakeholders. We were trying to understand better whether they were moving forward on their initiatives, whether they were adhering to the fidelity of the model, and giving them feedback, so they could improve along the way if they were able to. And so then we guided them on strategies and evaluated the impact.

This is just a lot of the common strategies that are used in retail environments, but I don't think I'll read them out to you because I feel like I'm already out of time. So I'm just going to talk about the kind of measures we use. We looked at the types of meetings that occurred, the topics that were covered, the composition of the meeting attendees. We looked at the types of interventions that were employed. We did surveys with businesses, both before and after the Safe City intervention. And, in the meantime, as part of this action research role, we also prepared a series of guides that we could share not just with the sites we were evaluating but also with the larger community.

So we have a bunch of very practically oriented guides that Target kindly put into a very nice looking format that's better than what we tend to do at the Urban Institute, and they printed them out, hard copy. We're able to distribute them to the sites, and, hopefully, that provided some good guidance along the way.

So, in terms of the impact measures, we looked at both perceptions of safety based on these pre and post surveys. We looked at level of communications and whether they increased over time, whether crime reporting increased over time, and then we looked at more of the hard impacts on the types of crimes that they were trying to reduce.

So what did we find? We found that two of the sites never really got off the ground, and those two sites had problems pretty much from the outset based on the fact that they never really followed the model. They were enamored by that big, blown up check concept and really had set their sights on using the money to buy cameras and hadn't gotten the right people around the table, the right stakeholders to engage in what is the nature of our crime problems, you know, how do we want to address them. There was no real problem analysis to understand what crimes were occurring as reported crimes. They didn't interview any real retailers. They really didn't do any of that work that needs to happen in order to get buy in for an intervention, and so various reasons, neither one of them really got off the ground.

Cincinnati was successful. They found an impact on increased perceptions of safety. More retailers were likely to report crimes. They also saw a total crime reduction in their areas. It wasn't huge, but it was cost beneficial. We did have a cost benefit component to the evaluation.

The thing about Cincinnati was they were already engaged in this huge crime prevention effort before Safe City and the Target Corporation stepped in. So it's very hard to identify cause and effect, whether they would have achieved those crime reductions anyway.

Chula Vista, however, is our poster child, which is why Captain Wedge is here today. They really embraced the model of looking at the nature of their crime problems, trying to understand what was causing them. They saw good results in terms of their surveys, increased perceptions of safety, reduced concerns about crime.

Unfortunately, we were unable to detect real significant reductions in the kinds of crimes that they were trying to reduce. We did see reductions in robberies, but we also some increases in property crimes. But the thing of this is and something that I want to talk about later if we have time, one of the problems with any kind of rigorous evaluation is that we always have this endpoint of our evaluation, even if the intervention is ongoing.

We had to stop. I don't know. We were probably on our third no cost extension as it was. What do you think, Winnie?

Reed: It was something like that.

La Vigne: You know, it was like we were out of time, we were out of money. We had to just say, “OK. This is the evaluation end date.” And I'm not so sure that our story is the right one. As a matter of fact, I think from what you'll hear from Captain Wedge, they went on and analyzed it further down the road, and they did detect a significant change. So these are some of the challenges we encounter when we're evaluating things.

We worked pretty closely with the Chula Vista PD to help interpret our findings and present them in a way that everyone felt good about.

To just summarize this Safe City evaluation, both Chula Vista and Cincinnati were more successful because they already had a strong grounding in community policing. They already had some good collaborations in place between law enforcement and the retailers. They weren't fixated on one silver bullet response. They weren't thinking CCTV. Actually, Cincinnati was, but they were thinking of it as part of a much larger crime prevention strategy. Chula Vista actually felt almost pressured, I think, by Target to implement CCTV, and they said, “No. That's not the way we want to do this,” and I'm sure we'll hear more about that soon.

So I think that there's reasons why we saw some success there, and the other two sites, as I already explained, they just weren't really following the model as intended.

Some implications that we learned from this project, one is that it really needs to be led by law enforcement but not dictated by it. It had to be a partnership and an authentic partnership with the retailers and the other members of the community, and that requires a lot of communication.

The problem solving, you know, I'm calling it “situational crime prevention.” A lot of us call it “problem oriented policing.” I find the two to be closely aligned, complementary if not overlapping strategies, and that problem solving was real key in developing responses that would be effective.

And then we already talked about evaluations that become derailed by a lot of different things, including local politics, which certainly came to play in Tucson.

And we talked about the confined evaluation period already.

So, two down, one to go, and this is a more unique application of situational crime prevention. As far as I know, there's not a lot of examples of evaluations in correctional settings. Richard Wortley wrote an excellent book on the topic of theoretically how would situational crime prevention be applied in correctional settings, but I think this is the only published evaluation of it.

And what we did was set out to look at reducing sexual violence in three jails. Now, this was funded with Prison Rape Elimination Act money that was through NIJ. So, of course, we were expected to focus on sexual violence, which, of course, we did, understanding that the rates of reporting of these acts are extremely low and unreliable, but we felt that theoretically if you look not just at those rare events but also looked in the larger context of violence and infractions, we could still develop some strategies and implement them and see if they had an impact.

I am ashamed to say that I want to acknowledge my colleague, Sara Debus Sherrill, and late last night, I added an “h” to her name, and I don't know where that came from, but I'll take it back later. She is S a r a, Debus Sherrill, and she's in the audience, and she's the project director. And I thank you very much for all your contributions on this.

So I told you to be forgiving, did I not? OK.

So this project, as I said, was funded under PREA. A few years back now, again, another action research partnership where we come in and we help the practitioners understand the nature of their problems, helping them analyze data, conducting site observations, interviewing staff, in this case interviewing inmates as well, and then trying to develop strategies that will help reduce the problems that are of issue.

So, earlier you saw a very fuzzy matrix of situational crime prevention in general. This is one that's applied specifically to looking at sexual assault, other violence and also acts of self harm which we felt were also important to look at in the jail setting.

And just to pull out any example, things like double bunking, which could be a precipitater for some kinds of violence in a correctional setting, actually is helpful in reducing acts of self harm, or target hardening, locking off risky areas like broom closets, which we found were likely areas of staff sexual misconduct, which is something that we ended up finding in some of our sites. So there's just the different kinds of things that you can do to close off opportunities for these events, even in correctional settings.

So what did we do? We had meetings with our three jail sites. We did these site observations where we systematically collected data and all the different characteristics of the facilities, the design, the visibility from where the correction officers sit, the frequency with which they do their rounds, staff perceptions about where the strengths and vulnerabilities are in the facility, interviews with inmates to learn the same from them, analysis of incident data which, of course, in the case of sexual assault was not that great but certainly lots of data on incidence of violence, of contraband, of self harm. And we took all that information, we synthesized it, and we developed a list of recommendations that stemmed from it.

And then we went back to each of our sites and we said, “OK, here is what we found.” Similar to our experience with the study of the car crime and the Metro facility, this really long creative list of things and then one by one, “No, can't do that. No, no, no, no.” But we found some interesting things to do in each of the three sites.

Here's some overall findings across three sites. In terms of sexual assault, again, it's just a really murky area. There's no real consensus on how often it happens. We found that because we were doing a lot of qualitative data collection on this that people would talk about an incident, and we discovered it was the same incident over and over again that everyone was talking about, but it was a sole incident.

There were some perceptions about sort of the common victims in terms of males being the stereotypical younger, weaker, smaller, you know, white males are more likely to be targets. We learned of some sexual harassment issues of female staff by the male inmates. There was talk of some staff sexual misconduct but no real sexual assault findings. With suicide and self harm, there was more to work off of and a lot of, I think, education that need to occur, at least in one of our sites, with the staff about what constitutes call for help versus an excuse to get out of the cell and how to treat those cases in the proper manner, and then a lot on physical assaults which are quite common, not in all the facilities. There was quite a bit of variation, but we looked at the type of weapons, the locations, the cause of the fights, whether they were mostly one-on-one or were they group and so forth.

So we came up with a series of recommendations, and as I said before, not all of them were implemented at any one of the three sites, but one that was prominent was this increased monitoring of the inmates. And we also looked at things like changing the classification system, which is what predicted who would be bunked with whom and in what proximity to other people. We looked at access to mental health care, which we found was an issue in one of the facilities, and then that led to a recommendation for training.

But what we're going to talk about today and then Chief Wildes is going to share her personal experiences with is this officer tour system that was implemented in the Jacksonville jail, one of our three study sites.

So what — and, Chief Wildes, I don't want to steal your thunder. Can I describe your facility?

Tara H. Wildes: Sure.

La Vigne: OK. So it's sort of like the panopticon. Does anyone know? It's like it's not direct supervision. OK? It's like you're in a sort of booth, and you can look down on everybody. And you're supposed to get out and make rounds periodically, and that's supposed to be documented, but in interviewing both staff and inmates, we came to learn that that was done on a very inconsistent basis, and we felt like the more the officers were interacting with the inmates, the more they're there. They're going to be increasing the risk of detection, right, but also removing excuses because they're going to be there to be approached if there's a personal problem that, if not addressed, might lead to self harm. So there's a lot of reasons why we think that this kind of physical interaction by ensuring that they're going to be doing the rounds in there physically could make a difference, and so this was the technology that was selected.

It's based on RFID technology. So it's like those cards that you use, proximity cards to get in and out of buildings, and there's boxes placed periodically around the floor, so that you can ensure that the officers, he or she, does the rounds taps with this blue — what do you call it?

Audience Member 1: Pipe.

La Vigne: Pipe, the pipe. They use the pipe and touch the pipe on this to demonstrate that they've been there physically, and that's about as much as they know about it. And Sara and Chief Wildes can tell you more because they've seen it implemented, and I have not.

So what we're doing right now is we're tracking implementation, and the end of that year, Sara, is, what, September? Yeah. So we're not quite through the year mark on the actual implementation of the intervention. Then we're looking forward to analyzing the impact, but I believe Chief Wildes has some anecdotal evidence to share with us.

So I have three minutes, and I'm just going to throw out some lessons learned, and then, hopefully, we'll have time. I know some of you are researchers, some of you are practitioners. You both know the pros and cons of engaging in these kinds of partnerships. They can really rewarding; they can also be quite challenging.

From our perspective as researchers across these three projects, I think that it's real important to find the true champions, people who really embrace research based practice, and, you know, I think by now I know how to find them, but I don't think I knew a few years ago. There's people who say “yes” to everything, and then there's people who really mean it. And then you've got to hope that they stick around because they're the really good ones and they often end up moving somewhere else. There's no control over that.

And that's one of the reasons why I increasingly think that when we write proposals, Winnie, we should propose more sites than we ever intend on evaluating. We've talked about this before, but more often than not, there's going to be one or two or more sites that just don't get off the ground for reasons that are beyond virtually any single individual's control.

Encouraging adherence to the model, the more you do it, the better, because we know that adherence to the model is more likely to yield the desired effect.

We do believe that it's important to not just be quantitative in our approach but also to collect qualitative data because that kind of context is really important in interpreting our findings. The longer the evaluation period, the better, resources permitting.

And then this is my own personal question mark. I always want to isolate what about an intervention makes a difference, but with situational crime prevention, that's sometimes difficult. Sometimes there's multiple interventions, so you don't really know what's making the difference, and I'm wondering if I should just stop worrying about that, but maybe we can talk about that later.

Thank you for your time.

Reed: Thank you, Nancy.

[Applause.]

Gary Wedge: Thank you.

OK. First of all, has everybody heard of Chula Vista? Everybody knows where Chula Vista is before today's presentation? If you haven't, we are about five minutes north of the U.S. Mexico border, one of the most southwesterly cities in the country. We are a population of about 235,000, predominantly Hispanic now, about 52 percent. That continues to change, looking at our elementary schools. We've got about 80, 81 percent Hispanic, so you can see how the demographics, the population is changing over the years.

About 51 square miles. No real distinction or claim to fame. Forbes magazine, I think it was last year, ranked us the most boring city in the United States. You know, having said that, it is a beautiful community. We enjoy a relatively low crime rate, home to one of four Olympic training centers in the country, got a very nice bay front that is as of yet completely undeveloped or partially developed.

Last year, that's our Little League Championship World Series team from Chula Vista, put us on the map.

But the city is changing, and, unfortunately, our city is tied — the budget is tied largely to the growth of our city, the housing, and when the housing market crashed, so did the budget and the city and actually in advance of some of the economic problems that other cities went through. So that impacted our project just a little bit, and I'll talk about that.

OK. I was going to talk a little bit about Safe City, but Nancy did a great job explaining what Safe City is. We were approached by the Target Corporation in spring of 2006. At that time, we decided to participate in the project. We gave the project to our then community relations unit, which consisted of a sworn police sergeant and two civilian police community relations specialists. We also had an analyst assigned to the project, Karin Schmerler, who is actually the reason behind the success of our project. I think Karin and I had a number of debates over whether things were appropriate, whether they weren't appropriate. I am, more or less, an advocate for cameras; Karin is not. We met sort of in between. I'll talk about that.

So it was given to our community relations unit, and, again, as Nancy said, it's not a whole lot different than a problem solving effort with perhaps a little more emphasis on the use of technology. Target did push heavily for the use of technology; in particular, cameras. When Target began their Safe City program in Minneapolis where their corporate headquarters are housed, there was that reliance on technology. They spend a significant amount of money, anywhere from $5[00,000] to $600,000, on infrastructure for the camera system, as well as radio communications for a lot of the businesses.

Those of you who are familiar with Minneapolis know that it's much like been described as a “Haba trail,” a lot of buildings with a lot of connecting tunnels. The radio communications enabled them to be successful in their project.

So, looking at our focus area, you can see where the orange oval is on the north part of the map. The yellow outlines our city and the current beat structure, but our Safe City area actually overlapped two cities, Chula Vista and then, our neighbor to the north, National City. And the reason we did this, at the time we implemented Safe City, the two cities were going through a process to realign the borders, realign the boundaries, and we thought it was going to happen within a year. Well, unfortunately, the problems with the economy hit, staffing changes, a lot of staffing reductions. So that was put on the back burner, but because it fit so nicely in the project area, we kept it in there. It did present a couple of challenges, which I'll touch on later on, but it was a natural fit.

And then more specifically, looking at our focus area, obviously Target being one of the anchor stores, there is a Wal Mart just to the west of the Target, a total of 55 businesses in our Safe City zone, as well as a government building, Department of Motor Vehicles, and a city park on the south side.

OK. Just real quickly on the process, again, not unlike a problem solving effort, this isn't necessarily in the order that they were done. We identified the focus area, as you've seen, did our analysis — I'll touch on that in just a minute — a couple of very lengthy, very thorough CPTED assessments as well as stakeholder meetings.

Now, we did have a number of meetings throughout the project initially with all the stakeholders who were interested in participating. We did get a good turnout; I want to say between 45 and 50 participants representing a significant number of businesses. And then we also established a steering committee made up of not only representatives from the police department and other city departments but also, obviously, the business owners. And then we staggered the meetings. One month, we would do an all partner meeting; the next month, we would do the steering committee.

We also had the funding drive. Nancy talked about how Target would roll out this great big check, great media event, great publicity for Target, but I got to tell you Target was an incredible corporation to work with throughout this whole process. They did push a lot for certain things, but when we pushed back, said we didn't want to go down that road, they were fine with it.

But at the funding drive, we had been told all along that Target was looking at funding us with about $35,000. They wouldn't commit to an exact amount. When we had our media event to roll out the Safe City project, they gave us a check for $100,000, and then we had raised several thousand dollars from some of the other businesses in the area, gave us the funding to start what we wanted, some of the projects that we wanted to do and then, of course, the project implementation.

Now, we had to decide what we were going to focus on. So, when we began the project after we took it on, we wrote up a little flyer on what Safe City was. We had a number of people from the police department go out to the Safe City area, hand deliver the flyers to every business in the area, talk about the Safe City model, what we were trying to accomplish, inviting the folks to our first stakeholder meeting, and then giving them a survey to ask them what concerns they had, had they experienced any one of these problems during the recent six months.

We also looked at calls for service, looked at crime reports, crime statistics, and then we surveyed the officers to get their perception on problems in the Safe City area, and from that, we came up with — you see the three areas that we focused on.

Far and away, the most significant reported problem was unwanted persons on property. This was transients, panhandlers, but also day laborers, and then, obviously, shoplifting and then vehicle theft.

OK. I mentioned the CPTED assessments. We did the daytime CPTED assessment. We had a number of people from the steering committee and the stakeholders from the businesses walk through the area with us to do the assessments as well as the community relations specialist from community relations to do the assessment and document what we found, and the we also did one at night, again, with a number of people. We also had a number of officers participate who had volunteered for the project and went along for the assessment, and we found what people typically find on CPTED assessments, shrubbery, a lot of the light issues, lighting being a problem, and then some of the access, interestingly enough, from some of the transient encampments.

Now, to get a quick win, one of the things that we did was worked with our city graphics, graphic artist, to develop a handful of signs that we could post throughout the Safe City area. Obviously, you can see them up there. I won't go into all the specifics, but, you know, again, what Nancy said, sort of reinforcing what is unlawful behavior, telling them what people can and cannot do.

We also wanted to brand the Safe City area, so we came up with a sign in the middle. If you're ever down in Chula Vista, you'll note that these signs are still prominently displayed throughout. We had initially identified about 175 locations. I think ultimately we ended up with just shy of a hundred signs, but, nonetheless, they are pretty much everywhere.

Now, we also worked with a steering committee to identify the order in which we wanted to tackle these projects, and, in retrospect, we were far too aggressive in our timeline. We didn't even really stick to this timeline, but this is what our goal was. Sequentially, more or less, that's what we stayed true to, but these were the areas we focused on.

Now, I mentioned the involvement of patrol. This was one of the more significant parts of the project as well for a number of reasons. Not only did it show the business owners that we had a great deal of interest on the part of patrol officers, but the patrol officers became more familiar with what was happening in that area. Consequently, they spent more time in the area. They knew what we were trying to accomplish. They knew what the issues were. So that in itself, we believe, impacted some of the perceptions that were out in the Safe City area.

More importantly, I think they were also that direct point of contact for the business owners and the employees of the businesses. Once we opened those line of communications between the patrol officers and the folks from the different businesses, the stakeholders, and they had a go to person, it addressed a number of their problems that in the past they had reported problems with getting done, not knowing who to go to, feeling like they were being given the runaround. Now they had somebody in particular that they could reach out to who could help them with their problems. And time and time again, we got feedback from the business owners that it made a difference, and it eliminated a lot of the red tape of the process that was typically involved.

Just to run through some of the accomplishments, we talked about signage. There were a number of things we accomplished, a number of things that we had hoped to accomplish, but for some reasons I'll touch on in a minute, we couldn't do.

But another one of those was window clings that we were able to do, put together small, little, static, vinyl clings that went on windows throughout the businesses, prominently displayed, essentially reminding people or asking patrons to call the police if they saw something suspicious, you know, a very minor thing but something that reinforces the fact that people are looking out for one another in the Safe City area.

We also cleared a number of transient encampments, not a huge transient problem, but, consistently, there'd be anywhere from three to five transients at any given time at any part in the Safe City area that were in this particular gateway, the Safe City area being a gateway on the north part of our city. So we were able to clear the transient encampments on the south side of the Safe City zone.

As part of that, we were also able to fence off an area that provided basically unrestricted access for the transients to get into the Safe City area. So, by erecting the fence, that limited their access or made it more difficult, more challenging for them, so they had to take a longer route and consequently were less likely to do it.

Now, one of the bigger parts of the project, of the Safe City project in general, was a day laborer issue. Now, this was kind of an interesting beast. Day laborers, if you look at calls for service only, wasn't a big problem, but when we went out and we talked to the business owners, we found that most of the businesses in the day laborer zone were directly impacted. They felt that it intimidated their customers, people weren't shopping there, and we asked them why they weren't calling, why the calls for service didn't reflect that, and they said they had just given up. They felt that there was nothing that could be done. They tried to call border patrol. Border patrol wouldn't come out. You know, there were just a number of issues.

So we took on a number of smaller parts to this project. We met with not only the business owners, but we also had a number of meetings with the day laborers themselves. Using some of the Safe City money, we used the services of a translator who could do simultaneous translation, and we sought their feedback.

What we found was most of the day laborers were people who had been there long term. The ones who were causing most of the problems were the folks who were coming in working for a week or two and then moving on. So the day laborers, obviously they didn't want us there, and they didn't want us there certainly anymore than we wanted to be there, but they knew that these transient day laborers were the ones causing the focus, causing the emphasis to be placed on them and really creating those issues.

So they came up with a number of suggestions, looking at ways to perhaps reduce that. Again, we had the signs identifying that it was an alcohol and a drug free zone. You know, a number of the complaints being people were drinking alcohol which caused them to urinate in the bushes, obviously created some issues.

To help the day laborers in what they were trying to accomplish, we made signs that businesses posted in the area directing them to a certain location on the southeast part of the day laborer zone.

We also implemented some traffic calming measures because we restricted the hiring of the day laborers to a small area, and we were able to do that with the help from a then unbeknownst to us, or initially unbeknownst to us, California vehicle code section which said solicitation was unlawful within 500 feet of any off ramp from a freeway.

Now, we interpreted that to mean 500 feet radius. So that conveniently happened to push the day laborers into an area on the south side of the map. You can see — if I can use this nifty little laser — the little white highlighted area, not the entire area, but on the north end of that white area is where we were able to restrict the day laborers from lurking, had a significant impact on the problem in the area.

I'll tell you what really had the greatest impact was the first meeting we held between the day laborers and the businesses. We had one business owner who was particularly vocal. We were a little concerned when he showed up to the first meeting because we thought there was going to be some pretty significant issues between him and the day laborers. He was clearly frustrated. He explained to the day laborers that he didn't want them standing in front of his driveway because it was intimidating customers and went on and spoke about a number of the concerns he had.

The following day, the day laborers had self policed themselves, moved away from the driveway and to this day have not gone back. So, again, the communication was a huge part of the success in that part of the project.

OK. A few more of the project accomplishments. Some of the security procedures were revised. We did meet early on with representatives from the big three: Target, Wal Mart, and then a strip mall to the north, South Bay Marketplace, to look at ways of joining security forces or coming up with some consistent ways of doing business. Wal Mart revised their security procedures. They were pretty close lipped about everything that they did, but suffice to say they did change their procedures, took a more proactive approach and had an impact as a result on robbery.

I'm going to wrap this up in the next few minutes. Thank you.

We also worked with a sober living facility to provide services to some of the folks in the area who had substance abuse problems and implemented most of the more than 50 CPTED recommendations.

So, now, Nancy had talked about our evaluation, crime being reduced by 23 percent, robbery seeing the most significant reduction, 53 percent, and then vehicle theft. Based on the survey, I should say, pre and post, we saw that the panhandling, the unwanted people, unwanted persons in the area was down as well.

Strengths. I talked about the communication, far and away the greatest strength, and the partnership that developed, but also having an active steering committee was one of the big benefits because they were the folks who really drove this process.

And, again, we didn't rely on the technology. We did have a vendor come out and we walked the Safe City area, got a bid on cameras, looked at erecting 12 cameras. The bid they gave us is about $350,000, cost prohibitive at that time. We felt that the efforts to raise the money would have taken away from the other things we were trying to accomplish, and besides, Karin is an intimidating person and talked me out of it.

[Laughter.]

Wedge: Some of the challenges. Projects overlapped in the two cities, didn't create a number of problems, but when we went to a PBID process — I'll talk about that briefly — it did create a few issues.

Personnel changes. Because of the economy, our city staff has gone from about 1,300 at its peak 3 years ago down to about 900 now. With those changes came changes in the police department. We lost about 8 or 9 percent of our sworn staffing. We eliminated our community relations unit. Fortunately, we've been able to save one person with some grant funding, but because of that reduction in staffing, some of the projects took longer to accomplish from that, and that resulted in a frustration among some of the partners, which in turn led to that loss of momentum.

And then, finally, where we're at now, we are, more or less, in a maintenance mode. We are still looking at some of these projects. One of the things we were going to do for the auto theft is subsidize a kill switch. We found in 2006 about 30 percent of the cars that were stolen in the Safe City area belonged to employees. So we were going to use some of the Safe City funding and work with the Best Buy business in the Safe City area to do low-cost kill switch implementations.

And where we're at now is working through a PBID process, property based improvement district. It's really just a self assessment where people can use the money for environmental enhancements, marketing, security, area identification, things like that.

And then, at some point in time, we hope to transition the leadership from the police department taking the lead on everything to where the steering committee is now driving more of what's happening in the Safe City area.

And I know we're going to do questions later on, so I will back out of that. Thank you.

Winnie Reed: Thank you, Gary.

[Applause.]

Tara H. Wildes: I cannot talk as much or as well about Jacksonville, Florida, as Gary could about Chula Vista. We are the homicide capital of Florida right now as we speak. We're also number one in several sexually transmitted diseases, and I can think of lots of other number ones that may not speak so well for where Jacksonville is in the Florida hierarchy.

We're not as famous as Orlando. We don't have Disney. We're not a tourist destination like Miami or Tampa. We're just this little spot up in the northeast corner of Florida, and we got the Jacksonville Jaguars. We've got an NFL team. Yay! That's about our claim to fame.

But we are a good sized city. We are one of the six major metropolitan areas in the state of Florida.

There it is. As you can see, it's a lovely facility. We built it right on the river, prime real estate property, and we did this before the real estate boom. So that kind of tells you where sometimes the thought processes of our particular community are.

In 1991, we built this facility. We didn't want to do direct supervision, even though that was the cutting edge corrections process for the day, but that cost too much in personnel. So we built this podular design facility, which is if you can remember the old Mad Max movies, the “Thunderdome,” that's kind of what the dorms look like. I'll show you that in a little while.

And, as you can see, our bed count, our capacity is 2,189, and it's currently, as of the day I made this slide, was 2,733. Now, you can address that by adding more bunks. One of the things we did when we first started this project was we had what was called “boat beds.” They're actually plastic portable bedding units, but “PBUs” were too hard to say, so we called them “boat beds.” And those were temporary beds that we sat inside the day rooms and the dorms for inmates to sleep in to help resolve some of the overcrowding.

Eventually, we added a third bunk to the already double bunked cells, and that, again — we've got the capacity now of a little bit over 3,000. So, even now, we're approaching that.

In 2006, we met with the Urban Institute, and I won't get back into what Nancy described as the whole backup of the research project. A lot of correctional facilities do not get involved in research because they don't see any end benefit to them. They see it as additional work with, you know, you're not getting paid additionally from a grant. You're not really gaining anything from it except you get a lot of hassle from people coming in, talking to your inmates, talking to your staff, and sometimes finding out things you really don't want to know. However, from my perspective, I can say that there are a great deal of benefits to that.

Number one, you need somebody to come in and tell you the things that you really don't want to hear. You need that independent perspective to come in and help you improve your system, and that's one of the things, one of the benefits that we have got from this process.

When we first started on this project, we found that there was not a — as I highly suspected, there wasn't a lot of sexual assault within our individual facility. There was some, but real legitimate assaults, inmate on inmate assaults were very rare.

However, I'll tell you a bad story. Right before we got started on the implementation part of this project in September of last year, we had a real sexual assault occur inside one of our cells, and this is a tragic story, but it just illustrates why it's important to follow up on these kind of things. A lot of times people think, ah, they're inmates, they're in jail, they get what they deserve. This was a young man. The victim was a young man. He was arrested for grand theft. So he's placed inside a felony cell, triple bunk cell with two other felony defendants. They're all classified as pretrial. They're all nonviolent offenders.

This young man was in jail for grand theft. He took a gold bracelet from his brother and went and pawned it and bought an iPod with it. Well, dad said, “I'm going to teach you a lesson,” and he called the police and reported the theft of the bracelet, and the young man was arrested and he went to jail to teach him a lesson.

Well, he got taught a lesson. He was sexually assaulted by one of his cell mates pretty brutally. He didn't tell anybody. The only way we found out about it was when the other cell mate who did not participate in the rape came forward on his way out of the system three days later and said, “Hey, you need to check on the kid in my cell because the other dude had his way with him.” We went up, investigated; sure enough found out that it happened.

Now, that's a bad story. The kid was bonded out the next day. Family decided, I guess, he had learned his lesson at that point, but anything that we can do to prevent that sort of thing from happening in a facility — our facilities are supposed to be safe. They're also supposed to be humane, and, yes, they're also supposed to be a place that's designed as punishment, but that's not the kind of punishment that it's designed for.

So, based on that, I was really delighted that we had the opportunity to try to do something to prevent that kind of violence from occurring in our facility.

Now, everybody up here has said something about the changes that have taken place during the research project. One thing you can say about people doing research with people, with human subjects, nothing is ever static. Everything changes. During the course of this project, we've increased, and we've upgraded our cameras. We've gone from an analog to a digital system. We've added recording of our recreation areas. We've started a jail intelligence unit where we track security threat groups and house high profile defendants in a different manner. We've improved our mental health care. We've replaced items that are used as weapons with more difficult to modify models. We've gone to a much smaller razor, for example, that's a lot harder to be made into a shank. We've gone to one piece mop buckets. Everything that you think of that can be made into a weapon; inmates can make it into a weapon.

And something else, too, and this will come up a little bit later, but no matter how rugged it is, if it can be torn up, an inmate can do it, and an officer can do it better.

Here's the way we used to keep data. It really wasn't quite this bad, but it was really —

[Laughter.]

Wildes: This was close to it. This was real close to it. When we started the project, we had a very flat data collection system. Our incident reporting system, we wrote reports for everything, but we collected very small bits, just the very essential pieces of data. The data that was required by law was all we kept. So it was really hard to pull some of the things that we needed to analyze out of that system.

Well, now, this is just an example of just one of the reports that comes from our reporting system, and Sara and Nancy haven't even got a chance to see some of the — because that's recently changed, but now we can get virtually every data element involved in incidents. I mean just common everyday incidents, the finding of contraband, inmate fights, of course, sexual assaults and all of the major things, but anything that we have to write an incident on now, we're able to. It goes into our data warehouse. We're able to slice it and dice it any way you can think of. Plus, that all rolls into our intelligence unit as well, and we are really able to do some really neat tracking of problems throughout the system that way.

One of the things that we found as recommendations from the Urban Institute's review of the surveys that they did was that we did need to revise our classification system to more of a behavioral based process. We needed to implement more staff rounds tracking, and that's the piece that we finally decided to focus on because what we found was what constitutes a round was different in the eyes of some correctional officers as opposed to the eyes of the correctional administration. When we would say make a round, we meant, you know, get up, walk, you know, out of the pod, down into the cell, walk around the cell, look around, make sure everybody is OK and walk back out. What they said, “Oh, we did a round. It was a visual round. Looks good,” and that was how a lot of cases — and they would type it in the system, did my round. And we found that there was a disconnect, and it wasn't just the fact that they weren't doing their rounds. It was that some were doing more rounds than others, and if you think about it, how that creates a conflict between officers, between staff at the line level, because some people are doing more work than other people, some people are getting away with doing less work or what's perceived to be less work.

So we decided if we implemented the rounds tracking devices, we would level that particular playing field, and we would also get some good quantifiable data about how often the officers are down in the dorms and interacting with inmates.

This is a view from inside the dorm. You're on the upper level of the dorm looking down. That big area there is the day room. These dorms, this particular one would originally have been a 48 person dorm, but now that we've triple bunked it, it's a 64 person dorm. And that's how large each one of the dorms are in the facility, and there's six floors in the facility.

Now, here's where the officers work. This is inside the pod, looking out. As you can see — now, it's a little bit glarier than it actually is because when we're actually in there, we turned the lights down, and you can see into the dorms better with those lights down. You don't have so much glare, but, still, that's — and there's four of those large dorms around each pod. So you can imagine that's — there's ample opportunities for inmates to get away with stuff. You've got one, maybe two security officers who are making rounds inside the dorms, and then you've got one officer up here in the pod who essentially are watching the officers making the rounds. And you've got now probably an average of 300 to 400 inmates on each one of these pods, and you can see that it's not easy to keep up with what's going on.

We talked already about the reasons why we went to the rounds tracking system, and we looked into a lot of different products. I would have loved to have had the full service RFID which is where it would just essentially track the officer everywhere they went, but you're talking upwards of a million dollars to install a wireless system, an RFID system in a large aging facility that we're likely going to move out of within the next decade. And I couldn't convince the budget powers to be that that was something that we needed to go with; however, we were able to get the budget model, which is the pipe, which essentially is just a series of those buttons. And you just have to go. As an officer, you go and you have to be within the proximity of that button to record your round, and then those rounds are then downloaded later on.

It's not a real time system, which I was disappointed in that. I wanted something that would give us that data right away, but now they go and they downloaded at the end of their shift, and we're able to see what kind of rounds they're making.

So we're still in the process of refining our reports, and the good thing is we've had a couple of issues that have come up right away that we've been able to determine things didn't happen because we were able to prove officers were in the proximity when things were alleged to have been occurring inside of dorms. So that's been one positive out of it.

We've been able to eliminate some manual data entry, some typing in of rounds and things like that on the computer because now they're collecting that with the pipe system.

The anecdotal information from the inmates has been really very positive. Again, this is just the inmates telling us things, but they say that they do have more access to the officers. They say that there is less commissary theft. One of the other recommendations was for us to try to get a place that the inmates could lock up their personal belongings, and we're still trying to come up with a solution for that. Whatever solution that we seem to come up with that we think that we may be able to work out, we've not been able to get funding to do. So we're still trying to look for a budget solution to that as well. But, in the meantime, the inmates are telling us that the officers making the increased rounds are holding down the commissary theft. And, again, like I said, some of our personnel have said, “Hey, I like it because Joe over here who won't ever do nothing, he's having to do the same amount of work that I'm having to do now.”

Now, the challenge is they told us when we got this pipe system, oh, they can't tear them up, they're in correctional facilities all over the United States, we don't have any problems, they're extremely rugged. Well, we have put their ruggedness to the test, and, in fact, right now they're talking to us about voiding our warranty on the products because we have had so many of them that we've had to have replaced. We're working through that with them.

We've had some negative reaction from the corrections personnel, and I think maybe that's why some of them haven't been quite as rugged as advertised. I think we've had some officers who may have been overly enthusiastic in their use of the devices as far as, you know, their hitting the button, so to speak. “Oh, I thought you had to hit the button.” No, you just have to get it in the proximity of the button.”

And we're also working through a process of how we are addressing. When we pull those reports and we see rounds weren't made in a timely manner, what is our response as management? Is that something that we look at for retraining? Do we begin to take disciplinary action? Do we progress in that disciplinary action? And what we've done from this point is we've looked at the first — we take the first instance that somebody isn't making their — somebody is not making their rounds properly, and we address that with counseling or training. And then from that step on, we will move with progressive discipline. Now, if there is a reason why they cannot make their rounds, there is an incident occurring, they are, you know, feeding up or they're doing something else, then they have a way of recording those things, so it's not an unforgiving system, so to speak. There are ways that that can be done, but yet they are responsible for making those rounds, and they're just going to have to get them done.

One of the other things, too, that made the project much more acceptable in the officers' eyes is throughout this research we included them in on the decision making process. We made sure that we had personnel, the personnel that would actually be using those devices in on the research. We had them in on the testing of the product, and we also had them in on the writing of the policy and the procedure and the eventual response to, you know, misproduced rounds. We had them come in and be completely involved in that, and that was a big help because I think when you make a systemic change like this, if you do not have that buy-in from the line level personnel, it is very easy for them to sabotage a project or prevent it from working as well as it could have.

So, hopefully, what we're going to do in the future is we are going to analyze the incident data. We're hoping that it will show a reduction in violence, in self harm attempts and things like that within our system. We do want to continue and add the inmate storage for personnel property, and we do have plans as well, budget permitting, to expand our camera system as well into the inmate housing areas. We currently have a camera system that watches all of our processing areas and our releasing areas, which is where our highest propensity for violence occurs, staff versus inmate type violence within the facility. Now we need to expand into those areas where inmate versus inmate violence has the highest propensity to occur.

So thank you, and you guys come see us.

Winnie Reed: Thank you very much.

[Applause.]

Andy Mao: [In progress] — notwithstanding the number of elder abuse, neglect and exploitation cases around the country that occur each year, the number of elder abuse prosecutions varied widely around the country.

This esteemed panel today will be focusing on perhaps some of the reasons for that variation and some of the factors that go into elder abuse prosecutions.

To my immediate right is Dr. Shelly Jackson. Dr. Shelly Jackson is an assistant professor at the University of Virginia in the Department of Psychiatry and Neurobehavioral Sciences. She has written and authored extensively on a wide array of issues involving child abuse, domestic violence. Recently, she has focused a lot of her attention on — focused and presented on issues involving elder abuse and mistreatment as well as financial exploitation. Dr. Jackson will be focusing today on a study, an NIJ-funded study, on some of the factors that go into a prosecutor's decision on whether to prosecute an elder abuse case.

To Dr. Jackson's right is Lori Stiegel. She is the senior attorney of the American Bar Association's Commission on Law and Aging and has specialized in elder abuse since 1995. Lori was a member of the National Research Council's study panel on Risk and Prevalence of Elder Abuse and has authored and co-authored extensively on elder abuse, a number of articles including Elder Abuse Detection and Intervention: A Collaborative Approach, Elder Abuse in the State Courts: Three Curricula for Judges and Court Staff, as well as The Power of Attorney Abuse: What States Can Do About It. Lori will be focusing on some innovative court models that have tailored themselves to deal with elder abuse cases and will be focusing on that link in the elder justice chain.

Last but not least, to my far right is Page Ulrey. Page is a senior prosecuting attorney with the King County Prosecutor's Office in Seattle, Washington. She started there in 1998 and in 2001 was appointed to the newly-created position of elder abuse prosecutor in the criminal division. During her time there, she founded and chaired the King County Elder Abuse Council and the Criminal Mistreatment Review Panel. Since 2007, Page has worked as the elder abuse prosecutor in the Economic Crimes Unit and has focused on cases involving vulnerable adult financial abuse, neglect and sexual assault.

With that, I am going to turn it over to Dr. Jackson, who will start us off on her presentation.

Shelly Jackson: OK. Good morning. I am going to be talking about some barriers and facilitators in prosecuting elder abuse in a pilot study that we've done. I want to acknowledge my colleague, Tom Hafemeister, who's the co PI on this project. He's also my husband, and he and my kids were upstairs still sleeping when I left.

[Laughter.]

Jackson: And then, of course, we'd like to thank NIJ for funding the project.

Now, before I get into the prosecution part of this, I wanted to tell you about how we became interested in prosecution. We were funded a few years ago to study financial exploitation of the elderly, and what we did was we looked at financial exploitation by comparing it to other types of abuse, like physical abuse, neglect and what we call “hybrid” cases. And that's where there's financial exploitation and something else going on. And you can see by the title here that we were interested in the dynamics, the risk factors, and when we conceptualized this study, we were thinking about Adult Protective Services as society's response. Well, we were gathering our data, we were running our analyses, and we found out that 18 percent of the cases in our study were prosecuted and received a conviction for elder abuse. They might have been prosecuted on something else, but for elder abuse, it was 18 percent of the cases. But the physical abuse cases were more likely than expected to be prosecuted compared to those other four types of abuse that was going on.

We also learned at that time that our caseworkers were finding it very difficult to get prosecutors to help them out in those cases. So almost three-fourths of our caseworkers said, “You know, if I go to law enforcement, they're less likely to help me in a financial exploitation case than a physical abuse or a neglect kind of case.” But then just in general, they were extremely frustrated with how prosecutors were responding to their cases. And so we engaged them in a conversation about why that might be, and you would hear things like, “Well, the elder actually told her son that she could — her son that he could write checks out of her checkbook,” or you've got power of attorney issues.

Elders make poor witnesses. If it is not $100,000 or more, prosecutors aren't going to take the case. And these are things that you would hear in the literature as well when you do a review of the literature; they're really reflected there as well. And that got us to thinking about what's going on here, and it seemed to us that the caseworkers have this perception and belief; they really believe that prosecutors aren't going to take their cases. So that leads them to think, “Well, why bother? I'm not going to send it over to the prosecutor anyway,” and this was particularly true in financial exploitation because what was happening with those kinds of cases is if there's physical abuse and financial exploitation, they might pursue physical abuse thinking, “Oh, prosecutors might take that, but I'm going to drop the financial part because nobody's going to do anything about that anyway.”

And then this is really anecdotal, but then we would go to conferences, and I would hear prosecutors say, “You know, elder abuse isn't a problem in my jurisdiction. Nobody ever sends me those kinds of cases,” and then we're thinking wait a minute, what's going on here, there is some kind of miscommunication between these players. And that led us to go back and think we were initially thinking about society's response in terms of Adult Protective Services, but we said, “Wow! What prosecutors are doing is directly influencing what our caseworkers are doing, what they're willing to pursue,” and so we went back to NIJ and asked whether we could expand our study a little bit to try and start figuring out what prosecutors are thinking when elder abuse cases come their way.

And so we've done a pilot study looking at some barriers and facilitators, and that's what I'll present the rest of the presentation on.

OK. So this is a pilot study, and we had interviews with prosecutors in four states, Virginia, California, Illinois and Pennsylvania, and I'll tell you why those states were chosen. But we had 17 prosecutors, and as I said, this is a pilot study. They had 13 years of experience on average, so, in general, they had some experience in prosecuting. Almost half of them had a hundred or more cases, which I can't even imagine what that's like, but maybe you can speak to that.

And almost three-fourths of them were not what you call a dedicated — I'm not sure exactly what the term is, but elder abuse is not the only thing they do. So, in almost three-fourths of the cases, they had 25 percent or less of their caseload was elder abuse.

OK. And then we created an interview instrument based on a review of the literature, some of the things we talked about before, but many, many other things, demographics and experience and then these facilitators and barriers to prosecution.

Now, we had five UVA law graduates. These were, I will confess, students of my husband's whose jobs were either deferred — this was during the economic downturn. People weren't walking out into great jobs. Their jobs were either deferred or they didn't have job, but they were going back to their states, to California, Illinois, Pennsylvania, and that's why we had those states. I mean, it was a convenient sample for sure.

So, once they would initially get the prosecutor in the office to agree to an interview, then we just used a snowball technique, “Is there somebody else that you know that we can interview about this?” And, on average, the interviews were 40 minutes. We tried to get them shorter, but we couldn't. There was a lot to talk about.

OK. So these are some of our results, and the first thing we wanted to know is whether it mattered to prosecutors if the case came from Adult Protective Services, whether the referral came from law enforcement or didn't matter, and in two-thirds of the cases, it didn't matter, but where there was a preference, it was for law enforcement, and that, of course has to do with evidence. They're generally seen as better at collecting evidence, and this is kind of the theme that we're going to hear throughout this, it depends on the evidence, and I think Page is going to speak to that as well.

OK. And then we asked them is elder abuse more difficult, the same, or easier to prosecute than other kinds of crimes? And I think it's 56 percent said that it is more difficult. And when we were talking to our caseworkers, over half of them thought that these were difficult cases to prosecute.

And then we got into why that might be. So we're going to look at a number of factors that might influence why these are harder, and one of the things we were interested in is whether the organizational climate of the office, the agency that the prosecutor worked in, whether they take these cases seriously, and 94 percent of them do take them seriously. A third, a quarter of them thought that the prosecutors in their office were even more willing to take an elder abuse case than other kinds of crime, and that they were more likely to receive a conviction in about a third of those cases. And that had to do with things like sympathy for the elder.

And then almost half of them perceived, anyway, that they were rewarded for taking elder abuse cases. And Pennsylvania was significantly more likely to feel that way, and Virginia was significantly less likely to feel rewarded for taking elder abuse cases.

OK. And then we were interested in prosecutor training, and 41 percent of our prosecutors received less training in elder abuse compared to other kinds of trainings that they might receive, so training might be an issue.

And then we have another, a range of issues that we asked them about, and we just put them all on one slide here. So you often hear elders make bad witnesses, and you can see 93 percent of our prosecutors felt like elders make poor witnesses; the elder is unavailable to be a witness. There is weak evidence in these cases. Perpetrators acted within their rights, and that is often what you'll hear with power of attorney, right? There's nothing. She signed it. He can do what he wants to with it, “he said/she said” kinds of cases.

So, over half of the prosecutors thought that these were real issues that they have to deal with. Less so, though, were uncooperative third parties, it's not perceived as a crime, the prosecutor is unqualified, or there's insufficient resources in the office. And we saw before that the offices that we talked to, anyway, generally take these cases seriously.

And then we asked about a range of victim characteristics as well, whether these would increase the willingness to prosecute a case. So, willing to testify, significant harm was done to the elder. The elder has the ability to testify. They press charges or the elder made the complaint or the elder is not deceased, and what do most of those have to do with, except for the harm, they really have to do with victim cooperation. And I think that's what prosecutors are seeing when they think about are they willing to press charges, are they willing to testify, that that signals to prosecutors whether the elder is going to be a cooperative witness. Now, less influential is whether the perpetrator is a stranger or a close relative.

And then some personal characteristics of the victim had to do with things like mental health, whether the victim is living — we just saw that — whether they're articulate, good cognitive status, their physical health. These were a little less endorsed than the previous group of characteristics, but still there is a third to a half that consider these important factors as well.

What we did find, though, is social network, education, age and gender were not endorsed by prosecutors, but you can kind of see that those would be things like sexism, ageism, classism, and so you would probably not expect those to be endorsed, I think, but probably related to some of those other characteristics, I would guess.

OK. We also asked them an open-ended question, whether there were any perpetrator characteristics that might be influential in deciding whether to take these cases, and generally, they said no. And here, again, it depends on the evidence.

One person said family members are less likely to be prosecuted, and another said nonfamily members are more likely to be prosecuted. And, in our study, we definitely found that relatives were less likely to be prosecuted, which is not surprising.

OK. We were also interested in law enforcement because very often the case comes to prosecutors through law enforcement, and all of our prosecutors thought that law enforcement take elder abuse cases seriously.

And, in Virginia, we heard some variations on that, and so that's why we decided to ask that question. But the prosecutors in our study thought they take elder abuse cases seriously and that they receive good evidence from prosecutors. And almost three-fourths of them, of the prosecutors, said that APS is involved in their cases, and when they are involved, 85 percent said that they receive really helpful assistance from Adult Protective Services, that they like Adult Protective Services being involved in their cases. And part of that has to do with kind of, you know, figuratively speaking, but holding the elder's hand through the criminal justice process, that they have much greater skill in doing that than some others.

OK. So we also asked them, though, are there any ways that law enforcement and APS might improve what they're doing, and, of course, they had some suggestions, such as collaboration, and this is one of my big themes. I've been in the world of child advocacy centers for over a decade, and I'm very fond of the multidisciplinary approach. And I'm going to come back to that at the end. I think it's one of the keys.

I thought it was really interesting, though, that they thought that law enforcement and APS should be more proactive, given that because the APS at least perceives they don't take those cases, they don't bring them to the prosecutors, document everything. And then also they thought law enforcement needed some training in interviewing individuals with cognitive deficits, and we heard from our caseworkers that law enforcement is very impatient when they interview elders. They like everything in a nice, neat package, and it takes elders a little longer to kind of get the story out, and they get frustrated and leave.

And then the limits of power of attorney authority, and I think this really has to do with Virginia. Virginia just passed a uniform power of attorney. Lori wrote that great, great, great article — or monograph? Anyway, on power of attorney, and so Virginia just passed that. It goes into effect July 1. So that will be less of an issue for us, but it's going to take a lot of training because we heard from our caseworkers that power of attorney cases, law enforcement doesn't do anything with those. So it's going to take some education to get them thinking in that direction.

We were also interested in whether we needed statutory reform or whether those statutes were working fine, and 81 percent thought that their statutes were helpful to them. Virginia, however, was significantly less likely to find their statutes helpful, and I think that has to do with their financial exploitation, lack of statute.

OK. We also wondered if there were some legal constraints that prosecutors were working within, and, of course, there are. There's the confrontation clause, and with our elders who become incapacitated for various reasons from cognitive deficits to death, confrontation clause becomes a big issue and also a lack of hearsay exception, so if we can't get the elder up on the stand, are there any ways that we can get information from other people into the courtroom. So those are two pretty huge barriers.

Power of attorney laws, I just mentioned, and narrow neglect statutes, and we're going to look at neglect in a minute specifically, and those seem to be really tough cases.

OK. And we have identified a few court reforms, and we were interested in whether prosecutors thought these were helpful, expedited trials, enhanced penalties for the perpetrators, of course, priority on the docket and accommodation for elders. And those were generally, over 60 percent, over two-thirds thought that those were really useful reforms. But you can see the expedited trials, priority on the docket, that's getting the elder in there as fast as we can, so that we can make sure we get them up, get the case in the courtroom.

And then, in terms of investigation and prosecution reforms, we've got some things going on there. Multidisciplinary teams were endorsed by, I think it was, 88 percent of the prosecutors. As I said, I'm a big fan of the specially trained prosecutors, equally powerful, specialists in the office, in the prosecutor's office, to aid investigation, the availability of social service agencies, these forensic centers — have you all heard of elder forensic centers? — which is another kind of multidisciplinary approach, and vertical prosecution.

But what you can see, I think what a lot of those reforms have to do with, is getting elder cooperation, victim cooperation, right? If you've got vertical prosecution, you've got the same person kind of bonding with the elder and helping them through the system. So I think that's what is at the base of a lot of these reforms, is trying to get our elders.

What I thought was interesting is no-drop policies and mandatory arrests were not nearly as endorsed, but I will tell you, in our study, with the physical abuse cases that were prosecuted, it's not that our elders wanted those cases prosecuted. They didn't, none of them — seventy-five percent of our elders didn't want prosecutors involved at all — and particularly the physical abuse, but what happened is they're in this brawl with their perpetrator, and they call law enforcement and they come out, and the elder says, “No, I'm sorry. I don't really want to do this,” but law enforcement says, “You either press charges or we will, but we're not dropping it.” And those are the cases that ended up the most likely to be prosecuted. It's because there was law enforcement involvement. I know we can argue the merits of whether that's a good thing or bad thing, but that's the explanation for why these cases were prosecuted.

OK. And then we asked them open-ended questions about prosecutor — about other kinds of reforms, and I think this is reflective of what we've just been talking about, that we need some hearsay exceptions in those cases where elders become incapacitated or die. And this one, I'm hoping Page or Lori can speak to a little bit more, but the appointment of a defense attorney and then cross-examine the elder before the defendant's been identified and before the elder dies, but you still can't cross-examine that.

Page Ulrey: Well, individual defendant is entitled to a right to confront the witnesses against him. It's not just a generic right.

Jackson: Yeah. Yeah. So it didn't seem like that would work, but, OK, good.

And then several people identified public education — that we need to get our elders educated about elder abuse and make them proactive in their own safety.

And then some statutory reforms, power of attorney laws, that was probably Virginia attorneys or prosecutors; widen the category of neglect and financial exploitation and harsher penalties.

OK. Now, if we do look at whether prosecutors think these types of elder abuse are harder or easier or the same to prosecute, 73 percent said neglect are harder to prosecute than other kinds of crimes, 56 percent thought financial exploitation were harder to prosecute than other kinds of crimes, and 43 percent thought physical abuse were harder, but you can see for physical abuse, actually, about half of them also thought that they were easier to prosecute.

So the neglect which we just heard somebody saying that those are really difficult cases to prosecute is reflected here as well. But, as I mentioned in the beginning, we were very interested in financial exploitation, and so what we did is we tried to dig a little deeper into financial exploitation.

Oh, I forgot. This slide just shows that Virginia was more likely than the other states to find financial exploitation very difficult to prosecute, and, again, that's because they don't have any kind of statute that allows them to pursue those kinds of cases.

OK. Now, we also heard from our caseworkers that prosecutors are unlikely to take their cases unless it involves a lot of money, but the prosecutors in our study said there's really no minimum amount required. It depends on the evidence, of course. But 20 percent of the prosecutors felt weaker to pursue a financial exploitation case, and this is what I was thinking earlier. Our caseworkers also thought that financial exploitation cases were harder for them to investigate as well. There's some different kinds of issues involved with financial exploitation.

And I also mentioned what we call “hybrid cases” or co-occurring financial exploitation and something else. Our caseworkers thought that about 50 percent of the cases that they get involved co-occurring financial exploitation and some other kind of abuse, but it looks like for the prosecutors that happens in less than 25 percent of the cases that at least make it to their office.

Half of them said that really doesn't matter if there's more than one thing going on, but there were a third of them who thought that it was more willing, and I think that that suggests that they were more willing to take those kinds of cases because there's a lot going on in those kinds of families, and we found with our hybrid cases that they were the most intractable cases. They involved family members. The abuse has gone on for decades, and it's really hard to get out of those kinds of cases, get out of that situation.

OK. And then some factors in deciding whether to pursue financial exploitation. Some of that had to do with the amount of evidence, but, generally, we heard that there's a lack of documentation, a lack of corroboration, and a lack of victim participation, that they're less likely to get involved in these kinds of cases. But one prosecutor kind of summed up what everybody else was saying, which is, “I have to prove the elements of the crime to the degree required by law,” and that's what it comes down to, and that the evidence in financial exploitation cases tends to be weaker.

OK. What are the goals in pursuing financial exploitation? We want to hold perpetrators accountable. We want to make sure that they don't harm others. We want to restore the victim's loss, and we want to make sure — we want to send a message to society that this is not OK to do, so deter other people from committing these kinds of crimes and then provide a sense of justice. So you can see that over two-thirds of the — I think if we did this again, we'd kind of try to rank-order this a little bit because, obviously, these are good goals.

Now, back to restoring the elder's loss, though, we heard from our prosecutors that they always ask for restitution, and in 75 percent of those cases, the elder is awarded some restitution, but, indeed, less than 25 percent — I mean, the elders received less than 25 percent of what they'd been awarded, suggesting that if you were going to be financially exploited, it is very unlikely that you are going to get anything back. And that was true in our study as well. Eighty-four percent of the elders in our study got nothing, and that's not restitution. That is just of the money they lost; 84 percent received nothing back.

OK. So, my conclusions, I have five minutes, and I'm good.

So over half of our prosecutors thought that elder abuse is harder to prosecute, that it comes down to evidence. And one of the things that we're really interested in is victim cooperation and how to get victims to cooperate, and I know there's a number of pieces of evidence that you can use, but victim cooperation seems to be a stumbling block in a lot of cases.

So some of the barriers have to do with elder characteristics, prosecutor training, the legal constraints that we mentioned, and that neglect and financial exploitation seem to be particularly difficult to prosecute, but there were facilitators, elder characteristics again. They can either be a barrier or a facilitator, depending on how they come out, but APS and law enforcement involvement seems to be helpful and useful. Most state statutes are working well, and that the reforms that have been instituted — and I know Lori has written a number of articles on reforms, and that's certainly where we got a lot of these, but that they've been helpful. They are perceived by prosecutors as being useful.

OK. So, I end with this idea about a multidisciplinary response, because what we are interested in is trying to figure out how we can increase victim cooperation. And we certainly saw, as I mentioned, our elders, 75 percent of them didn't want prosecutors involved in their case at all, and one of the things that a multidisciplinary response can do is kind of help that process along, I think help the elder along. So you've got Adult Protective Services who knows how to talk to elders. They come in with a social service kind of agenda, “I'm here to help you.” Law enforcement goes in and they get the evidence that they need, and the prosecutor says, “I need this from you. I need this from you,” and I've seen this work in child advocacy centers, and I just really love the idea of using this model with elders. And I know I've been out to California to their elder forensic centers, and I've watched how the multidisciplinary approach works in those cases. It's phenomenal, and we would love to do some more research on multidisciplinary response to elder abuse.

OK. I think that's it.

Lori Stiegel: Good morning. I want to thank NIJ for funding our project and for asking me to be here today to talk about it, and thank all of you for coming and expressing your interest in this subject.

Our project entails an assessment of five court-focused elder abuse initiatives. We called them that because they're all a little bit different. One is actually a court. The other four are connected to in or just very closely connected to and involved with courts. So we just had to make up our own term to describe them, and I will tell you more about them in a while.

So, first, just to give you an overview of what I'll do in my time today is talk about our goal, how we did our project, and then for some context as well as to give you the measures that we used to assess the five projects, I'll talk about barriers to pursuit of legal remedies and guidelines for the state courts, and some of it will be what we just heard Shelly talking about, too. I'll describe the five court-focused initiatives and our findings from them and then just talk real quickly about next steps.

So, our goal, this is the long statement of the goal. Our goal was really to look at what these five projects are doing. Because there is no baseline data about any of them, and we're not doing an evaluation, but we're really looking at their processes, their structure, to some extent their outcomes, and really trying to make some assessment of whether these are good projects that should be replicated in other communities and other states.

Those of you who've been involved in the elder abuse field a lot know that because of the lack of research in the field, people come up with what they think are good ideas and often are good ideas, but then everybody is like, “Oh, we have to go do this,” and there is never any research to determine whether that's really a good idea and whether they are making a difference. So we were, to the extent that we were able to, trying to do that.

My colleague, Pam Teaster, from the University of Kentucky, was our subcontractor and co-PI on this project. She's the researcher; I'm the lawyer, so we made a good team. We had a great multidisciplinary advisory committee, and Page is actually one of the members of that advisory committee.

We used the court-recommended guidelines that Shelly talked about and that I'll talk more about a bit and some other guidelines that were created after our early project as the standards for assessing the five projects, and we've looked at an enormous amount of qualitative and less enormous amount of quantitative data for this project.

For each of the five initiatives, we identified a key informant that was usually the founder or the manager of the program. We then did a very in-depth survey with those people to get background and a lot more information than we had when we wrote the proposal about the project, how it was working, who was involved with it, and then used them to snowball and identify stakeholders in the five communities, so that we would know who to interview when we went and made our five site visits.

So we interviewed 92 stakeholders across those five sites visits. We talked to the chief judge in each community. We talked to other judges who were involved with the project. We talked to court managers, court staff, court clerks, all kinds of court people. We talked to Adult Protective Services. We talked to law enforcement if they were involved with the project. We talked to the prosecutors. We talked to public defenders. We talked to lawyers in private practice. We talked to all kinds of folks at all of those sites.

We talked to three victims as well. We certainly had hoped to talk to more, but even managing to talk to those three was quite a challenge, unfortunately.

We also reviewed court case files, and we reviewed 73 files across the five projects. And we were looking for a variety of things in doing that, and I'll talk more about that later.

I can't tell jokes, but I like cartoons. So, in starting to talk about barriers to going to court, I thought this one nicely illustrates. If you can't read it, it says, “It's bad enough being cheated out of my life savings, but what I really hate is having to get it back in small claims court.”

Older people don't like to go to court. They're very afraid of it, and younger people are, too, but I think older people are more so. Back in the mid '90s, when we did the project that developed the recommendations for state courts handling elder abuse cases, we did a research project. This was funded by the State Justice Institute, which is a quasi governmental agency that provides research and technical assistance to the state courts. And we looked at experts all across the country, including prosecutors, to the extent that there were elder abuse prosecutors in the early '90s, civil lawyers, APS, and all kinds of other folks, judges and other folks, and asked them, “Are elder abuse cases going into court? If not, why not, and what can we do to help get those cases that should be in court into court?”

So lots of barriers to the pursuit of legal remedies were identified. Now, these are the system ones. I think we often talk about the more personal ones, and Shelly talked somewhat about the more personal ones of just that fear of going to court, the embarrassment that, you know, your life savings were so small that you have to go to small claims court to get it back, but that's all the money that person had. The reluctance to get your abusers in trouble, particularly when they are your family members, the fear that intervention will result in you having a guardian appointed or ending up in a nursing home, all those are the personal barriers about going to court or reporting to APS or some other authority. These, again, are the more systemic barriers about the pursuit of legal remedies.

First, victims often just don't know what's available out there, and they don't understand what's available. So we often hear about victims who don't want their abuser prosecuted because they don't want him to go to jail, and they don't understand that judges have many other remedies besides jail that they might want, such as substance abuse or mental health treatment.

The lack of legal services — and this particularly related to civil legal services, but the lack of a lawyer to help you try to get your money back, if it's your money that has been taken, or to get a protection order to keep physical abuse from happening.

Victim difficulty just getting to court, victim difficulty sitting in court.

How many of you have ever been to court? So you know it's a zoo, right? A circus is often an apt description for it, and they have a cattle call, and, you know, lots and lots of people show up for hearings. And you have to get there at 8 or 8:30 in the morning, sort of like today, and then you can sit there for hours before your case is called. And if you are an old person who is frail or you have a medication schedule that you have to keep on or you have to eat at certain times in the day, that sort of environment can be extraordinarily difficult, if not impossible, for you to do.

I remember a case that I handled as a legal services lawyer at the beginning of my career where our client bailed, basically. We went to court, we sat there for a while, and he couldn't take it anymore and he left. And he just didn't pursue his rights because of that.

Victim fear of, ignorance of court proceedings, like I said, older people just tend to really be terrified by this process. And I think particularly when we have people in minority cultures who, whether here in the United States or in countries if they came here from other countries, had very different experiences with the court system and with legal authority maybe than white people have had here, there may be enhanced reluctance and fear of the court system.

Professionals' lack of knowledge — and that's what we heard Shelly talk about and what Page is going to focus on, I think in her comments, but certainly early in the '90s, I think this was a bigger problem than it is even now of prosecutors not have training on elder abuse. Judges didn't have training. Lawyers certainly didn't have training. This isn't stuff you learned in law school back then. You don't learn it very much in law school now, 15, 18 years later.

Where I'll probably focus my comments here is on the disparate impact of court practices. Courts like to think, and they say all the time — and we heard it during our experiences — that they treat everybody the same, and I think they really believe that they do, but the reality whether they really do or not, the reality is that treating older people the same has a disparate, more onerous impact on them than on younger people for the reasons that I just talked about. If you are reliant on public transportation and particularly on transportation for disabled people to get to court, it may literally be impossible for you to get there at 8 or 8:30 for the cattle call. I talked about the difficulty of sitting there for hours. All those things then mean that court just doesn't work for you the same way it does for a person who has a car and the ability to sit on the hard benches in the courtroom for hours at a time.

As a result of finding those things during our study in the early '90s, we developed a set of 29 recommendations for state courts handling cases involving elder abuse. These that are up here represent about half of those. I focused on pulling out the ones that were most relevant to what I was going to talk about today, and these sort of consolidate the 15.

Clearly, in our first recommendations back then were that judges, court staff, and other professionals, including prosecutors, law enforcement, lawyers, and even APS needed to have more training about elder abuse and how the court system, the judicial system needed to respond to it.

Expediting cases. So, here in 2010, Shelly is coming up with some of the same recommendations that we came up with in the early to mid '90s, but, of course, with older people who are more likely to die in the first place and maybe as a result of their abuse or exploitation or neglect, even more likely to die, expediting cases is an important issue for them.

Accommodating persons with disabilities. Some of the ways I have talked about already represented several of the recommendations. Providing emotional support and assistance, helping people get to court, explaining how it works, trying to make them less fearful to overcome that reluctance to pursue their legal remedies, to cooperate with law enforcement or prosecutors, those were several of our recommendations.

Referrals to community services. Very often, older people who have long-standing abuse problems or other problems, too, going on end up in court and have never gone to any other social services program or anything else in the community for help, and it's the court that then ends up trying to refer them to appropriate helping agencies. Well, the courts need to know more about those agencies, then, in order to do that, and that was that recommendation.

Enhancing intra-court communication and coordination. I remember very distinctly Judge Kirkendall, who was then a probate judge in Ann Arbor, Michigan, said — and probate judges, he handled guardianship cases. Most probate judges do but not all. He said, “I can be in probate court deciding whether to appoint son as guardian for mom, and I have no idea that down the hall in the criminal courtroom, son is being prosecuted for having abused, neglected or exploited mom.” And I am going to talk about why that's still a problem 17 years later.

We also recommended that the courts could provide leadership in the community about elder abuse. Much was being done and is still being done with domestic violence and with child abuse. We thought they could and should do the same thing with elder abuse. So that again gives you the context of what the problems are, what recommendations we suggested to help solve those problems, and those were the standards that we then used, as I said, to assess the five projects.

So let me tell you quickly about the five projects. One is an actual court. It's the Elder Protection Court in Alameda County, California, which is Oakland, right across from San Francisco. And it was started by a Judge Julie Conger, who was a criminal trial judge at the time, had been for a long time and had been handling their domestic violence protection order docket as part of her role. And at about the same time that this got started, California expanded its protection order law, so that it became easier for older people to get protection orders against their abusers. Sometimes protection order laws don't really include elder abuse situations very well. They may cover partner or spouse but not cover if it's your kid and your grandkid who's beating you. So California broadened its law, which meant the court was starting to see more of these cases.

So she, out of concern that they were seeing more of these cases and because of the problems that I talked about earlier, decided to work with the court to create a special docket for elder abuse cases. And the way this works is that on Friday mornings, the Elder Protection Court is held. Friday is normally not a trial day in Alameda County, and court starts later than usual, so that deals with the problem there that I talked about. Many fewer people, it's much less of a cattle call than typical court, and it's a much calmer, quieter, softer, gentler process. We sat there, and it was really very interesting how it worked and how respectful it was of older people in that they were a little slow.

The Prosecutor's Office in Alameda County is very, very strong on elder abuse — had a dedicated elder abuse unit before the court was started with several prosecutors — and, interestingly, the Public Defender's Office ended up playing a key role, which we did not expect to find in this situation, and I'll talk a little bit more about it later.

The Elder Justice Centers are both in Florida, and that's about their only similarity there is their name and that they're both in Florida. One is in Hillsborough County, which is Tampa, actually where I used to practice; the other is in Palm Beach County, West Palm Beach.

The Hillsborough County project wanted to become an elder abuse court, and the Florida Supreme Court said no when that idea was developed. And that was sort of early on in the development of specialty courts, and they just didn't want to do that. So, instead, the Elder Justice Center there became a resource center as well as provided victim advocacy services and now is doing much more on guardianship monitoring as well, and I'll talk a little more about findings after I give the description of all of them.

Palm Beach County sort of misled us about what they do, I think. It turns out that they are — they help criminal defendants, and they help them by actually — they help the court in recognizing whether older criminal defendants may have Alzheimer's or some other dementia or some other problem that really means that they are best suited to be dealt with outside of the criminal justice process.

They had told us that some of those criminal defendants were elder abuse victims who maybe were now shoplifting because they had been exploited and needed to steal, for example, in order to live. You know, I think in reality, it turned out that they weren't helping elder abuse victims very much, and I'll talk again more about why not, where they're going, maybe how they are hoping to change that in the future.

The Elder Protection Order Projects are more similar than the Elder Justice Centers are to each other. One is in Jefferson County, Kentucky, which is Louisville. The other's in Kings County, New York, which is Brooklyn. Interestingly, they both started at about the same time. Neither one of them knew about each other until we found out and started — about them and started our project.

What they do is help homebound older people who need a protection order apply for and obtain their protection order from the court without going to court. So they do it telephonically, and both of them have a process whereby law enforcement and a service provider — it might be Adult Protective Services or somebody else — go out to the person's house and do the paperwork and then hook up with the court by telephone and the judge hears the application, the petition for the protection order, makes a decision whether to grant it or not, and then if it is granted, the service providers and law enforcement help serve the defendant, the respondent, the person who now needs to stay away from the older person, and then helps protect them in that way.

OK. Let me give you a sense of findings. I'm going to, for the most part, glom all of them together in this part, and then I'll go a little more specific about each of them and focus on how they relate to prosecution.

So what we sort of need to do here, though, is take out the West Palm, the Palm Beach Elder Justice Center, out of the picture because, as I said, it's really not helping victims very much. OK?

But, for the most part, the other four projects did help enhance access to justice for elder abuse victims. The victims received emotional support. They received help in getting to court. They received accompaniment in court. They received referrals to other services as needed. They received, often, protection from law enforcement. They helped them get accommodations in court. So, if it was necessary to have an interpreter or to try and schedule the hearing at a certain time because of a medication schedule or something, they helped in doing that. I told you they helped them get to court by arranging transportation, sometimes driving them themselves, I think, even though they probably shouldn't be doing that.

In general, the four projects felt that more elder abuse cases were heard as a result of the project. Certainly, with the Protection Order Projects, those cases would never have gone to court in the first place because the victims simply could not get to court to obtain the paperwork to do the protection order and then to present their case to the judge. In the other cases in the Elder Protection Court, it's just an environment that's more conducive to older people getting to court, to feeling comfortable being there, and to the prosecution and the public defenders being involved with these cases. Everybody felt, even if more cases weren't coming to their specific project, that cases were heard more quickly, more efficiently. They were expedited on the calendar and just handled much more effectively as a result of the project.

They also felt that, a sort of ripple effect — and there were many ripple effects of the projects — that the judges and not just the judge who might be handling the specific cases but the other judges in the courthouse, too, were made more knowledgeable about and more sensitive to elder abuse as a result of the project being there. They felt that service providers learned more about elder abuse and about the court and about the court's role, and because of these projects, that politicians were made more sensitive; the media was made more sensitive as a result, too.

All of the projects helped link older victims to other services in the community, which was very important, as I talked about before, and, again, enhanced public and professional awareness, far beyond the immediate players in the court projects and initiatives but beyond that as well.

So let me focus a little bit — how am I on time? — on prosecution a little more closely. The Elder Protection Court, as I said, in general, enhanced knowledge and sensitivity. Everybody felt — and again, it was the Prosecutor's Office, it was the Public Defender's Office, it was the investigators who worked with the Prosecutor's Office and others — that having that sort of central focus for elder abuse in the court really enhanced everybody's gain. It made them understand each other's roles better. It made them understand the approach and had a consistent approach because they were dealing with one judge rather than with multiple judges, and the judges had — even the prosecution and the public defender both had some concerns about the judge and about being before one judge.

And interestingly, Judge Conger had formerly been a public defender. So the Prosecutor's Office was a little worried that maybe she would be too lenient on abusers, and the Public Defender's Office was a little concerned that maybe, again, just having one judge and having — you know, the Prosecutor's Office is usually always stronger. It's better funded than the Public Defender's Office. So they had that concern as well, but I think they all felt fairly comfortable with it.

They all, by working together so closely, consistently and often, developed trust in each other. They realized that they all had a shared goal of pursuing justice, and that was really important because it enabled everybody, as they put it, to sort of look outside the box and trying to come up with remedies that best helped the older person, not just the system.

The environment was conducive to elder abuse prosecution there. As I told you, the Prosecutor's Office had a long standing, multi-person unit on elder abuse prosecution.

The Public Defender's Office role was critical because they had to basically consolidate — Alameda is a big county with multiple courthouses — they had to consolidate all the elder abuse cases into the main Prosecutor's Office in Oakland, which basically put all the workload on one person, who was already overextended, and so, if the Public Defender's Office had not been willing to play, this whole system never would have worked. The court would never have been able to take off, which was quite interesting. We did not expect that finding.

In terms of the Elder Justice Center in Hillsborough County, where it relates to prosecution is by helping to prevent guardianship abuse. Their main focus now is on guardianship monitoring. That has come about due to budget cuts and because they think that that is what will make the program indispensable to the court and will keep it from being destroyed as a result of the current fiscal situation. So, by doing that monitoring, they are preventing abuse and they are detecting abuse that's going on. Interestingly, law enforcement was not really involved in that project, and we did not interview anybody in law enforcement or the Prosecutor's Office there, but there's the tangential key from that project.

I told you about the Elder Justice Center in Palm Beach, which is really much more focused on diverting older criminal defendants with dementia or other problems out of the justice system. So they're not helping victims directly, but maybe because they're doing that, they're making it easier for victims to report, to go to law enforcement. Back to what I said earlier about you don't want your abuser to go to jail, you want your abuser to get the treatment that he or she may need, that helps there. And the prosecutor felt more comfortable about these cases and the outcome of these cases because he knew that the Elder Justice Center folks would do some follow-up and some monitoring, and so they felt like this was the right system.

The Elder Protection Projects, of course, helped prevent further crimes by the abuser in this case, and because the violations of the protection orders are crimes, there may be more prosecutions of those cases as a result. And it also helps to develop evidence that can be used by the prosecutor in these cases.

Intra-court coordination, I mentioned was still a problem, 17 years after Judge Kirkendall in Ann Arbor mentioned it. I had sort of hoped that with the advent of computers that these things would be less of a problem, but, apparently, they're not. It still is a problem. The courts still aren't communicating with each other.

Now, a lot of judges will say, “Well, that's because we're not supposed to know what happens in other cases. We can't consider that in making our decisions,” and that's valid to some extent, but I think to some extent, it may also be an excuse. Judges were telling us that unified courts do help with that problem, but just having a court-focused initiative or having computers isn't helping with that problem.

Providing leadership in the community is still an issue. In Alameda County, Judge Conger started, and the judge who has replaced her now that she has retired, was continuing the Elder Access Committee, which the court sponsored quarterly, held at the court, brought everybody together to come to talk with each other, to share, to talk about issues, problems with the court, as well as problems systemically in the community related to elder abuse.

I think we still see judges very reluctant to take a leadership role on these issues. Some of that is for ethical reasons. Judges, of course, have to be careful about not taking sides, but I think also there, too, that can often be an excuse, that they just don't know about this issue still, they don't care about this issue still, and aren't willing to forge ahead in a leadership role, as has been done with domestic violence and child abuse. So those are things that hopefully will change.

In terms of the court-filed data, I will just say that we found very little. What was in the court files was about as basic as it gets in terms of demographic information about the victim, maybe about the perpetrator. Often information about the perpetrator was in a probation record or in a sealed court file, a health or mental health file that the judges gave us authority to look at, but, of course, what that means is that it's very difficult to do research and evaluation on these issues and that has policy and funding implications, of course.

I'll wrap up just telling you we are in the process of wrapping up and writing our report. We'll do some articles as well. The project ends on December 31, and we have — information will be on our website once we can put stuff up on this project, and there's lots of other elder abuse information up on our Web page, too.

Thank you.

Page Ulrey: Good morning. I'm Page Ulrey from the King County Prosecutor's Office, and I'm going to just talk a little bit about how these studies relate to my experience as an elder abuse prosecutor.

I work for a prosecuting unit that is now comprised of two people and a half time paralegal. We prosecute cases of adult neglect, sexual assault, financial exploitation and homicide. We also are mandated by our elected to do trainings of first responders, the theory being that if we don't train people about how to respond to these cases, they're not going to come into our office.

And we also work on the coordinating community response to elder abuse, that theory being that if we don't work on working with APS and working with geriatricians and others in the field who are dealing directly with elders, we're not going to be handling these cases properly.

So that's just a little bit of background, and I want to spend a few minutes taking you inside the Prosecutor's Office, because that will give some context to some of the things that Shelly was told by the prosecutors she interviewed. And I think before I do that, it's important to remember that very few of the people she talked to were specialized elder abuse prosecutors, and even of the ones that were, I can bet you that they had very little training on elder abuse prosecution since there hardly is any.

So, usually, in most prosecutors' offices, there are no specialized elder abuse prosecutors, and a prosecutor who's handling one or two elder abuse cases on top of their other 98 cases is not going to be having an easy time of it. They're not likely to even know what a multidisciplinary team is. They're not likely to have ever heard of Adult Protective Services. They're not likely to have connections with the elder abuse detectives in their jurisdictions, if there are any.

And other cases that are murder cases or bank robberies or kidnappings that are kind of higher visibility in the office are inevitably going to take precedence, and they're also not going to have the knowledge to handle those cases effectively.

Elder abuse is not sexy in the prosecutor's world. The cases are considered to be onerous. The victims, as Shelly articulated, are considered to be difficult and high maintenance, and the cases do not have cachet in a typical prosecutor's office, and that affects how they get handled.

And it's true that they're difficult. These are big cases. The trials take longer. They're more complicated. They involve issues like capacity and dementia and powers of attorney and guardianships and things that, as Lori said, none of us get taught in law school.

And, also, elder abuse prosecution is more expensive. In almost every case I've tried, we've needed an expert witness, and that can be thousands and thousands of dollars, which is tough on a budget that is limited.

Also, a lot of prosecutors' offices really emphasize convictions over justice, and convictions aren't always the best thing for our victims. So, if we handle a case and it turns out that if we dismiss the case and arrange for different care for our victim, and treatment and offenders in treatment, that could actually be a better outcome for our victim than sending the defendant off to prison for the rest of his life.

But the way we tend to operate as prosecutors is going to trial is valued, getting a conviction is valued, and the longer the prison term, the better, with very little regard for what's best for our victims. So that is also a thing that I think complicates how we're handling these cases, especially when we're not specialized.

We also are not really trained to be empathetic with elders and elder victims in particular. A lot of prosecutors will insist on their coming in to testify, even if it means damaging their relationship with the offender who may be the last person they have left in their lives.

Again, failing to consider the needs of the victim in negotiating a case, so as Lori was talking about, what the victim wants more than anything is for her son, the abuser, to get alcohol treatment. That is not consistent typically with how a prosecutor is going to handle a case. That's not going to trump the esteem they get from their colleagues for sending that defendant off to prison.

We also just automatically request no-contact orders in our cases. There's no services or resources for supervised visitation, so she can see her son. There's no thought, even, that maybe a blanket no-contact order is actually going to do more harm to her than good. And, again — and I think this is also consistent with what people told Shelly — there is a perception of elder abuse as a family matter, as not being criminal, the same way we used to think of domestic violence decades ago.

We have little or no training on abuse of power of attorney cases. We don't know what undue influence is. We have no idea how to handle an adult neglect case. When we get one of those cases, we get a stack of medical records and maybe a description of a crime scene, and for most prosecutors handling that, they have absolutely no idea how to proceed and they assume they can't. So the statute's too narrow or they don't have enough evidence or there's some other explanation that they come up with, but it may actually be a lack of training.

We aren't taught about competency and capacity, especially with regard to elders. We don't know how to present massive bank records, which we have to do in almost every financial exploitation case we handle. We don't know how to present medical records to a jury. What do you do with that stack of records when you're trying to convey the symptoms and signs of neglect someone had, how do you present that evidence, it's just not something any of us are taught in law school or in any subsequent training that we get. We're not taught about the need to make accommodations for victims with physical disabilities or victims with dementia.

And Crawford, which also came up in Shelly's talk, is a case that basically says that elder hearsay, in other words statements that elders make to law enforcement in an interview context, are not admissible at trial unless that elder is available to testify at trial. This is a case that came out of my lovely state, and that pretty much eradicated all of the state's elder hearsay statutes that existed. So where beforehand, officers could interview victims and do videotapes and those videotapes could be offered against the defendant at trial, even if the victim had died or was no longer able to remember what had happened, after Crawford v. Washington, all of those hearsay statutes became unconstitutional. So those videotapes no longer are admissible, but there are ways around that. There are other hearsay exceptions that many victim statements still apply to, but a lot of prosecutors think about Crawford and they say, “Oh, we can't get in any victim statements. We can't even try if we don't have a victim. No victim, no case,” and that really is, again, a training issue.

We're also used to working in a silo, we as non-elder abuse prosecutors. Again, I'm familiar with APS. We're not used to staffing cases with a multidisciplinary team, if we even know what that is. We're not used to working on the coordinated community response to a problem. All of those things are foreign to how we're raised in a prosecutor's office.

So these are the different explanations that Shelly was given about why elder abuse is harder to prosecute, and I just want to break down each one of those and talk about it briefly.

Elders don't make good witnesses. Prosecutors, like other people, often assume that if you're older, you lack credibility; that if you're older or you suffer from dementia and that even if you have dementia, that means automatically that you can't testify at trial.

We typically lack training as to what dementia is, as to the fact that a victim actually, even with dementia, may still be able to testify, depending on what time of day it is, what kind of medications they're on, how their other underlying health conditions are doing.

They are unfamiliar with the UCI study that came out recently — and Aileen Wiglesworth is here with us, who worked on that study — regarding the impact of dementia on the ability to remember traumatic events, and Aileen's study actually says that even with dementia, there is evidence that you have a better recall of traumatic events than non-traumatic events. So, with studies like that, we may actually be able to put on testimony of a victim who has dementia, even if they're unable to recall other parts of their history or their lives.

We also lack training and knowledge of the fact that if consent is the defense and there's a dementia issue, we need to get a psychological or psychiatric evaluation of the victim. We don't just write off the case.

The statement that the prosecutors made to Shelly that victims are unavailable, therefore elder abuse is harder, and certainly that's true, but I also want to give some background as to why that's true. Elder abuse, as you all probably know well, is lethal to many of our victims. Mark Lachs found that elders are 300 percent more likely to die a premature death, no matter what kind of abuse they've suffered.

Still, we can prosecute cases without victims. In most of the trials that I have, we don't have a victim. The victim has either died or has lost their memory of what happened, but you have medical records and you have financial records. We have evaluations that should be conducted immediately in an investigation by a geriatric psychologist or psychiatrist where dementia is an issue with your victim.

We have hearsay statements that the victim made to other people. If a victim makes any statements to medical personnel about what happened to them, that's admissible in court. If the victim made excited statements to law enforcement or to a 911 operator, that's still admissible in court, even with Crawford. So a lot of the way we look at cases now is we go through all the different victim statements and we say which ones of these still fall within hearsay exceptions and can be admitted, despite the fact that Crawfordexists.

And on the financial records point, I think people also told Shelly that financial cases are very difficult to prosecute, and I actually think they're the easiest type of elder abuse to prosecute because we have bank records, and the bank records tell a beautiful story of what happened to that victim's assets before and after the perpetrator got into their life. They also show what happened to the perpetrator's assets. So, typically, we will put on evidence of the perpetrator having nothing and then moving into the victim's life and suddenly buy new cars and new houses and spending money like crazy. Meanwhile, at the same time, the victim, who had a lot of money, their assets very quickly go down to nothing, and at the point when they hit nothing, the perpetrator leaves their life, typically. So that kind of evidence, even without a victim, can be very compelling for a jury.

Obviously, you have issues if the victim consented to that transaction, but if you have evidence — or to those transactions, but if you have evidence from family members saying that the victim isolated them from — or the perpetrator isolated them from the victim, that the perpetrator was emotionally abusive to the victim, that the victim never spent their money the way they're spending the money once the perpetrator is in their life, all of those things can come in as evidence in trial and can be used successfully, but, again, it's a training issue.

The next statement, weak or lack of evidence, this is purely subjective. It's claimed. Sometimes it's true, but it's also claimed when a case is difficult, particularly when you have a caseload where you have one or two elder abuse cases and a bunch of other, much easier, much more compelling cases.

It's also claimed when you don't have the training to really understand how to prosecute a case, like the one on your desk, and it's also claimed when you don't have the resources to properly try a case, when you don't have funds for expert witnesses to help prove your case, when you don't have funds for an expert to help you look at your medical evidence or your financial evidence and explain how a case like that can be proved.

The statement by the prosecutors that the perpetrator acted within their rights, that scares me. I think it rings of, well, abuse, in some way, frankly. Consent is the most common defense that we see in elder abuse cases. We see it in sexual assault cases and neglect and financial exploitation cases, and, typically, our perpetrators claim that what they did was within their right, that the victim consented, that this was what they were entitled to do.

But we have to look beyond that simple consent that's alleged. We have to look at whether or not a new influence was used to obtain that consent; in other words, whether the perpetrator took over the victim's free will and forced that consent. We have to look at whether the victim had capacity to consent, was their dementia so severe that they really couldn't adequately know what it was they were consenting to and give permission to it.

Abuse of power of attorney. Again, perpetrators who steal money under a power of attorney document typically claim it was within their right, but the reality is that a power of attorney creates a fiduciary duty to the elder. When you have a power of attorney, you are not allowed under the law of — I'm guessing all states, Lori. Is that true? — to spend that money for your own benefit. It must be spent for the benefit of the elder.

There usually is some kind of gifting provision in the power of attorney document, but it typically creates a very clear limitation on what you can give to yourself. So we can prosecute these cases. We prosecute them as thefts, theft by embezzlement, the same way we prosecute accounting abuse cases or bookkeeper financial abuse cases. We don't need a special power of attorney statute, even though they're wonderful. So this is, again, a common misconception amongst law enforcement and amongst prosecutors and one that has explained tremendously why we aren't handling these power of attorney abuse cases properly. Again, it's a training issue.

“He said/she said,” that was, I think, the last explanation that they gave Shelly about why these cases are so hard. Inherent in most elder abuse cases is a lack of witnesses. These are almost always “he said/she said,” if the “she” is still alive. But the reality is that a key indicator of elder abuse is the loneliness or isolation of the victim. So, almost always, we only have one witness to our crimes, and that is the victim, again, if she is still alive. So these are always “he said/she said” cases. The way we have to prove these cases is circumstantially. We need to look at the records that we have. Like I said before, we need to look at what the family members say, the friends who had a relationship with the victim before the incident occurred; the medical records, the geriatric psychiatrist or psychological evaluation of the victim. All of those things are how we build our cases, even if we don't have any witnesses to the crime.

We don't look at child abuse and write it off saying it's a “he said/she said” case and we can't prosecute. We don't look at domestic violence anymore and say that.

So the question for us is not whether there are witnesses, but is there sufficient corroborating evidence to prove the case beyond a reasonable doubt, and if there isn't, we should be able to articulate why, what evidence on what element is not sufficient, and why is it we can't prove that evidence through the surrounding circumstances and the corroboration that we have.

I'm not saying these aren't hard cases and we don't decline a lot of these cases. We do, but they're not impossible simply because we don't have any witnesses.

I want to move a little bit more to Lori's topic about the specialized courts, and I think it just ties in beautifully to the need for specialized prosecutors. What a typical elder abuse victim has to go through if their case is ever prosecuted is, first, they have to give a statement to law enforcement about whatever horrible thing happened to them.

If it's a sexual assault case, they have to undergo a rape exam. Then they usually have to meet with a prosecutor and give a statement. Then they have to submit to a typically ugly interview with the defense attorney. Then they get to wait for months or years for their case to go to trial, if it doesn't plead guilty, and then at trial, they get to tell their story of having been exploited by the person they love to 12 plus strangers. Then they get to be cross-examined again by the defense attorney. Then they get to wait for sentencing, which usually takes weeks or months after the trial if there's a conviction. Then they get to wait for the appeal, which usually takes another several years before it's done, and if that is successful for the defendant, they could have to go through a retrial, all of this when they're probably in their 70s, 80s, 90s, and are having to deal with having been abused by someone they loved.

The non-specialized courts that are typical in our jurisdictions, as with non-specialized prosecutors, end up losing the value of these elder abuse cases in the crush of the rest of the calendar. We end up being frustrated by the delays that are caused by the special needs of elder victims. We don't have understanding of dementia and how to accommodate victims with dementia. We typically don't have victim advocates who have specialized knowledge about how to handle disabilities and dementia and what services are available to elder victims.

The prosecutors that Shelly talked to had really great things to say about what needs to happen to reform courts, and I just wanted to put them up here again because I think they fit in beautifully with what Lori talked about as far as what the courts around the country are beginning to do.

Expedited trials, enhanced penalties, priority on dockets and courtroom accommodations for elders.

Bethany Backes: So I’m gonna be very brief and just say that today we’ll be hearing about two ongoing NIJ-funded studies on custody evaluation. Both studies will be concluding within the year, and the final reports will be available and accessible through the National Criminal Justice Reference Service.

So our panelists today are Dr. Daniel Saunders, a professor at the University of Michigan’s School of Social Work; Dr. Chris O’Sullivan, who is a research consultant currently working with the New York Legal Assistance Group; and the honorable Dale Koche, Koch, sorry, a senior judge with the state of Oregon. And we hope that today is the start of many discussions on this topic. And at this time I just want to ask everyone to please turn off their cell phones or any electronic devices to vibrate or silent. And we’re gonna begin with Dan.

Daniel Saunders: Good morning, everyone. Hope you’re doing well. It’s good to be here with you. Thank you, Bethany, thank you for putting this panel together and for your introduction. And I wanna commend NIJ for making this move into a new area — the family law side of the law.

In the early days, the focus in helping survivors of domestic violence was to make sure that offenders were arrested just like any other offender and that there were restraining order laws and that we had good stalking legislation. And it’s been only fairly recently that advocates and researchers have become aware of the horrible injustice when survivors finally escape from domestic violence and then are faced with continued stalking, harassment, abuse and then, low and behold, the worst trauma that survivors, I think, can ever go through is to get that piece of paper in the mail that says your partner, your ex-partner wants custody of your children. And so the trauma is multiplied times three where women — it’s usually women — are faced with sometimes losing custody to a person, they believe, will continue their abuse — abuse of the children, abuse of them.

And so my interest was piqued many years ago doing divorce counseling, working with men who batter and then more recently with the supervised visitation program evaluation that I helped conduct of safe havens. And there we saw on average across agencies about 20 percent, 10 to 20 percent of the noncustodial parents were women ordered now to come and visit their children. So the, the awareness I think within Office on Violence Against Women and NIJ has really been heightened, so I really appreciate the focus that’s now being given.

So I’m gonna present some pilot data of a survey of custody evaluators and then a little bit, I’ll give a preview of some of the interviews that we’ve done with survivors.

First, I have to find the right button. Oh, OK.

So my collaborators at the University of Michigan are Rich Tolman, Karen Staller, Kathleen Faller, and then a whole team of research associates and assistants, and then consultants from, from law, from child custody evaluators, social scientists — about two pages worth of names that I could put up there for you.

You probably know this already that when women, and I’m talking primarily about women — men are battered as well, and you know, I have a whole presentation on gender equality and inequality regarding who are perpetrators and who are victims — I’m gonna speak mostly about women as survivors that when people leave an abusive relationship that’s not the end of the abuse, and in fact, there’s a tremendous amount of stalking that goes on; the risk of homicide increases, which is one reason why many women realistically stay in these relationships — they’ve been threatened with their lives.

And then we have many other consequences as well. These men are likely, half of them are likely to find a new partner to abuse; therefore, exposing the children again to more violence in that relationship. They’re likely to undermine the parenting of the non-abusive parent and on and on. I’m sure many of you are aware of, of these consequences.

For a long time, custody evaluators and judges talked about high-conflict relationships and didn’t realize that hidden within these high-conflict relationships, there is a great deal of domestic violence. And now we know from some very good, well-documented studies that 25 to 75 percent of contested custody cases involve domestic violence. And yet we still hear evaluators and judges talk about high conflict and don’t put the emphasis where, I think, it needs to be, which is on the violence that occurs within visitation and exchange cases based on some national surveys. The primary reason that families are referred for visitation and exchange is domestic violence followed very closely by child abuse.

There are a number of negative consequences. I mentioned losing custody to an abuser is an obvious one. We have other negative consequences as well — having unsafe supervision, supervision by a friend or relative when what’s really needed is supervision in a safe environment, a supervised visitation program with close supervision by a paraprofessional or a professional with separate entrances, separate parking lots, you know, all the safety mechanisms. And then we also see where too often, I think, there are referrals for couples counseling for mediation, which can undermine victims’ rights or place them in more danger. In terms of what we know about negative outcomes and the risk factors, it’s a very sad situation that too often the abuse is never even detected by evaluators, by mediators. And then even if it is detected, it is not considered seriously in the determinations, so it may sit there in the record but not be used adequately.

We know that if judges are well trained, there’s a tendency for them to give sole custody to abused mothers. We have a bind in about half the states now: We have what’s called a friendly-parent statute, which means that, you know, one of the factors for determining custody is that the parent must show that they’re willing to facilitate a good relationship with the other parent. But that puts victims in a real bind because they, for good reason, are reluctant to co-parent, to go into mediation, and as a result, they’re labeled as uncooperative, unfriendly, and that goes against them. And that’s a powerful effect, even in cases where there’s a presumption that custody should not go to the abuser. So we’re starting to uncover some of these risk factors for the negative outcomes.

OK, what is this big figure here? This is our conceptual framework that we started with on this project, it lays out the major research questions and hypotheses. So starting in the middle with my red laser pointer that I don’t have, but it’s the one that’s, it’s easy to see because it’s that big bold one, kind of in the middle, off to the right. So the main variable that we’re interested in, “To what extent do evaluators believe that survivors are making up stories, that they’re making false allegations of domestic violence?” And then if you go just up from there, we’re trying to see, ”Is that related to other beliefs that child abuse allegations are made up, that parents are alienating the children from the other parent and other kinds of beliefs? And then backing up, we’re interested to look at the background, the training of evaluators. Does that feed into the belief that survivors are making up stories about abuse? Gender, whether they themselves know survivors of domestic violence. I’m over in the background traits, the lower left here.

And then certainly, we want to look at myths about domestic violence, whether those myths are held by evaluators and if those are related to a belief that stories are being made up. And then what we call these “core beliefs,” these distal factors, which are more abstract beliefs — beliefs that social hierarchies are a good thing, social hierarchies between groups, a social dominance orientation that patriarchal norms will be related to myths about domestic violence. The belief in the “just world,” which is when someone thinks, basically, the world is a just place. People get what they deserve, and deserve what they get. And we don’t have to worry about oppression and differences in power. So we have, we have some of those abstract beliefs put in there, which can still be used in training to help people become aware of their values, their core beliefs and allow them to think and perhaps shift some of their more surface beliefs as a result. So all of this is designed to try to figure out what can we do better for training evaluators, in particular, but also other professionals?

OK, so those are the background kinds of variables that we’re looking at — beliefs, training, gender — and then of course the outcome; these, these negative outcomes that I mentioned already: that joint custody would be given or sole custody to an abuser, that mediation would be pushed, that there’s unsupervised or poor supervision, unsafe supervision.

OK, so this kinda lays out the hypotheses then; you can see many variables, many kinds of possible relationships. And I’ve put for you, in the handouts, these hypotheses in sentences. If you prefer not looking at maps, but, I mean people have a preference, right? When you’re getting directions, do you want to look at a map or do you want to get the directions written out for you? So here are the directions written out for you, the hypotheses, and you know there’s gender differences there, right? So who is it that likes maps and who likes written directions, you know, I think. In the discussion section, if you wanna know the answer, I’ll tell you then. OK, so I’ve laid out the hypotheses here. And I won’t take more time; I want to jump into methods.

What we have done so far is to complete two pilot surveys with a total of 62 evaluators. We used Web-based and mail-based administration. And we were able then to hone down and refine our measures, to improve on them because we also asked them at the end, “What do you think of this survey? Was it any good? What are your reactions? How could it be improved? What questions couldn’t you understand?” But we learned a lot. You know, evaluators don’t like responding to a little case vignette that we had. They said it would be unethical for me to put down a reaction because I would have to do the entire evaluation; you know, I cannot respond to your vignette. So we changed that. We said what hypotheses would you have, not what recommendations would you make? So we learned a lot through these pilots. And we’re about to launch the major survey.

We have measures of history of practice, practice experience with domestic cases, training, the, the scope of training, but also the modality — do you go to workshops or do you read articles? So. And then we have this vignette, which is at the end of your handouts, was modified from a journal article by Claire Dalton. And then we created various responses for them. You know, what’s the likelihood that you think the mother or the father are minimizing, are minimizing the abuse or exaggerating the abuse? What’s likelihood of future harm by either of the mother or the father? And then we have measures of these distal factors. We borrowed and stole items on beliefs about custody visitation and domestic violence from other sources, and then we created many of our own. And through this pilot, we found some very reliable subscales around these different themes.

So, the belief that there are false allegations of domestic violence that are extensive, that domestic violence has a severe impact on the children — that was another subscale — the friendly-parents standard that I mentioned, and then a number of myths, myths that for example, domestic violence ends at the separation, that abusers don’t undermine the other parent’s ability to parent, those kinds of things. So, I mean, the questionnaire, the first one we did was very long — I think 13 pages, 125 questions. And now we’re down to 80 questions. It’s much shorter, it takes about 25 minutes, 20, 25 minutes to complete. And we’re trying to hone it down even further.

So we’re gonna have a wealth of information here when we’re all, when we’re all done with this. So from the pilot survey, or two surveys combined together 60, 62 people. Most of them were in private practice; some worked directly with course. And we need to balance our sample more and, and go out to court settings, where there’s a lot of good training in many of those settings where family court personnel are doing a lot of the evaluations. This tended to be a group that had lots of experience, about half of them 20 years or more. There’s 60 percent men, an older sample. Many of them were Ph.D.s in psychology. And 20 percent had master’s degrees.

We also asked about their own, their own personal acquaintances regarding who, who they knew as a survivor. And so we asked about friends, coworkers, and here you see that about 20 percent of them said that their mother was a victim and survivor, and we’re gonna use that to figure out does that make them more sensitive or less sensitive in responding because previous research shows it can go either way, you know, with child welfare workers and, and other workers it can, it can go both ways, both ways. Forty percent had a friend who was a victim.

OK, here are some results then. Major question was “In your opinion, what percentage of mothers in child custody disputes make false allegations?” And then we asked the same thing about fathers. And this is in general then; this wasn’t first asking, “Of those who make allegations of domestic violence, what percentage are false?” So we’re gonna refine that a bit ’cause this is capturing the whole group — overall, what percentage of mothers do you think make false allegations of domestic violence? So the top one here is the mothers — and this is simply the distribution — so to interpret this, you see on the right-hand side, there were a couple people who thought that 50 percent and 60 percent of the mothers were making false allegations of domestic violence — quite a large grouping around, you know, the belief that 10 percent of them were making false allegations, 5 percent and so forth.

So that’s just a, a distribution across how many people the number of respondents who believe that 0 percent, 10 percent, 30, 50, 60 percent, etc., were making false allegations. OK? And then at the bottom, the question is how many of you believe that, you know, what is your belief about the percentage of fathers who are making false allegations? And there you don’t see it skewed as far on the right, and it’s clumping more around the 0 and 5 percent area. So it looks like in terms of gender of the parent, we’re seeing some effects here. And like I said, we have to refine this further, so we’re tightening all this down to domestic violence cases. But we may have what people have said all along in critiquing some of the theories about parent alienation, for example, that it, in the past it’s been gender, that it’s been more blaming the mothers and having less creditability in their stories. OK? So that might be coming through here. Very interesting.

Okay, then we did find, as we thought, that a belief that there’s false allegations about domestic violence also relates to belief that the stories about child abuse, sexual, physical abuse are being made up in fathers and mothers alike. When it came to child sexual abuse beliefs, those were tied together very closely. If, if an evaluator believed that mom was making up stories about child sexual abuse, they also tended to believe that a father would do the same. OK, and then what are the other relationships here? These beliefs about false allegations?

Well we found — I’m not putting any of the statistics in here; this is all just straight out statements for you — these were bi-variant correlations, so these were significant R-values coming through. Some were quite high, but they were all significant. So the belief in false allegations related to the belief in friendly-parents standard that the mother in the vignette would be likely to harm the son psychologically, that she was exaggerating the violence, that the father was not likely to harm the son psychologically. And when it came to outcomes their, their tendencies in the response to this vignette — what’s the likelihood that you would recommend various actions — the belief in false allegations of DV was related positively to their tendency to recommend joint legal and physical custody, sole physical custody to dad, mediation, couples counseling, no supervision of visits with the child — so unsupervised visits. So some of the concerns that we’ve had in starting the study are coming through. And then there was a negative relationship between false allegations, the belief in false allegations and the tendency to recommend sole physical custody to the mom and supervised visits. So as you’d expect, just the flip of the others.

One of the questions we asked, “How do you ask about domestic violence? Do you directly inquire about domestic violence?” Most of them did. And then we said, “What measures do you use to screen for domestic violence?” and then “What percentage of the time do you screen?” And what this graph shows is that almost 40 percent of them said they never do any formal screening; they don’t have any measure, any protocol that they use. And then a whole other group on the other end use such an instrument all of the time. So we’ve got quite a split here, and we’ll do a lot of other analysis to look at the differences here and how these line up.

So in the follow up question, one person said, “I use a clinicalized version of the conflict-tactics scale measured, about one month apart.” Some used the domestic violence inventory, the SARA — the Spouse Assault Risk Assessment. Quite a few of them used traditional psychological measures — the MMPI, the Rorschach. And one person said, “I find I never have to directly inquire about these problems. If they are present, the abused party always mentions them.” And that is why I think we need to be quite concerned. I mean, we found in many different settings — if you look at hospital settings, mental health settings — there’s a tremendous under detection of violence. And I think evaluators, with their good intentions to come in with a neutral view, don’t want to rock the boat, or they don’t wanna raise issues that aren’t there.

I’m not sure what’s happening here.

And there are some models of conducting evaluations that say, let’s base it on the allegations that come to us; let’s not base it on what we try to uncover. We’re only going to deal with what comes to us from the parties involved. So I hope that as we work on this research that we’re able to have some impact on that.

Regarding background factors, the sample sizes aren’t real large to detect differences here. We did find some gender differences that men are slightly more likely than women to believe that mothers make false allegations, that they’re gonna harm the son in the, in the vignette.

Oh, is this for me? Oh, OK.

That they … OK, we’re gonna wait for the larger sample to analyze knowing a victim personally as it relates to all these beliefs and then, very importantly of course, is how they acquire knowledge. Have they gone to workshops in the past? Have they read articles? Have they learned about post-separation violence, about the impact of violence on the children? We have all of those items in there that we’ll be analyzing.

In relation to core beliefs, as we hypothesized, two out of these three core beliefs were related to belief in false allegations, patriarchal norms, belief in just world. And then patriarchal beliefs also were directly related to some of the outcome measures — referring the mother but not the father for parent education, for example, psychological evaluation for the mom is needed. And then we’ve conducted 11 out of 24 interviews with survivors in four communities, four different states. So we’re gonna have state laws also that differ. California is one where they have a lot of training provided for judges, evaluators, standards that they have. And so we’re interviewing women who have lost custody or children ordered to unsafe supervision. We’re also gonna analyze documents that these women have from their evaluations — the judge’s orders, the court, the custody evaluation, other orders. In some, I mean, one woman had nine different documents, three different evaluators, a domestic violence assessment etc., etc., etc. So there’s a lot of work we’re gonna need to do there.

In terms of themes coming through, I’m not gonna take time to read from this one, but the focus that this survivor said needed to be on the emotional, on the psychological pain, as well as the physical. It’s not, it’s not enough just to focus on the scratches and the obvious injuries but also on the emotional pain that occurs. And then we’ve heard this before, this survivor points this out, but we’ve heard it also, that evaluators often focus on what happened during the relationship and forget to ask about what’s going on now, during the separation period. And this person brings it through that domestic violence, whether it’s past or present, needs to be considered.

We’re gonna … In these four communities, we’re gonna conduct a community analysis, interviewing key players, finding out from them a snap shot of what seems to work well and what doesn’t work well in their communities.

So then, in wrapping up, we certainly have limitations to these pilots as based on self report, and we are gonna supplement that with document reviews. We’ll have a total sample of 445, and we’re gonna expand beyond private practitioners. We have a lot of Web searches that we’ve done. And we’re also gonna approach family court to be able to also survey people that work in family court settings — I think that’s very important. And we’ve got a number of communities that we’ll be going to for that. So to summarize the, the survey results, it was surprising in this very small pilot how many of the hypotheses were affirmed that false allegations of DV were related to the friendly-parent standard, that, that’s related to the belief that mothers are likely to harm the son psychologically, the tendency for joint custody, sole physical custody, relationship with these distal factors —these very abstract ones — patriarchal beliefs, belief in a just world.

So overall, background and core beliefs seem to be related to the belief about false allegations, which in turn are related to a tendency to order joint custody or father sole custody. So we have a number of hypotheses that are affirmed, and with the larger sample, we’ll be able to do a lot more. In terms of which set of predictors, overall, are gonna be the most powerful, we can weigh them with each other, we can put them together additively to figure out, you know, is it patriarchal norms and gender together that is going to be the most powerful predictor? So we can look at all of these effects, direct, direct effects, indirect effects, etc., and have a lot more to tell.

So stay tuned. Bethany says maybe next year we can do a poster. But certainly e-mail me; I have my e-mail address if you want more information. And then stay tuned for the final report, which will be up on the Criminal Justice Reference Service Web site when we’re all done. So thank you very much for your attention. And we don’t have time for questions now, we’re gonna…

Backes: At the end.

Saunders: We’re gonna take questions at the end. So thank you very much.

[Applause.]

Chris O’Sullivan: I’m at a terrible disadvantage behind these podium, which were designed by men, I think, very tall men — sort of disappear back here.

The study that we’re doing is pretty complementary to what Dan is doing. We’ve got a very eclectic group here. Michael is a parent coordinator, clinical psychologist, and divorce researcher in, in Boston with very little domestic violence background, which he’s getting very quickly. Marjorie Fields is a, is a former family court and Supreme Court judge who left the bench because she wanted to become a lawyer in private practice advocating for families in which there was domestic violence in custody cases. And Kim Selcerhas been our savior; she’s the one who sort of instigated this study by saying we’re having a real problem with our, our cases in family court and with the evaluators. So that was sort of the basis for starting this. April is an custody evaluator in New York City who’s a consultant on the project. And then we have people from various agencies in New York City who have cases in family court who … these are mostly, they do, they represent victims of domestic violence, primarily women, in contested custody cases and only indigent clients, so they’re, they’re free legal services.

OK, this is, this is a, this is a really difficult area. I don’t know how judges stand it. I hope to hear from Judge Koch. How is a judge to know what’s going on in a family and how can you tell from what, what, how they present themselves in court who is a good parent? So it’s really an impossible task to begin with, and then you add to it a domestic violence allegation, and how do you, how do you know if that’s really going on? So they rely a lot on custody evaluators. And then what do the custody evaluators do is really the — oops — the question for this study.

And we’ve got two different contexts for these cases. The legal context puts the best interests of the child first, parental rights second, and then they have to factor in the impact. In New York State, it’s a factor, in some cases a presumption that a parent should not get custody if they’re perpetrating domestic violence. And then there’s the social clinical context of, you know, what happens to the child in these situations if the parents are fighting just to begin with. If there is domestic violence the custodial parent has to be safe in order to provide a good home for the child, and domestic violence has an impact on parenting as well.

So, we’ve got these two dynamics going on. The custody battles set up one kind of dynamics. In divorce there’s a lot of conflict. It’s a time when evaluators and judges might suspect that, that one party will make false allegations of domestic violence. There may be violence that is just due to the heightened emotions around the time of separation and fighting over custody. And then there’s the issue of whether one parent is trying to turn the children against the other parent.

And then we’ve got the dynamics of intimate partner violence, which may make the, the same actions look very different or the same scenario look very different. Maybe there’s a violence at the time of separation not because it’s, of heightened emotion, because, but because abusers escalate their violence when they feel they’re losing control, so the same behavior can look very different.

One of the things we looked at was evaluator’s perception of causes of domestic violence and whether that influences how they interpret the situation and what the risks are. And then drug and alcohol abuse is another factor that can influence the dangerousness of the situation, how the evaluator of the core perceives each parent. So, we have a huge list of research questions. We also have a huge data set. These are the questions we’re looking at, I’ll try to go through this fairly quickly. These are all the factors that might be considered, what evaluators look at: the impact on the child, domestic violence history, parenting, who they interviewed. Did they interview just the family? Or did they go outside to interview neighbors, teachers, family members on each side? How do they interview the parents and children? Together, separately? Did they interview the kids as a group? What did they ask them? What did they ask the kids? How did they go about it? Did they do it with children? Are they doing stories and pictures or are they just asking them outright, “Does your mother beat your father? Does your father beat your mother?” And do they use psychological tests?” I’m not going to go over that because our findings are very similar to Dan’s. They use the MMPI and the Rorschach; they don’t really rely on it very much. 

What way do they get to the different sources? There’s enormous records, criminal court records, family court records. Are they looking at it? Do they think it’s informative? Did they recommend a parenting plan? In New York they’re not supposed to, at least in the city. The judges are supposed to be making the decision; they’re just supposed to do a clinical portrait of the family. In fact, they do. If they’re instructed by the judge not to present a parenting plan, they’ll just make their conclusion so obvious with the way it should go. So this is not worked as a solution to putting it back on the judge. And they all recommend treatment. They are clinicians.

We’re looking at some of these same factors. Are they knowledgeable about domestic violence? Does their basic theoretical orientation influence the way they go about evaluating? What do they think about parental alienation? What do they think about the importance of maintaining a relationship with each parent no matter how lousy they are?

Oops, I skipped one.

So our method was we got 70 cases through these legal assistance agencies in which domestic violencewas alleged and there was a forensic evaluation. We got enormous case files, and we had staff of — we’re not allowed to see them as researchers, but we got staff of these agencies to go through these huge files and create a database with all of the factors in the case. So there might be divorce proceedings, what happened? There might be 10 family offense petitions in family court or Supreme Court. In New York, State  Supreme Court is divorce court basically. We have this big data set; 300 variables on each case. Then we got the redacted evaluation reports, which range from shortest is probably 20 pages, and they go up to 50 pages. So we have these huge stacks, and for each case, we got the evaluation report, we got the order pointing sort of a reverse order, the judge’s order pointing the forensic, because they might want them to look for certain factors. Investigate whether there is domestic violence, investigate whether there is parental alienation, interview multiple people, get all the records, or it might just say interview the parties, and keep it very simple. So we can’t judge the evaluator on what they did without looking at the order. And then we got a copy of the final order. And in addition, we’re interviewing some evaluators from these 70 cases and giving them a brief survey. We’re also looking at their CVs to look at their background and training.

With the interviews, we’re going to try to match it up to whether what they say they do looks like what they do. We have preliminary findings also; we have the kind of data, but, we’re beginning the analysis. The attorneys — this is a cognitive salience issue — the attorneys thought they had these cases happen all of the time, where there is a forensic evaluator and a domestic violence allegation. In fact, it’s quite a small percentage of their cases. We also found that this sample we got from these legal service agencies are a little bit bias to the positive because they tend to get the evaluator they want. So these horror cases that they have are the minority of our examples. There’s a little bit of a distortion. And basically what we know from the interviews and, the evaluators, there’s no standard. They all think they’re different. They don’t know what anybody else does. They all perceive their job differently. Although most of them, I think there’s just one exception, they think their job is to figure out if the allegations are true, and that’s really what they think they’re doing. One said, “I’m really a detective.” So they’re amassing the evidence through interviews, through record reviews, and then they really want to tell the judge what the custody and visitation order should look like. Basically we found that, just a preliminary review, that they all think they’re different, I do it differently from everybody else even if they do it the same. No matter what their beliefs are, they are all pretty conscientious when they get in there. They tend to end up looking pretty much alike. There was one man that said, “There was a domestic violence expert. I hate them. They make such a big deal out of this, a little slap what does it matter? That’s not domestic violence. Psychological abuse, not a big deal.” Yet when you look at his evaluations, he is pretty thorough and just as likely as one who says, “Oh, there is so much domestic violence, and I believe most of the allegations.”

They’re equally likely to find domestic violence, or not, in the cases that we look at that they did. So that’s a little bit reassuring. The biggest difference we’ve found so far is that some of them take a very clinical approach; they don’t look at any records. They just do interviews. They just do their clinical interviews and Rorschach test and protective tests, and they really think they can get at the truth that way, and they don’t need to look at any records; whereas, others just spend enormous amounts of hours going through every petition trying to look for an overall fact pattern, which is not a clinical psychologist’s job. So that’s the biggest difference.

I have a handout — probably not enough — with the statistics, and I want to just talk about a case to illustrate all of the issues. This case was very typical in a lot of ways, but it was atypical in the fact that there were three different custody evaluators on it. It just sort of shifted around, and they came up with different conclusions, so I wanted to show how their approach led them to different conclusions about what was going on in the family. Custody kept shifting back and forth depending on which evaluator held sway over the judge at that moment and that’s sort of a mystery too. Here’s the mother, this is kind of typical in a way — the part about her husband trying to kill the district attorney in a prior case is not typical, but this education class disparity between the father and mother is quite common. She had a learning disability, dropped out of school, got married and had a child. Her husband ended up in prison. She went to prison briefly as an accomplice and was on probation for five years. She met the father in this case, and he helped her win a custody battle with the father of her son who was in prison. And they had two children together. They never married. And then she began to say he was abusive and controlling. And he was quite educated, quite self aggrandizing, claimed all these heavy connections, like he worked in the Manhattan D.A.’s office, or he’d done this or done that. He had claimed he knew all the jazz greats because he had been a jazz musician. He described himself as a health nut, and it was quite extreme. He boiled water for five minutes before he drank any or cooked in it. But he also saw himself as helping this woman. He was going to get her through high school, get her through college, so that was part of his controlling, perfectionist personality. He was arrested for physical assault on the mother twice. The first time it was what we call “ACD” in New York. It’s basically if he follows these conditions from the court, the case will be dismissed. He was told to go to an anger management program, and if he went to the anger management program, the case would be dismissed.

But he never went; nobody followed up and the case was dismissed. But the mother got an order of protection, and she eventually signed an agreement with him saying that she wanted to reconcile, and she wouldn’t press charges against him. So they were back together, and then he was arrested for another assault; he pled guilty, and there don’t seem to be any penalties associated with that guilty plea except that she got another one-year order of protection. The family split up. He had to move out. She kept the apartment; she got a new boyfriend. He moved to the suburbs, got a house. She had custody; father had them on weekends; they had to go to a precinct to exchange the kids. The father took the kids to a therapist. He also, after this second arrest, he took them to a hospital to the emergency room, on the grounds that the mother was abusing them when she had them; she strangled the older son. She was beating the younger two girls. So he went to court. The mother went to court. Her attorney left because he didn’t think anything was going to happen that day, so she was unrepresented. He testified that he had hospital records showing that she was abusing the kids, in fact, the kids were seen by a doctor and a social worker, and they said nothing happened to these children. But he went up there and claimed it so the judge yanked the kids and put them with the father. The mother still had unsupervised visitation on weekends. This is sort of a mystery. So that was in November. December, ACS is the child protective services agency in New York City, and the Society for the Prevention of Cruelty to Children did investigations; they found no child abuse. They recommended returning the children to the mother. The guardian who represents the children in court recommended returning the kids to the mother. The father hired a forensic evaluator. The first therapist the father took the kids to I’m calling “Dr. A,” and the evaluator hired by the father I’m calling “Dr. B,” just to keep them in order.

He found child abuse and recommended the children remain with the father, so the judge said the kids are going to stay with the father. Then the mother hired a forensic evaluator, and she found no child abuse. Did lots of tests on the mother, found she was average intelligence, and found she was fine and recommended custody to go back to the mother. The judge ordered the kids to stay with the father, but increased her visitation. In March the court appointed an evaluator who produced a report in July. This evaluator did a really thorough review and found that the father had engineered this claim of abuse, said the kids had to go right back to the mother. They moved out to the suburbs. They had been at a different school. They had been with the father for nine months. She said the summer was a good time to put them back in their old school in the city with their mother. And she said the father had done all of these terrible things, but the kids were really attached to him, especially the son whose father was in prison. He had become very attached; he had a lot of issues and was very attached to this guy. So in the end, it took another year or two years. They finally settled with custody of the mother and the father with liberal visitation.

So, isn’t that scary? I put down a couple of reasons it’s scary, and if we had more time, I think there are about five reasons it’s scary, but these are my first two: How did all of these educated evaluators — three of them were clinical psychologists with Ph.D.s; one of them was a clinical social worker — how did they look at this family and come up with such different conclusions? And the other thing I thought was scary is, if the court really abused, the mother had strangled the son and was beating the girls, why is she having unsupervised visits all weekend? And then there are a few more. So how did it happen? These are the factors that I think influenced, and I’ll go through it with each one, these are the typical features. What’s typical about it is there are two stories and who to believe. Is the mother abused by the father and he’s alienating the kids against her or is she making this stuff up, and she’s really abusive and dangerous to the kids, and he’s the better parent? How do you know?

The therapist hired by the father saw all three kids together; they were 8, 6, and 4. So one of the criticisms of “Dr. D,” the court appointed evaluator, was that you don’t see the kids at these different developmental stages with different parents and different issues; you shouldn’t be treating them together. And this gave the kids the chance to provide this united front and talk about what the mother had done to them, and the father is paying the therapist, which is another issue. The evaluators we interviewed said they really believed the child protective services that they do, on the whole, good investigations. Yet that didn’t prevail in this case. They did pretty well, they interviewed the children, the mother, the neighbors, two of the neighbors were friends of the father and said, “You know he never abused her, but she did abuse the children.”

Five minutes? I thought you were going to tell me half way. Okay, rushing along.

The court-appointed evaluator did a really thorough investigation. By now the case file was this thick, looked at everything, looked at the documents. Dr. B, the evaluator hired by the father only interviewed the children. One of my conclusions, just to jump ahead a little bit, is relying on these clinical interviews can work very well. Some of these clinicians are amazing at teasing out the truth from the kids, and they’re very clever. They do things like they show photographs of the family that they got from the mother or the father; well this doesn’t look like what you are describing to me. But others are terrible. They just ask outright, and they get whatever the parent has told them, and they don’t seem to be suspicious when the kid is using language that isn’t really appropriate for their age, so obviously they’re rehearsed. So on the whole, I think it looks better if they really do the investigation even though the judges say they’re not supposed to. There are all of these documents available. Here is what they actually look at. The majority are not really looking at these records, which is a problem. How did they determine the truth, consistency, if they can do a good interview with the kids?

Timing can influence whether they think it’s false. “I can’t do this” — what they look for to see if it’s dangerous. Parental alienation is so controversial, and I came into this with my own set of, “We don’t use that language, this is a myth —” There’s no way around it. It’s going on in these cases all the time. Some of the kids are coached to say things like these three children, “Mom strangled me.” In the end, when the final evaluator did the interviews, the kids said different things each time — she saw them three or four times — each time the story was different. “Mom lies, dad lies. He hits us, she hits us.” So it went back and forth, and it really was clear there wasn’t much going on there.

They were very confident in their conclusions. This is a little disturbing because they really can be wrong. And there’s a much more egregious case, where the judge followed the evaluator, and the evaluator was clearly wrong.

OK, the evaluator is the key person. I saw it a few times when the evaluator says, “There’s no domestic violence here.” And the judge says, “There’s a medical record; I’m giving custody to the mother. I’m going to put protections in place.”

The evaluators complain that the judges don’t read their reports. They really need to read them but read them skeptically and look at the methodology. OK, I’ve already said all this — sorry, three minutes. I’m good; I’m on my last few slides. I think the judges should specify the scope of the evaluation and not limit it to just do interviews, a single interview especially, but sometimes multiple is just not enough.

Some of these evaluators know nothing about domestic violence. There was a case where a psychiatrist interviewed all three parties and said the mother’s allegations were bizarre — they were actually fairly typical. She asked the father. He said, “No, this never happened.” And she said, “Well, there it is.” So she gave custody to the father, and the mother had no visitation because she thought she was fabricating the fact that the father kept her captive and that the father looked at pornography and was sexually abusive to her. And she said, “This doesn’t happen” — the psychiatrist said. So read the reports and question those conclusions.

We still don’t know what’s best for kids. In this sample case I was talking about, the father had alienated the kids from the mother, gotten them to make these accusations, nobody knew what to do about that, because the kids really needed him. And they were very attached to him. So they sort of had no way to deal with that. They were going to go on spending a lot of time with him even though he was going to continue to try and turn the kids against the mother. And we don’t know what to do, so we need further research in these areas. And really, there should be more follow-up; everybody says. They need to be brought back to court, follow-up evaluations. Is it working? And what’s going on? What are the developments, and can we now see the picture better now that we’re like a year away, and is it safe for the kids?

I had quantitative data that I put into the handout, but I think Judge Koch is going to talk about that.

[Applause.]

Dale Koch: My job is to try put a little judicial perspective on the research and maybe what questions it raises for judges and judicial education, so my thoughts are going to be a little bit random in terms of the research that is being done at this point, partly because both these projects still have some work to do before they can really reach all of their conclusions. What they’re doing at this point is raising questions for us and not necessarily providing all of the answers. One of the thoughts from Dr. O’Sullivan’s research project at this point. One of her first assumptions that’s contained in the project is that custody evaluations have a great influence on judicial decision-making. And, that’s really true, but it does depend, and it should depend, at least on the quality of the evaluator and the quality of the evaluation.

It also raises questions in my mind, andthe case example that was provided here raised some of these questions, about whether we should be looking to having dedicated family court benches. In other words, whether we need to have judges who are really trained on these issues and just hear domestic relations cases or at least that’s the primary part of the work that they do. Whether we should be looking at the model of “one judge, one family” model so that in this situation that Dr. O’Sullivan is talking about, whether these cases are always coming back in front of the same judicial officer or coming in front of different judicial officers. It raises questions by judicial education and the need for the education on domestic violence issues, on child development issues, on parental alienation issues, on child abuse issues. It also raises some questions in my mind about the appropriate roles and expectations of the evaluators and the decision makers. I think Dr. O’Sullivan’s comment about the fact that the expectations for the evaluator should be made very clear.

The concept that we are going to ask somebody that’s an expert — at least we’re assuming they’re an expert — to go out and do an evaluation and not come to us with a recommendation in regard to that evaluation, that somehow just because we got elected as a judge or appointed as a judge that we’re smarter than the person doing the evaluation doesn’t make a whole lot of sense to me. So I would agree with her initial assessment that we need to be providing better direction and then asking for recommendations rather than just partial evaluations. But, if we’re doing that, the evaluation has to be worth something to us and the other point raised — and I am probably going to touch on it a couple of times — is that unless the evaluator is looking at collateral sources of information, that’s probably not a very reliable evaluation for us.

I guess I would contrast this, what the judge’s role is in a family court context versus some other context we’re in. Judges in a family court context have a tendency to view their role more traditionally, in terms of being very passive. We receive the information that comes in front of us. We don’t do much ourselves in terms of getting information, whereas in the context of a civil protection order for instance, particularly in the process of issuing a civil protection order, judges at this point are much more active. They’re looking at other court files that are in their court room. They’re looking at, in our case an OJIN printout, which tells us all the different filings that have been made in that file. We don’t do that in the family court context. We perceive ourselves as being back in this more traditional role so it’s an interesting dichotomy, and we rely upon others then to provide us that information, including the attorneys and the evaluators. And that I found interesting.

Going to the research done by Dr. O’Sullivan, in terms of what the evaluators were doing, in the cases that she was looking at and in only about 1 out of 2 cases, only about half were they looking if there’d been a CPS investigation, were they even looking at the CPS investigation. In only 1 out of 3 cases, if there’s a police complaint, were they looking at the police reports and that underlying information.

Now it’s not that because a police complaint was made that it’s necessarily true. It is an allegation at that point, but it is something that should be raising questions that need to be followed up on and looked at further. And then in only 1 out of 6 cases were they looking at the mother’s medical records. When those records and these — and when I’m saying 1 out of 6, these are situations where the records were in the attorney’s file, so they’re available to look at. And in only 1 out of 6 cases were they actually looking at them. So I thought that that was very interesting in terms of what wasn’t happening, and as a result I think it obviously has a very negative impact on the quality of the evaluation that’s being done. Because how can the judge or how can anyone evaluate the context of what’s going on in the relationship without having the full information? The concept that I find troubling from a judge’s perspective is this concept that somebody can just by talking to somebody and looking them in the eye, figure out if they’re telling you the truth or not.

That’s what evaluators are oftentimes doing, and that’s what theoretically we’re doing up on the bench. I could not look somebody in the eye when they’re up on the stand and tell you if they’re telling me the truth or not. Last case I did Friday afternoon before I got on the plane this weekend, two people came in, no children and no allegations of violence. The issue in question, because there’s a temporary support, who’s going to pay the bills, is did mother, two months before have a shoe box in her closet that had $30,000 in cash in it? He said she did, and she said she didn’t.

Well, to tell you the truth, and I told them, “I don’t have a clue which one of you is telling me the truth, and I don’t have any way to figure that out. I know that you both lied to me about the income situation because it’s totally inconsistent with what’s on your tax returns, or you lied to the government, one or the other, but I can’t find that truth.” And in that situation I didn’t have to make that decision, there were other basis on which I could make my decision about what the support issue is going to be. But in the context if they were coming in making an allegation regarding domestic violence, just talking to them there, I wasn’t going to make that decision, but there may have been a lot of collateral information out there that would help me make that decision appropriately.

The role of the attorneys, where there are attorneys, just can’t be underestimated. The best case I ever had an attorney try in front of me, regarding a custody situation, where there’s an evaluation, was a situation where there were allegations among other things of domestic violence — among other things — in the relationship. And the two parties had agreed on somebody to do a custody evaluation through their attorneys. That evaluation was done, and the evaluation came back and recommended custody to the father. The mother wasn’t satisfied with that, and she decided to change attorneys — a very good choice for her at that point. And the rule that we have in our court is we will not order a second evaluation. If the parties have stipulated to an evaluation and we’ve signed the order proving that, we will not order a second evaluation. So that attorney did what the evaluator should have done in the first place. The attorney went and looked at all of the collateral sources that could have confirmed or proven not true the assumptions that the evaluator made when they reached the conclusions for the report.

For instance, and in relation to an instance of domestic violence, where the evaluator concluded, because there had been no complaint filed, that it wasn’t worth looking at, that it must not have been true. And therefore, that’s part of the pattern of the mother making up stuff. The attorney subpoenaed him to court, the police officer who took that report and the eye witness to the event. And those people came in and testified, and it was very clear that there was the significant event of domestic violence that occurred. If the evaluator had even taken a look at the police report, which they hadn’t chosen to do because there’d been no complaint filed, probably would have at least raised some questions in their minds. So, I think it’s extremely important that the attorneys don’t just accept the evaluator’s report either in terms of what comes in front of the court.

Now, just a couple of other points, and then we’ll open this up for questions. It’s kind of — the questions raised to me, first from Dr. Saunders’ study, project is could these same hypotheses be applied to evaluation of judges? In other words, if you’re assuming that evaluators have these predispositions and that their decision-making might be based upon gender, personal family history, training, is that true of judicial officers as well who are making those decisions? And the answer as our Supreme Court nominee indicated in some of her remarks is, “You know how I grew up probably does make a difference in terms of how I view life in general and the lens in which I view cases that come in front of me in a court room.”

Now, it’s probably a little more delicate to do that kind of examination of judicial decision-making than for evaluators, and partly that happens to us on a daily basis anyway because the decision-making the judges make is much more visible than other decision-making because it occurs in an open court room; it’s a reported proceeding, and everybody can come in there, and as you all know, judges have been criticized from both sides about how they view domestic violence issues. But yet it’s true that the lens that judges view cases through does affect that decision-making process. It has to. The decision-making that I do now is a lot different than what I was doing 15 years ago. My gender hasn’t changed, my background hasn’t changed, but my education’s changed significantly, particularly on domestic violence issues.

The other context in which we actually take a look at judicial decision-making, and I touched up on this earlier is what court room we’re sitting in. In other words what hat we’re wearing. It’s interesting in a judicial institute that we do for judges on domestic violence issues, we do an exercise where we give the judges three different scenarios. And in one scenario, they’re sitting as a judge in a dependency court in a child abuse court. One scenario, they’re sitting in child protection court. And the other, they’re sitting in a family court. And we ask them to make decisions. They’re given exactly the same scenario, but it’s interesting. Their decision-making is different because they’re viewing it from a little different context.

The other thing that I don’t think we can lose sight of here in this discussion, we’re talking about evaluators; we’re talking about cases with attorneys. The majority of the cases that come in front of us, there’s never an evaluation done. The majority of the cases that come in front of the courts, one or both of the parties are unrepresented. So, what we need to make sure we do with the research, I think, is it needs to go out beyond the events, evaluators and the researchers and the lawyers. It does definitely need to get to the bench because unfortunately judges are called upon to make these decisions all the time without proper evaluations.

So, I guess my question is where do we go from here? My question last night to Dan was, “OK, we’ve got this great research, so what?” But there are ways that this can use us, help us to make better evaluations to look at the role of attorneys in this process, to look at what weight we should give to evaluations, and to help us take a look at, properly take a look at the context of domestic violence in relation to the decision-making we’re making. So, thank you very much.

[Applause.]

John T. Picarelli, National Institute of Justice, U.S.

John T. Picarelli: ... interest of trying to keep us on time, I'm going to get started in the hopes that well have a few late stragglers, but that's OK.

[Inaudible]

Picarelli: Excuse me?

[Inaudible]

Picarelli: You want me to speak like this? OK.

[Laughter.]

Audience Member 1: Just speak up.

Picarelli: Hi everybody.

[Laughter.]

Picarelli: I'm John Picarelli. I'm with the National Institute of Justice and its International Center. And its my privilege to welcome you to this panel today on the International Trends in Fighting Child Pornography, as one of my colleagues has corrected me.

This panel came about mainly because of the mission of the International Center. The mission of the International Center at NIJ is two-fold. One, of course, is to focus on international criminal issues, and I can think of no other issue that is more international and more internationally challenging than child exploitation. And the other is to act as a bridge between the United States and foreign countries in the importation and exportation of both ideas and technologies. And what we have here today are three panelists that represent that ideal that both through their concepts and through the use of technology are building international coalitions in order to fight child exploitation online.

And so today what you will hear is the truest form of what we call ideas across borders. The idea that borders may act as an inhibitor at times but we need to transcend those borders in order to deal with many different forms of crime. So today well have three speakers. The first is Catherine Cummings. She's the executive director of the Financial Coalition Against Child Pornography at the International Center for Missing and Exploited Children, which is closely associated with the National Center for Missing and Exploited Children, right here in Alexandria, Va.

Then, after Ms. Cummings speaks, well have Bjrn-Erik Ludvigsen, who is a police superintendent with the National Criminal Investigative Service in Norway, but also serves as one of the coordinators for what is known as CIRCAMP, the COSPOL International --Internet-Related Child Abusive Material Project. COSPOL is another acronym that's basically an effort through the European Chiefs of Police Task Force in order to focus on criminal issues that transcend national borders in Europe. So this is one of these thematic networks, and Bjrn-Erik will go into more detail on how that works.

And then following Bjrn-Erik, well hear from Per-Ake Wecksell, who is a detective inspector with the Swedish National Criminal Police in Stockholm. Per-Ake is in a position — a more interesting position in that he's working with both. He is the primary liaison for the CIRCAMP effort in Sweden, but because of the Financial Coalitions work within Sweden, he has had contacts with them as well. So hell focus most of his remarks on CIRCAMP, but you may hear him mention how the Financial Coalition operates in Sweden.

So each will have 20 minutes to speak. Afterwards we should have about 20 to 30 minutes for question and answer, so I hope you'll have plenty of good questions to throw at them. So without further ado, Ms. Cummings ...

Catherine J. Cummings, Executive Director, Financial Coalition Against Child Pornography, International Center for Missing and Exploited Children, Alexandria, Va.

Catherine J. Cummings: Good afternoon everyone. I appreciate the introduction from John, and I appreciate the opportunity to speak with you this afternoon about a very critical problem that many groups are fighting globally. The group that I will describe to you is the Financial Coalition Against Child Pornography. It started in 2006, largely as a U.S. initiative, although we do have a good number of global companies involved. Were very active now and expanding the network and the concept to other parts of the globe. And its very nice to see my colleague from Sweden, Per-Ake, because we've seen each other at a couple of meetings in Stockholm, which is a very nice city, by the way.

So what Id like to do is just tell you quickly about the sponsoring organizations for the coalition, and then a little bit about the problem were trying to solve, and then Ill tell you about the coalition.

As John mentioned, the National Center for Missing & Exploited Children is based just down the highway in Alexandria. I would assume that some of you in the room are familiar with this organization, since we work very closely with the Department of Justice and the Department of Homeland Security to provide services for families and law enforcement in the United States. The National Center is celebrating its 25th anniversary this year and plays a role, or originally was started to play a role to link law enforcement across the country so that everyone could do a more effective job of helping to find children who had been abducted and gone missing.

The group has expanded its mandate, and in addition to supporting law enforcement and missing children, we also focus on child sexual exploitation issues. In 1998, the United States Congress asked the National Center to set up a hotline, or a 911 for the Internet, which we call the CyberTipline. The CyberTipline receives reports from the general public and Internet service providers when they find suspicious sites or other elements that have to do with child exploitation on the Internet.

The sister agency is the International Centre for Missing & Exploited Children, also based in Alexandria. We work to combat child abduction and child sexual exploitation globally. Some of the ways we do that is by providing training and assistance to law enforcement, legal professionals, governments, and NGO's around the world, as well as advocating for changes in laws, and treaties, and systems to help protect children worldwide.

A quick side note, particularly since so many people at this conference have a background and an interest in research. The International Centre compiled a study and released it, initially in 2006, and it actually has been updated five times. You can find it on the Web site: www.icmec.org. This is a study of child pornography laws around the world in 187 Interpol member countries, and what we found is more than half have no laws at all that speak to child pornography specifically. So part of our work is to try to build a global landscape that is more consistent and more effective when it comes to fighting child pornography.

A little bit about the problem. The Financial Coalition Against Child Pornography is focused on commercial child pornography. There is a very robust trade of these images and videos between the offenders or the fans of this stuff where no money changes hands. Its peer-to-peer trading and things like that. What we focus on is the commercial — a lot of times its organized crime that collects these images and sells them, or collects these videos and sells them for subscription prices, and people buy monthly or yearly subscriptions to these Web sites.

It is a global problem. It has been fueled by electronic payment tools like credit cards, debit cards, PayPal, Western Union, as well as newer currencies that are popping up. When we talk to people about this problem — I doubt this is true with this group, but other groups will say to me, Well, isn't this really adult entertainment or adult pornography where women might be dressed up to look like teenagers or underage girls? And that is not the case. Tragically, there is a study recently done that the National Center was affiliated with, that looked at the images held by the people arrested. And 83 percent of those arrested were holding images of children 6 to 12 years old; 39 percent had images of children 3 to 5 years old; and 19 percent had images of infants and toddlers under the age of 3 being raped and sexually abused. And probably the individuals who are on the front lines fighting this will tell you that those trends are getting worse and not better.

Why is this a business? Why do we put together a coalition of Internet and financial companies that has anything to do with child pornography? If you can step away from the emotion of the problem and look at it from an economic standpoint, children around the world are plentiful and easily accessible, unfortunately. Child pornography is easy and inexpensive to produce because of technology today. There is a very significant consumer market for this in the United States and outside of the United States. It is enormously profitable, and there's virtually no risk, although I can tell you point number five, we are changing that through the very concerted efforts of law enforcement around the world, as well as help from private industry, which is where we come in.

These are some of the solutions to fight the problem. And I'm going to talk to you about the last bullet. The Financial Coalition Against Child Pornography has one goal, which is to disrupt the economics of this business. Law enforcement, arrest, prosecution is always the first priority in anything we do, but this is a civil initiative. So it was formed on the premise that we cannot arrest and prosecute our way out of this problem. So in 2006, Senator Richard Shelby of Alabama, who at the time was the head of the Senate Banking Committee, convened a meeting of some of the credit card and financial companies and said, If we were buying and selling heroin and cocaine over the Internet and people were using their credit cards, we would do something about it, and this is far worse.

So, I am happy to report that the companies have stepped up to the challenge. This is a list of our current members, and it might be hard to read so Ill read off some of the names for you: the American Express Company, Banco Bradesco of Brazil, Capital One, Citigroup, Deutsche Bank Americas, Discover, Google, HSBC North America, JPMorgan Chase, MasterCard, Microsoft, PayPal, Visa, Wells Fargo, Western Union, and Yahoo!. Importantly you'll see its a collection of Internet companies and financial companies because together we all do a much better job of fighting this problem. Each of the companies on the list does whatever they can to fight the problem but we have learned that we can really make much more progress if we share information and help one another.

In 2006, the first priority was to develop a process for undercover credit card transactions in collaboration with law enforcement. A little bit of background. If a company like Visa or MasterCard or one of the banks suspects fraudulent transactions on their system, the security and risk people at those companies will do an undercover transaction themselves to follow the flow of funds and find the bad players who are in the payment stream. In the United States the mere attempt to purchase child pornography is a crime. So if you're working for MasterCard's security department, you cant do an undercover test transaction or you're going to have the FBI knocking on your door saying, You have an employee who is trying to access child pornography.

So because of the relationships that the National Center for Missing & Exploited Children had and has with law enforcement, we were able to recruit Immigration, Customs and Enforcement, FBI and some other law enforcement agencies to help us out with undercover transactions. Then we built out that tip line that I mentioned to you, to accommodate information from the financial companies. Something that I want you to keep in mind is that all of the sharing of the data is well within privacy and data protection laws. We have a lot of lawyers at every meeting and they keep us honest. Also, the process is to follow the merchant side of the payment stream, not the consumer side. Any action that might be taken against consumers would be taken by law enforcement. What were after is to shut down the payment account that a merchant has fraudulently opened — lets say on the credit card system.

How am I doing on time, John?

John Picarelli: ... 10 minutes left.

Cummings: OK. Great. Thank you.

Id like to take you quickly through the CyberTipline process so you get a sense of how the financial services companies interact with this. A URL is reported into the CyberTipline, as I mentioned, by a member of the general public or an Internet service provider. And that can be done by phone or via the Internet. An analyst at the National Center for Missing & Exploited Children visits the Web site and confirms that it is commercial child pornography. An undercover financial transaction is conducted by law enforcement. Right now were doing them with credit card accounts that have been contributed by some of the banks who are members. Specific details of that transaction are provided to the payment company. So an American Express or a PayPal or a Discover will get an e-mail or an alert that says, such and such a date, an undercover transaction was performed by ICE for this child pornography Web site. Please keep an eye out for when that transaction goes through. And sometimes it doesn't go through, and Ill tell you about that in a moment. When the transaction goes through the payment company isolates it and identifies the business and the location of the merchant responsible for selling the child pornography. The payment company then goes into the CyberTipline, and this is through a virtual private network, a very secure network, and only very specific companies have access to it. They amend the original report to include information on the merchant like city, state, location — or country. Also the merchant bank, city, state, country, et cetera. And other details.

The completed CyberTipline report is available to federal law enforcement for 10 days, where they can take the tip and say, Were going to add it to an ongoing investigation, or we might — they wont tell us the exact details, but they say, Were going to keep this. And if that is the case — I beg your pardon — if that is the case, if law enforcement indicates further action, the entire process stops and the financial industry is notified of a possible investigation. And then they're given instructions about what to do next as it relates to that merchant account.

In box number eight, if law enforcement declines to take the tip, the financial company is alerted and pursues the violation of association rules or merchant agreements that they would do under normal conditions when they find illegal content on their system. That basically means that they contact the merchant bank and say, You have this player on your system who is selling this contraband. Please close down that merchant account immediately. And that is what happens.

So what I just described to you is the backbone of what we do in the Financial Coalition. What we have found that it is a great form for helping each other in other ways such as disseminating trend information and other data about these Web sites. It might include price points. It might include key words that the commercial child pornography trade now uses to attract people to their sites. And so we send out these alerts, this type of information so that an AOL, a Yahoo!, a MasterCard, everyone is able to run this information against their systems and do a better job of cleansing their systems of these businesses.

We have also published two thought-leadership pieces: a white paper and a best practices document. And we have found a lot of benefit in aligning with other industries. In this case, specifically the mobile telephone and mobile provider industry in the form of the GSMA based in London, as they try to get ahead of the mobile Internet being available on phones, and people accessing this content via phone. So more and more we are collaborating with that industry.

Now people will ask us, OK, you've been in existence since 2006, how do you know anything is happening as a result of your efforts? They ask us quite often, What about arrests? We do not measure our success by arrests because we are a civil initiative. But the things I can tell you is that the subscription prices to these Web sites has gone much, much higher. It used to be $29.95 a month. Now they can be $300, $500; sometimes we see prices as high as $1,200. Its increasingly difficult for law enforcement to do an undercover test transaction with a traditional payment tool, meaning credit cards, debit cards, PayPal, et cetera. And if its more difficult for them to do it, we've got to assume that the average consumer is having trouble as well. We've gotten very good feedback from law enforcement, and fewer commercial child pornography sites are being reported into the National Centers CyberTipline.

But this is a global problem and it requires a global solution. We have been very active in regions around the world supporting efforts in any way we can. You cant just take this U.S. model and plop it down into another country for a host of reasons. But there are things we've learned, successes we've had, challenges we've had that we are sharing with other regions of the world, and we are making great progress. You'll hear more about the Swedish Financial Coalition, and we've played a small part in that, and were very proud of that, and proud of where that group is headed. And a little while ago, a couple of weeks ago, I got back from Tokyo, where there is significant interest in the Asia-Pacific region. And some things brewing in Latin America; that's going to take a little while longer.

I very much appreciate your attention and would be happy to answer questions or further the conversation after our speakers are finished. Thank you very much.

[Applause]

Bjørn-Erik Ludvigsen, Police Superintendent, National Criminal Investigation Service, Oslo, Norway

Bjrn-Erik Ludvigsen: Good afternoon ladies and gentlemen. My name is Bjrn-Erik Ludvigsen. I [am] with the Norwegian National Police. Would be the equivalent of the FBI without the big budget and the three-piece suits.

[Laughter.]

Ludvigsen: I'm also the project manager of CIRCAMP, the COSPOL Internet-Related Child Abusive Material Project. We do a lot of aggregations, and we do it because we want people to actually read what it says. And I will explain a little more why were called that and what other names we use.

I'm going to talk a little bit about how we came about. In 2004, OCTA, the organized crime threat assessment within Europol, tried to look into their glass bowl and see — or the crystal ball — to see what type of crimes would be a challenge for us in the future. And they usually try to see things that go cross-border or international because none of the law enforcement agencies in Europe or here in the U.S. are able to handle some of these crime types by themselves. Then they give the job to the European Police Chief Task Force, the EPCTF, and they started up these COSPOL groups. It actually stands for Comprehensive Operational Planning for the Police — no sorry, Comprehensive Operational Strategic Planning for the Police.

And they identify the different types of crime like drugs and also the distribution of child abuse material. Then they started our group. Our group was originally called the COSPOL Group on Child Pornography. One of the first things we did when we had our first meeting — and I don't mean to offend anyone because you have child pornography in your names, but we changed the name to CIRCAMP. We don't deal in child pornography; we deal in child exploitation and child abuse images and films and files. Pornography is legal in most countries. Pornography is consensual in most countries. Many people use pornography. The raping of a child is not pornography. That is our stand, anyway.

We are the project manager from Norway. You see the other countries that are members. Being a [forerunner] country means it is a country that is supposed to try out different things that we do and see if it works. And if it does, well try to translate that to other countries that are not members of this group. So we try to make kind of a standard of how to do things.

Up, sorry. By the way, were also supported by both Europol and Interpol so we have the international level covered with both communication and strategic and operational handling of information.

I'm not going to read all this to you. Our primary goal is to try to make it difficult for bad guys to make money off children being sexually abused, especially on the Internet. We also to find a common way for us to police the Internet. In our view, the Internet is just a continuation of general society and national laws should apply on the Internet. That part of the Internet which is located in my country, the Norwegian part of this network, Norwegian law applies. Its not a different dimension; its not a different country; its not a different place. Its just a bunch of computers and wires tying things together. And people are doing stuff on the Internet that needs to be policed.

We think that child abuse material has a harmful effect on society, and we would like to try to limit that as much as we can. One thing is the people like Catherine was talking about, the ones who buy child abuse material, you want to stop them but you also want to stop the people that are unwillingly being exposed to this kind of information.

We started out doing what every police Internet project does. We started going after the peer-to-peer people because its easy, because they're plentiful, because they distribute really severe child abuse material and we know how to take those people down. We know how to secure the evidence. We know arrest them. And we know how to search their houses and their computers. This is something we did from 2004 to 2006. And then we kind of figured out, well this is something we can do by ourselves; we don't need an international group to do this. I can do this in my country, as we did actually in 2004, and make all these cases and send them around the world. In 48 hours in 2004, we generated 13,000 cases all over the world — 2,000 cases for the U.S. It is an easy way to make cases. And we decided we don't have to do that as a group.

So we went to the police chiefs and we said we want to do something else — were going to target the child abusive material that is distributed commercially, especially that on the Web. And we made a three-point action plan that we tried to follow. The first part being to block child abuse material. We have done this — were now in 2006; we had done this in Norway since 2004. And we said this is a good idea but Norway is not really the center of the earth. We need to have more people that work with us. So we suggested that would be part of the plan. The second plan would be to go after the legal element, the payment systems, like Catherine said. Identify them, tell them that they are actually being abused so that they will cut off the merchant. And the third part is try to arrest the people who actually make money off this. This is, of course, the most difficult ... most difficult part, and we haven't really succeeded much in that.

I'm going to talk about the blocking today. We want to prevent the children being abused again and again as masturbation material for adults, because that's what they use it for. They don't collect it because its like stamps. They use it to masturbate. They are sexually interested in children. The distribution is illegal in most countries that have any kind of child abuse legislation. As Catherine said, many countries do not. But those that have, distribution is always illegal, while possession may be legal still. In my country, display is illegal. If you actually go to a site looking for this kind of material willfully and repeatedly, that is a crime by itself, because you don't have to download anything anymore. Everything is online and with the speed we have on our Internet connections, you don't have to possess anything in the ordinary sense of the word.

And by reducing the access to child abuse material we think that we can reduce the market. Take away the customer, there is no need to produce and offer as much material. This is, you know, capitalism at its best and its worst. If there's a need, there will be a supply. Take away the demand for child abuse material. And we do it by targeting the whole chain: the contents, the links sites, the payment sites, anything we can see is a link in the commercial distribution of child abuse material.

And we wanted also to raise the awareness in the population — both the ones that look for it and the ones that are accidentally exposed to it. And with this I'm making another aggregation, of course. And instead of calling it the child pornography filter, we call it the child sexual abuse anti-distribution filter because that's what it does. It limits the distribution of child abusive material. And this is how it works. Well, this is really simplified, but you'll get the idea. This is kind of like how the Internet works. Most of you, if you ever spend time on the Internet know that the Internet does not work with letters. It doesn't work with words, it works with numbers and IP addresses. So when you type something into your browser, it needs to be translated to an IP address so that your computer — your browser knows where the content you're looking for is. And it does this by going to a DNS server, looking up who is responsible for childporn.com, in this case. And the DNS will say its IP address this and that. And your browser will connect directly to that server. It may be in Russia, it may be in U.S. You don't know. And if you punch child pornography or childporn.com into your browser, you will get childporn.com. Of course, I couldn't show you how this site looks, as I didn't want to spend time in your jails.

[Laughter.]

Ludvigsen: You have really good laws against the people you arrest for this crime, really good laws. Unfortunately, of course, you don't arrest as many as I would like but when you get them you actually give them a good sentencing, which I cant say about my own country unfortunately. Well, you type in what you want, you get what you want. This could be your online newspaper, this could be children being raped. By introducing a new layer where you will still punch in what you want — the child pornography or the childpornographysite.com — you will check it against a list of domains that we have given to the ISP. All of the domains on this list have been checked by the police. The contents have been seized. We have downloaded everything. Everything is traced and saved and secured. And then we say that the images or the content on this site is in breach of our laws. Our national legislation says that this is illegal to distribute. Then we will put it on this list. The ISP will check it against that list when you try to access the site and if its on the list, you will get this site instead. Instead of the child abusive material you saw before, you will get this stop page. And we will put — this is the Norwegian one, by the way. And I hope you can read what it says.

We don't say that you have been trying to access child abuse material, but we say your browser has been trying to access child abuse material.

[Laughter.]

Ludvigsen: So were not pointing any fingers here. And the reason for this is that, quite frankly, some people are not that intelligent when they are on the Internet. Some people are mindlessly surfing. Some people are clicking everything that looks like a link, even if they get it in an e-mail. Some people surf borderline pornography. They look ... they'll go for the teen sites with the 18 and 19 year olds. And they're very quickly linked to sites that contain children. So its not always a willing act to end up on one of these child abuse material sites. That's why we said your browser. In addition, there's Trojans and viruses and all kinds of bad things that will redirect you to sites that you don't want to go.

In addition we say that we don't trace anything. We don't know who you are. We don't make any cases from the people that end up getting this site or this page. This is just a pure preventive measure against the distribution of child abuse material. And in my view, prevention is the most noble of all police work.

Also we have information about the operation of [indistinguishable], the CIRCAMP group. Just to tell people, this is not something we only do in Norway. This is something we do together because, as you all know, the Internet is not limited to one country or one continent.

These are the countries that are in CIRCAMP that run the blocking system right now. And what happens when I find a new site, I download and [indistinguishable]; I do all the things the police should do when they secure evidence, and I will share it with all these guys, so they don't have to go looking for these kinds of sites on the Internet. They will just get it from me, check it according to their own legislation. If its illegal in Finland, then they will just add it to their local list of sites that I've blocked. So even though we share all the material, all of this will look different because we all have different laws. In my country, for instance, almost everything is illegal. Even computer-generated, drawn pictures are illegal if it depicts a child.

These are countries outside of our little group that we also work with. So New Zealand on the other side of the world and Switzerland, its a European country, but they're not in the European Union. And also there's the United Kingdom that has a similar system driven by the Internet Watch Foundation, which is a non-governmental organization, and is financed by the Internet industry.

These are just some examples of stop pages. This is the Swiss one, Danish, Finnish, Italian, New Zealand's — New Zealand's The Swedish one, which I think Per-Ake will tell you about. And the one from Malta. And this is very important people, because this is not the Holy Grail of child abuse material fighting. This doesn't stop any of the child abuse material from being produced or from being distributed to other countries that are not members of this.

How am I on time?

[Inaudible]

Ludvigsen: Oh. So this is something we do in addition to ordinary police work, not instead of. So you cant introduce blocking and say, Oh, well problem solved. I don't have to do anymore. Then your work really starts. But then you will have a huge chunk of work that you don't have to do. Again, if you are able to limit the access to that material.

This is just information about our project. You can find it on the Europol page. This is on the Interpol page, on the THB, the Trafficking and Human Being Web site for Interpol. We also have our own Web site. Its on CIRCAMP.eu, where well have — its kind of limited thus far, but we will have more information on that site about terminology, which I think is very important. And a little bit about how we think when we try to limit this kind of material being distributed.

Our plan ahead is to continue the work that we do now until 2010. That doesn't mean that were going to quit doing this in 2010, but CIRCAMP as a group will probably have another aim. And the countries that have the blocking will continue to do that on their own account. We plan to make a worst-of list of domains. And a worst-of would be — because now, the list we have in Norway is based on Norwegian legislation. In Sweden, its Swedish, et cetera. But there are a number of countries where we don't have any contacts with the police. The police don't have the people, the knowledge, the interest to do so. And we plan to make a worst-of list of sites that would be illegal anywhere. That means it will be with children that are younger than 13. It will be the most horrific abuse of children, or very particular sexual focus on any of the images on that site. It has to be a real child, for instance. Not computer-generated, which is legal in some countries. And we plan to distribute this through the Interpol National Central Bureaus, because every one of the 187, was it, Catherine? Yeah. Every one of the 187 countries had a contact bureau for Interpol, and we need to spread this list of worst-of sites to all those contact points and then the ISPs so the access service providers can contact the Interpol police office and get access to the group and implement it, based on their terms of service, their ethical standpoint or their policy. And we are often contacted by Internet service providers that want to do something good for the Internet, want to make the Internet a better place, but have no one to turn to because the police in that country are unwilling or unable to provide a list according to that country's legislation. The list is something that we will do.

We will also continue our cooperation with Germany, for instance, which actually wrote contracts with six other larger-sized ISPs in April that will start blocking very soon. It is very important for us because it is a big country, or by European standards its a big country with 90 million people. And we will work with law enforcement in any country — any country in the world, although we are a European project.

To sum up the whole thing, we are a very sharing group, but we will only share with law enforcement when it comes to child abuse material because we look at child abuse material as evidence of a crime. Its a picture that usually the perpetrator himself has taken, and it shows the crime scene, and it shows at least parts of the perpetrator, and it will show the victim. And all these things are important things for the police to use to try to identify the child and try to stop the abuse. So that's why we will only share child abuse material with the police. Well happily work with any NGO on other things, but we will not share the images with them. And we will share every smart thing that we've ever thought, any software that we ever made and any contract that we have ever written with any ISP to bring people up to speed much quicker than we were when we first started.

The benefits are that its very cheap. This is — some people will say differently, and especially some ISPs will say differently, but this is actually pretty simple, simple readdressing on the Internet and they do this every day. This is not brain surgery in any way. We use ordinary software, ordinary techniques, off-the-shelf things that you can buy. There's nothing really peculiar about the operation. I know Per-Ake will talk a little about a special software that we have, but you don't really need that.

It is very effective. Compared to what you invest in this system and what you get out, its very, very effective. I'm not going to give you so much numbers — actually I can see from my country, but we have very strict rules, we have about one display of this stop page that I showed you before per 300 people in my country per day. So we have like 15 to 18,000 displays of our stop page on a 4.7 million population. So it is quite a number of people that are actually looking for, or accidentally end up on child abusive sites.

And we believe that it is preventive. It will give people looking for this willingly, a kind of a heads up in saying, What are you doing? It will give people that are swimming in murky waters kind of a way to step back, age-wise, when it comes to the pornography that they use. And it will give the people that are not-so-knowledgeable when they surf the Internet a way not to be exposed to children being raped, which they easily can be.

That is the presentation that I had planned, and Ill be happy to answer any questions when the Q and A starts after Per-Ake. Thank you.

Per-Ake Wecksell, Detective Inspector, Swedish National Criminal Police, Stockholm, Sweden

Per-Ake Wecksell: Ill have to put this down again. My name is Per-Ake Wecksell, but you can call me, as Bjrn, Pela. I work for the Swedish National Criminal Police, our anti-crime section on the child protection team. I come from Sweden, the land of IKEA, Volvo, ABBA ...

[Laughter.]

Wecksell: ABSOLUT Vodka, and Greta Garbo.

[Laughter.]

Wecksell: I'm going to talk about the collaboration with the industry in this matter, about challenges that we have met, and finally short about the Swedish Financial Coalition.

As I will talk on a national level now, instead of the CIRCAMP level, I will tell you the Swedish matter. And it all started as an initiative from Norway and Bjrn-Eriks team. It was their criminal investigation team who taught us, and we gathered together the representatives from law enforcement and industry to discuss whether we should have this blocking system in Sweden or not. And suddenly, an agreement was made and a contract was written between several Swedish Internet service providers and the Swedish National Criminal Police.

And this collaboration between the police and the Internet service providers is completely built on a volatile basis and is the crime preventive work. As Bjrn-Erik told, in Norway its forbidden to watch these sites of child abusive images but its not in Sweden. There is a law going to be, but its not still written. So, however, its not forbidden, but it is forbidden to be in possession or distribute this kind of material.

This is crime preventive work. Today we have a collaboration, contracted 16 ISPs in this blocking solution. And they have a coverage of about 80 to 90 percent of the Swedish Internet users. And new contracts are assigned, on the way, and now we are talking to the local networks on the cities around Sweden.

However, one of the largest Swedish ISPs didn't want to participate in this collaboration from the beginning. I don't want to mention their name because its audio taped here, but ...

[Laughter.]

Wecksell: The biggest Swedish newspaper told about it. A poll was made and a huge majority of the Swedish population said they should block — they should contribute to this work against this matter. And the criticism was too difficult for the company, and the next day, after the article was published, they told us that they want to be in the collaboration. But I'm sure that you already had read it.

[Laughter.]

Wecksell: Yeah? You have recently heard Mr. Bjrn-Erik Ludvigsen from Norway talk about how the blocking of a Web site actually works. I'm not going to repeat that. However, we, the Swedish NCPs, receive data information from [the] public regarding child abusive images on Internet sites. We get referrals from ISPs. We get information from the other European network, the CIRCAMP network, and from the EPCOT, the Swedish non-governmental organization, EPCOT's hotline. Also, we got these URLs that were found on computers which were seized all around Sweden during investigations. The role of the law enforcement is to collect those reported URLs and to review and investigate and make a judgment if these Web sites are legal or not, regarding Swedish law. If they are considered illegal, they will be blocked. We create and send a text file to the providers that will see that the sites will be blocked.

Bjrn-Erik also told you about a special program, and here is the program we use to make life a little easier when we deal with this matter. Its created by the Danish police. And we put in a text link for all the information we got, and it [indistinguishable] to have it on the domain level, as you can see. And we click on it and check all these sites, and if we consider them illegal we save the sites and we give them a red flag, as in blocked. If its not illegal, we give them a green flag. You can see, you have this flag here from some country. In this case, its just one country's flag here, but otherwise it will show all the countries where the Web sites belong to. This is a good instrument that we use. The role of the Internet service providers is simply to implement the blocking list or the DNS servers. They have to provide statistic referrals and implement the stop page, which you recently have seen. As Bjrn-Erik told you it says, Your Web browser has tried to contact an Internet Web site, and what penal code is used, in this case, Chapter 16. And where to turn to if one has some complaints to do.

And this stop site is shown double as many times as the Norwegian — 1,000 times per day because we are double the population in Sweden regarding Norway — compared to Norway.

I will show you our written agreement. I will not.

[Laughter.]

Wecksell: I will. OK. This is signed by the chief of the National Criminal Police and the chief of the Internet service provider. I will just go through it very quickly. You see that the corporation is aimed at restricting access to, and circulation of, descriptions of children in pornographic pictures on the Internet, as well as in that way preventing children from being sexually abused. It says about NCPs commitments, the Internet service providers commitments. I hope you all manage to read this now. And it states that the parties shall meet at least once a year, in addition to that when necessary to elevate the corporation. I can say that we meet three to four times a year. The agreement can be cancelled by one of the parties with immediate effect. Now I think I have this in the slides, so you can read it more carefully if you want, later.

OK. So is everything going smooth and nice with this? Yes, it is. But we have a few small challenges. The Swedish Post and Telecom Agency monitors the electronic communications and postal sectors in Sweden. They have been with this meeting for two years, and after two years they came to the meeting and brought up the issue of the agreements between customers and ISP approval to regulate Internet traffic carrying child abusive material. Lack of such regulation could lead to imprisonment for the responsible person at the Internet service providers. I can tell you that participants got their coffee stuck in their throats. This person from the Swedish Post and Telecom Agency, he really liked that, so he told them time and time again about this chance to go to jail if they have no regulation. But they went home and checked their agreements between them and the customers and everything was written down and good.

Another challenge is, OK, you are blocking child abusive images today, but what will the next step be? Bestiality or something else? Is this the first step in blocking the whole Internet or what? Norway started early 2004 and we started 2005 and it will be nothing else. It will stay with child abusive images. I can tell you for that by our countries. I'm sure that Bjrn-Erik will agree with me.

By the end of 2008 we made a new agreement between us and the ISPs for the enrollment of referrals, the page before coming to the Web site, with the stop sign. Some other adjustments were also made. For example, to involve non-commercial pages in the agreement. But it proved controversial to make changes in the agreement. Now were talking about [indistinguishable] and slippery slope again. Two of the largest Internet Service Providers, one that you remember from the earlier slide, and another said they didn't want to help the police with the investigation. And they said that we could do it by ourselves. The investigation, giving us the site, the referrals. I don't know about that, but ... finally we have these meetings and they signed a new agreement. But a response came quickly from ...

[Laughter.]

Wecksell: The first opportunity to give us referrals, and we expected tens of thousands of referrals. We got 50 — five, zero — because the agreement says nothing about how many referrals we were supposed to get.

Finally, the last challenge was BitTorrent, Pirate Bay. Anybody heard about Pirate Bay? You know about BitTorrent? We received information that a lot of the worlds largest BitTorrent tracker had files that contained child abusive images and material. Files, it was films and pictures. BitTorrent is a file sharing protocol that enables file transfers. Pirate Bay offers — uses the latest Hollywood movies, the latest games, or whatever the latest books, e-books. And the tips, information about child abusive images and material on Pirate Bay increased, so we had to decide whether this site was going to be blocked or not. We called a meeting, in a very high level, international police, Swedish National Police, and it was decided that if they will not take away this abusive material over the weekend, they will be blocked. So I sent out this information to the Internet Service Providers, just to prepare them for some Internet attacks, who could be. And I sent it on a secure way to those responsible on the ISPs. It took — then I went to my holiday. I go down to the south of Sweden to meet my son, and I just come one hours drive when the telephone called. Hi, its Pirate Bay. And the name of the guy. Are you going to block Pirate Bay? And directly after we hang up, the Aftonbladet, the Swedish biggest newspaper called. And after that, a lawyer for one of the biggest ISPs called me.

[Laughter.]

Wecksell: Are you going to block Pirate Bay? The circus started. It was a big fuss in the mass media, and we received a lot of complaints by mail. People said, This is going to be the new China. We want these Torrents to live. Keep away from Pirate Bay. Three major Swedish Web sites were hacked, and they put in our stop page instead of the material who belongs to it. One of the Web sites was from one of the most popular soccer teams in Sweden, and their supporters were crazy about us. People thought we were the ones that blocked the Web site. And the circus went on. However, on Monday morning, I went to work. The team which I worked with checked out the Pirate Bay, and the child abusive material was gone. However, the turbulence continued a couple of weeks. But it led to something good. We [had] shaken the Pirate Bay a little bit and now we have a good collaboration with them, and they even put our e-mail address on their Web site.

OK. Results in the end of this blocking. We can see that the child abusive material is less available in Sweden. ISPs take more responsibility. The Swedish public is very much positive about this work. I can show you that we got mail every day, people thanking us. And we also get telephone calls by people who run into these sites, come to the stop page, and they call us and say, What is going to happen now? Are you going to come to get me?

[Laughter.]

So. Yeah. But were not.

This is a political, very well accepted method. And I will say this has a large preventive effect. So, the next step. Like Catherine talked about, there was a Financial Coalition going on here in the United States, and the next stop for us has now been taken. The nongovernment organization, EPCOT has looked at the Financial Coalition in the USA. And I would like to thank you, Catherine, for all the help to bring the Financial Coalition to Sweden. We have a collaboration now between Skandiabanken and the Swedish National Criminal Police. And it differs some from the U.S. Financial Coalition. In Sweden we have no domains or servers that are hosted here, and if we have, we will start an investigation, at least we don't find them. What we do is that we receive a number of accounts from the bank with complete information to make purchases on the Internet, like Catherine recently told you about. So we do it, something like the Financial Coalition in the States do, but not really like that. We have a cooperation between the banks, the Visa, the MasterCard's, and the aim is to prevent and obstruct payments for child abusive material through the financial system.

Yes. I think I will stop there. And here is my address if you want to e-mail me, and I will be glad to — thank you for your attention and I will take questions later. Thank you.

Phelan Wyrick: Welcome to the session this morning. We are thrilled to have the opportunity to present today at the NIJ Conference and talk to you about what I've been sort of calling “What Counts as Evidence?”, and that's shorthand, but it's a major point of discussion these days, more and more discussion around evidence-based practices, evidence-based programs. You hear more and more legislators at different levels, policymakers talking about evidence-based work, and, in general, our experience is that there's a fair amount of confusion about what that means and how these terms are used and the definitions of these terms.

We don't necessarily claim to be coming to full conclusions here today, but we offer you our insights and share with you some of the work that's been done to add some clarity, I think, to the terms and the approaches that would constitute evidence-based practice and programs.

We've seen as — at this time as there's a number of pressures that are bringing this particular issue to the forefront. One of them is the administration itself. You'll recall in his inaugural address, President Obama talked about not the distinction between big or small government but government that works, and focusing in on trying to support those programs that work and discontinue support for things that don't work.

That was a general statement about effective governance, and I think at its core, a lot of what we are talking about when we think of evidence, when we think about what counts as evidence and evidence-based programs, is really at its core about effective governance. But, of course, we're also in an economic crisis, and that puts another level of pressure on states, localities and federal government for us to be thinking about this issue and to be focusing in on using our investments most wisely. As we know, departments, police departments all across the country are reducing their staff levels, losing officers. You've got prisons and jails that are overcrowded. You've got state and city budgets that have to be balanced, and so more and more attention is coming back to this issue.

Today, we've got three presentations, and I will reiterate one of the notes that is in your materials that we've had to reduce our presenters to the three of us.

Mike Farrell is not able to join us unfortunately, but we do have Stephanie Shipman here from the General Accounting Office, and she is going to talk to you about a recent work that they've done and a report that, if you haven't seen it, I highly recommend, on program evaluation and a variety of rigorous methods. And it talks about a variety of rigorous methods to identify effective interventions, and so she'll talk about that work.

And then we've also got Ed McGarrell from Michigan State University, director of their criminal justice programs and a professor there, who's worked extensively on gun programs including the Department of Justice's Project Safe Neighborhoods.

And then I will be talking about work that we are doing out of the Office of Justice Programs that we call our “Evidence Integration Initiative,” and I should quickly introduce myself. I'm Phelan Wyrick. I'm a senior adviser with the Office of the Assistant Attorney General in the Office of Justice Programs.

Stephanie Shipman: All right. Thank you. I'm glad to be here.

I'll preface my remarks, I do not have any background in the justice area. Most of my work is actually government-wide. I specialize at the Center for Evaluation, Methods and Issues in trying to further program evaluation across the federal government, and so we do that through a variety of studies of agency evaluation activities, use of different kinds of methods for addressing various analytical problems, and then we do a lot of outreach, so this is part of that.

And what I wanted to do was, before I actually talk about our report, I wanted to — because I don't want to presume that the answer to the question, what is rigorous evidence, is only what I'm going to be talking about today and in our report, but rather, in terms of prime uses of research and evaluation in program management, I would push hard for — in all program management, that you'd be using performance monitoring and environmental scans to find out what's going on around you in the context that your program is operating, so you keep up to date, and use process and outcome evaluations to find out how to improve your program over time. And then there's a question, particularly at the federal level, about making large, broad-based decisions for national programs, what do we want other people to adopt, right, as opposed to managing your own program. And that's where we start getting to these questions about rigorous evidence of a program impact that I'm going to spend most of my time on.

As Phelan was talking already spoke to this issue about big pressure on trying to get more improved government performance with more information about what works, right? Also, as they've started reviewing the evaluation evidence for some of the federal programs, get very disappointed at the quality of the vast majority of the program evaluations that they've seen, and so there's been a push for more rigorous evaluation, and, particularly, there's been a push for more experimental methods to improve the estimates of program impact.

And where this comes from — and I'm assuming everybody has a certain amount of research background, but not a lot, OK, so the basic assumption, why would you use random assignment. When we're trying to assess whether any of our social programs work, we often use a method that actually started back in agricultural research and is adopted in medicine and is adopted to some extent but not as much in the social services, is we're trying to control. We know that the outcomes we're looking for are influenced by a wide variety of other factors outside the program, OK. All the really interesting stuff that happens is not completely controlled by our programs. Get past the IRS Debt Collection.

And so the idea is that we're going to look at the experience of people who are in the program or are receiving the treatment and those who did not receive the treatment, look at their results at the end and see if they're different.

Now, if we randomly assign them to those comparison groups, there shouldn't be anything significantly different between those groups except their exposure to this program. So then your deduction at the end is, “Well, if we see a difference in their outcomes at the end of our study, it must be the result of the program.” OK? That's the logic we're working with.

Now, in pushes to try to focus federal funds on effective programs, the Coalition for Evidence Based Policy, which is a non profit group, had suggested a — what they call the “Top Tier Initiative” to review evaluation evidence, to identify those interventions that were really, really strong, the ones that had been tested with experimental designs and found to have strong outcomes.

Senate Homeland Security and Government Affairs Committee then requested GAO, us, to examine this Top Tier Evidence Initiative, how is that working, and, specifically, they said, “Well, what about this criteria of only those being found effective in randomized controlled experiments? Is that too narrow? What kinds of interventions would be best suited for that kind of criterion, and what types of interventions would not actually be — would this be a good criteria for?” OK? And then, effectively, what else could we — what else — what other methods could we use? Right? The latter is what I'm going to talk about today.

We went about this essentially by reviewing the evaluation methods literature. We've got decades of people talking about this problem, and summarized it in terms of, however — what kinds of programs or interventions. “Interventions” is a tighter name about — it's like what specific activities. And then did a summary and then worked with evaluation experts, senior folks, to review what we come to summarize and gives a sense of are we capturing sort of the consensus view.

The first thing that you come up with, the first point all of them made is that you wouldn't be looking at an effectiveness evaluation at all unless the intervention is important, and you actually want to know whether it's worth somebody else adopting it. Right? It's got to be clearly defined, so we know what it is and what it isn't, and well implemented, so we don't make the mistake of saying, “Oh. Well, we tested it,” and then find out that actually it was never put in place in the first place. They just said they had.

And second — thirdly, the study needs to be adequately resourced because some of the problem with the evaluation literature is that we have not spent enough resources to make sure that we got credible evidence, that we got, let's say, good survey responses, and so there were lots of questions left at the end of the study. This is a waste of resources. We don't have that many program evaluation resources, frankly, in the federal government to be spending it on studies that are not sufficiently well done to provide a conclusive answer.

The next point is it actually has to be a program where we need to worry about this, this external factors business, and, like I said, there are some programs like IRS Debt Collection. It's really pretty simple. You know, we want them to collect the right monies from the right people and do it in a reasonable period of time. Those are not the programs we're talking about. We're talking about the ones where we're trying to influence human behavior usually, and there's a lot of other factors going on besides our program.

Now, so now we've passed those hurdles, random assignment is absolutely considered very strong design when it's possible, practical and ethical.

First issue is you actually — the evaluator needs to control exposure to the intervention. We can't have people volunteering because we don't know if there is something different between those who volunteer and those who don't. The evaluator, in order to do this random assignment, we have to control who is exposed to the program.

Second, there's got to be limited coverage of the population, so we have somebody left over who's not receiving the intervention, who's not in the program. This sounds so obvious. Right?

The comparison groups have to be kept separate and distinct throughout the study. This is important, particularly in social interventions, because we're often working with knowledge, attitudes that are affecting behavior.

Sometimes the people who have learned that they're in the special new program behave differently because they perceive themselves differently and they perceive what they're actually experiencing a little differently. You want to keep out these other reasons why we might see differences at the end of the study. Right? So we usually want to keep them apart, and we want to make sure that the comparison group is not going out and getting the same services from somebody else out in the community.

And outcomes have to be observed within a reasonable time frame. This is the practical issue. Right? If you have to wait a long time to see these outcomes occur later, several years later in people's lives, you're going to have to follow them all that time. Right? So you are talking expense.

Thinking about the programs, you just flip that on the other side, you can start to see some of the issues. Entitlement programs, these are pensions, veterans' benefits and the like. Every single person who is eligible for those benefits must receive them. We cannot make decisions about withholding those.

Similarly, laws that prescribe particular activities occur or not occur, we cannot selectively apply those laws and randomly assign some people to be exposed to them and other people not to be.

You're thinking of state laws, yes, but I can't randomly assign which legislator will pass which version of those laws. Right? We can make comparisons but not random assignment.

Broadcast media. Radio, TV, Internet. Right? We don't control who's exposed. You, the participants, turn on the radio, turn it off when you hear that stupid message you get. Right? You skip over the ads and the other thing. You have control over whether you're exposed to those messages. Right? The evaluator isn't.

Comprehensive social reforms much more occurred in the last 10, 15 years. This is an effort to say what we want to do is have a variety of multiple interventions and activities going on in a community to try to change some of the relationships, perhaps between institutions or between people in the neighborhood and their institutions, and what you're trying to do is initiate a variety of organic activity. This is very hard to control, identify and randomize, therefore, so it's not a good choice for the randomization.

Negative events. And this is, you know, beyond we can't control hurricanes. We also are usually not allowed to expose people to a real substantive risk of harm just to find out whether our preventive or remedial programs are effective. We have to wait for those, those negative events to occur.

Random assignment is also not practical in a variety of settings. In the welfare area, where I've spent a lot of my time, you'll find staff — and this also happens in education where you're talking about needy populations that needs particular services. Staff will be very unwilling to deny services to people who they believe deserve them. So what you have to do is try to play with maybe alternative services or things like that. You have to work hard in some of those situations. It can be done, but it's going to be difficult.

Rare events and long time lags. Rare events is the same kind of problem, got to have large sample sizes in order to get statistically significant differences. It is possible. That's not the problem. The problem is expensive. So, if we've already decided this is an important question to understand, that we really get a good estimate of the effectiveness of these interventions, then, by all means, spend the money, but don't do it hoping that, you know, you can do it on the cheap.

And, finally, the broad, flexible programs. For us at the federal level, this is a big issue with a lot of the, what we call, “formula” or “block” grants, where we actually encourage localities to all be using different services, treat different populations. It makes it very difficult now to try to arrange a randomized trial experiment.

So, but there are alternatives. We are not painted into a box saying that we can't do anything else. We can, and there are a variety of rigorous methods that are available. I am going to do a very quick cartoon picture of the big — of the standard ones. By all means, read our report, and go through any major text and look for help to get more details.

The first is obvious, is you having a non randomized comparison group; that is, you often have a new intervention that is being tested out someplace, and you're selecting people from the rest of the population who look as similar as possible to these folks in order to try to deal with that problem, is there anything substantially different between them to begin with. You have to be very careful and check all the baseline characteristics beforehand and the like.

Regression discontinuity analysis is not as common. It's a very powerful design. Here, we actually are assigning people into two groups, not randomly though, but deliberately. We use a pretest, a baseline measure that is quantitative, and say everybody below a certain score will be in the program, receiving services, and everybody above it will not. This gets at that “I don't want to withhold services from my needy population” problem.

Then what you do is you just analyze the data on the people right around the cutoff because, frankly, they're pretty similar. They're very similar compared to everybody who is brought in. Now, the catch is that this is very expensive. You're going to throw out the data on the rest of the group. So — but it is very powerful.

Alternatively, you have statistical analysis as observational data. We don't have control over who's exposed at all. So what we do is we measure their exposure to the program. When you think of dosage in medicine, it's like, you know, how much, how much violence on TV do you watch, those kinds of studies. Right?

And then we also measure the outcome variable on those people, and we correlate to see if there's a relationship between the extent and whether or not there's exposure to the program, and then whether that's related to the outcomes. You're going to be using other statistical analysis to make sure there aren't any other differences associated with that exposure choice, right, to see if that might not be explaining. You got more statistical analysis to do than in the randomized trials.

Interrupted time series analysis is like performance monitoring. You're collecting data and outcome for a period of time before a policy change which occurs to everybody, OK, like those law changes, a big moment in time, and then watch the data change afterwards or not. Right?

OK. Long series, you're usually using administrative data. You're probably doing some other statistical analyses to deal with the waviness. Maybe there's time cycles, you know, that deal with it in your outcome variable.

And, finally, for the really tough one on the comprehensive reforms where everything is going on and you really don't have control over it, the best people could come up with, because I really can't find a good comparison site, is to use an in-depth case study.

And beforehand — that's critical. Beforehand, predict what changes you expect to see, both in the operations of the programs, the way they relate to individuals, what kinds of outcome behavior changes you want to see. List all that out beforehand, where, who, et cetera, and then, as that program starts developing, map it, measure all of those process and outcomes and see if they match what your hypothesis was.

Now, there are, of course, other rigorous methods but, basically, what we are talking about, using those main designs, you can do a lot more to — what we're after is trying to isolate the impact of that particular program effect.

Basically, you're collecting more data, OK, more data, assembling more evidence to rule out all those alternative hypotheses that could be explaining a difference or a change in outcome behaviors, OK, so both baseline data, targeting comparisons. There's often — what about all the dropouts? The people who dropped out of the program early, what were their results afterwards?

What about — it's even better if it's because of a program administrative problem that one office shut down and didn't enroll as well. What happened to that office's participants or clients?

Basically, you want to gather a diverse body of evidence. What we do not want to encourage is relying on a single study. Right? This is a problem. A single study is a moment in time, a particular set of people in a particular moment, in a particular place. It's only a sample of all the occurrences that you might expect under this program, and you're worrying about trying to encourage everybody to do it this way. You want more data. So you want to see it in different settings, with different populations, right, and you might actually learn. It may not — it may be robust, but it may be robust for certain kinds of people. Right? It's important to know that.

Our basic observations from the study. As you may have guessed by now, we definitely believe that simply requiring evidence from randomized controlled trials alone would not be a good idea if you really want to identify the full set of effective practices. You're going to miss some.

Secondly, frankly, I think most people in the field are not thinking only of effectiveness when they decide what kinds of policies they're going to adopt. They want to know what the cost is. They want to know what other kinds of resources they need to have in order to implement this. They're going to want to know whether this type of program is going to be accepted by their community. That's a big one. That's a big one. It's not effectiveness alone.

And then, finally, from GAO, it's like, ah, for those of us, we review tons of evaluations. We would know, generally, GAO, government-wide, even nationwide. We would know a lot more if we had better designed and implemented evaluations, if we had better reporting of what people did in the study, and also, particularly, what did the comparison group experience. That often is not described very well, and more evaluations that actually directly compare alternatives that you're considering. Right? You're not considering, you know, a new program of services versus no services. You're saying I would like to know should I do it this way or that way. So you'd like to see evaluations that directly compare them.

To sum up, this first one is the title of the report with a report number, and I was thinking I was going to make handouts and I didn't have time. Sorry. But, essentially, if you go to the GAO website and you look for Reports and Publications, they'll have a little space, and you can just type in the number, GAO 1030, and it'll pop up for you in PDF form.

I want to make a pitch for a couple other documents, and one is — I am on a task force with AEA, American Evaluation Association, and we had last spring written up a little white paper that describes the kinds of good practices, key elements for federal agencies to have in their evaluation capacity in order to actually integrate that into government management and improve programs, and so I would like to sell that here. And so, if you go to the eval.org — the economists beat us to AEA — and look up the road map for the Evaluation Policy Task Force, you will find it.

Also, I'm involved with a informal network of federal evaluators, folks from all over the federal government, and on our website, fedeval.net, we actually have lists of resources, program evaluations, agency websites, GAO reports, oh, books that we really enjoy and think are really good texts and sources and the like. So you can look there. It's not updated very often, but, you know, some of the stuff is classic. And here's how you can find me, at [email protected].

Thanks.

[Applause.]

Ed McGarrell: Well, good morning. I really enjoyed Stephanie's presentation, was sitting there thinking we're going use the videotape of that in our evaluation courses at Michigan State.

You also might have seen me chuckling a few times because, in many of those challenges that were described, it felt like it was describing kind of my life. And, at one point, I looked at my NIJ monitor for a current project that involves the evaluation of a comprehensive anti-gang initiative across the U.S., when Stephanie made the comment about the importance of having timely evaluations in kind of the reality of the world, when you're looking at multiple sites, we have 10 sites that are implementing three different program components, all at different times and different places and creating real challenges. So I just recently had asked Louis, we need another extension on this evaluation. So I don't know how timely we'll be.

We also have a natural experiment this morning. I noticed that Stephanie had nine PowerPoint slides, following the dictates of good PowerPoint presentations. I will violate that principle. I have a ton of slides. We'll be moving quickly and maybe an inch deep on a lot of these topics, but, hopefully — and I think that the two presentations complement each other quite well.

You know, I think there's plenty of evidence that we haven't always used the goal of evidence-based practice to guide our research. And many of you are familiar with the University of Maryland study about 13 years ago and reviewed large numbers of evaluations in criminal justice research and different types of research, and they concluded that only about 13 percent of those studies met the standard of methodological rigor.

David Weisburd has a very recent article in Criminology and Public Policy, again, reviewed a large number of policing evaluations, specifically looking at problem solving, and you can see — now, David's pretty strict on his standards of what meets methodological rigor, but if only 10 out of 5,500 are meeting that standard, we've got a ways to go.

And what you typically see — and this is — you know, I think Stephanie described this — is the kind of usual work involves looking at a single site, doing a simple pre- and post-assessment with no comparisons, and that does open us up to all this variety of rival explanations.

For the baseball fans out there that have followed the All-Star game over about the last 40 years, if you do a simple correlation of the tendency of the National League to win those games versus the tendency of the American League, you will find a strong correlation with violent crime trends in the United States.

[Laughter.]

McGarrell: As the American League has come to re assert dominance, violent crime has dropped dramatically. Now, I don't believe that that's the likely cause of that decline, but that's the problem with those kinds of evaluations.

On the other hand, I think, as Phelan said in his introductory comments, there's a lot of evidence that we are entering an era where there is a serious commitment towards building this kind of evidence-based practice. And you hear that in the leadership of OJP and Laurie Robinson, Mary Lou Leary and others who've been pushing this for quite some time. You also see it within the field of criminology and a lot of different indicators.

Essentially, what we're trying to do is to build this evidence-based practice, and I would argue through three ways. One, as Stephanie laid out very nicely, is to build stronger research designs and to move from these simple designs to controlled comparisons, quasi-experiments and experiments, also to accumulating evidence through a number of studies and building that body of evidence through systematic reviews and meta-analyses and similar approaches. And, thirdly, I think, doing more to try to connect theory to our evaluation. If we can identify the theoretical components behind why we think an intervention should have an effect, if we see an effect or we don't see an effect, but we can link that back to should the intervention have had an effect according to the theory, then we will have stronger conclusions as we look at that evidence.

So what I want to do is to draw on about 15 years of evidence in these three areas. I do this at the risk of being very egocentric and that these are areas that I've been working in, but I think this is kind of a glass half full/glass half empty presentation. You know, you look at the Maryland study and the Weisburd study, and you might feel a little bit pessimistic in our ability to put together this evidence-based — but, from another perspective, if you compare where we are today in 2010 to where we were in the early to mid 1990s, I think we've come a significant way in building evidence. And I'll use these as examples. Other researchers could look at correctional research or some drug intervention research and make similar comments, but these just happen to be the areas that I think I know a little bit about.

So let's talk about addressing non violence. In the early 1990s, I was working in Spokane, Washington, with their police department, with the chiefs and sheriffs in the state of Washington, and doing a lot of interesting, what I thought was interesting work, in community policing and problem solving, and we were seeing some impact in terms of addressing neighborhood level disorder and the relationship between police and citizens, but I remember being asked by the chief of Spokane, “What do we do about this violence problem?”

About that time, I moved back to the state of Indiana, and I began working in partnership with the Indianapolis Police Department. And the mayor at that time was a former prosecutor, and Indianapolis was experiencing big increases in homicide and gun violence. And I remember meeting with him saying, “What does the research say in terms of effective interventions to deal with gun violence and homicide?” And I sat there in silence. There wasn't anything I could point to, and I think that was a pretty fair reading of the evidence at the time.

Since that time, there's been, again, a lot of evidence emerging that there are things that we can do. So I want to work through the development of this knowledge base because I think it builds on Stephanie's points in terms of how we've moved from weak designs but pointing us in a suggestive way towards promising interventions to continually building stronger designs, and I guess that would be the theme that I would hope to leave us with today.

So about this time that the mayor of Indianapolis was posing this question — and I was trying to remember this this morning. It was either an NIJ conference or it was an ASC conference, and Larry Sherman did a presentation, hadn't been published yet, on research he was doing in Kansas City, in which they had done a quasi-experiment. And what they looked at was, if we could use directed police patrol in gun crime hot spots and tell those police officers to focus on illegally possessed firearms, could we have an impact on gun crime? And, basically, the findings that came out of that was, as the seizures of illegally possessed firearms increased by 70 percent, those areas experienced a 49 percent decrease in firearms crimes, so pretty promising evidence here but again a single site, one point in time.

Indianapolis decided to see if they could implement a similar kind of approach. They had two target areas and two comparison areas. The target areas, you'll see labeled and as the north and east districts. They were slightly different strategies, a pretty significant drop in gun violence in that north area, a decline in the east area as well, and they had the comparison, so, in a quasi-experimental kind of approach, had a comparison area that actually had witnessed an increase. And then we also looked at what was happening at the city as a whole, and so the data seemed to suggest that in these two targeted areas using a strategy of directed police patrol looking for illegally possessed guns, that you could have an impact consistent with Kansas City.

We did time series analysis, which ruled out some rival hypotheses but, basically, came away with this conclusion that when you focused on these gun crime hot spots and those illegally possessed firearms, it appeared that you could have a significant impact.

Then there was a third study in the series that came out of Pittsburgh, a very similar kind of approach. They added one additional piece to this research in that they, in addition to using police statistics, they also went to the trauma center and looked at gunshot injuries, and those gunshot injuries declined very consistently with the police data, so, again, suggesting that this kind of approach was having an impact.

When the Pittsburgh findings were presented, Sherman wrote a commentary on that, and he brought several additional studies. So, at that point, you ended up with eight tests of this intervention, all of which indicated a decline in gun crime, and suggesting — you know, you could question any one of these studies, but when you looked collectively, it did suggest a promising approach to addressing gun crime.

About the same time, in mid 1990s, many of you are probably familiar with the Boston gun project. Boston Ceasefire used what some have referred to as a “pulling levers” approach to trying to address youth homicide. One of the things that I think is interesting about the Boston study is it followed this progression of trying to increasingly build the rigor of these evaluations. So the first study that came out of this was a simple pre/post comparison, and after implementing this intervention, they had a 65 percent reduction in youth homicide, pretty impressive. They later tried to strengthen that evidence by comparing what had happened in Boston to a large sample of other cities. Boston was the only place that had experienced this kind of decline, so, again, adding to that evidence base.

Indianapolis became one of the first places to try to replicate that. I put “replication” in quotation marks. The natural scientists in the room would probably throw something at me and say, “That's not really replication,” because it's almost impossible in criminological research where the context is always somewhat different to have a true replication but a very similar kind of approach, very similar kind of findings.

And in our comparison, because this was a citywide kind of intervention, was to look at what happened in other similar midwestern cities, and Indianapolis was the only one to experience that kind of decline, very similar to the Boston findings.

Los Angeles saw a similar reduction or a somewhat more modest reduction, but they did — and, again, consistent with what Stephanie was saying — look at how the program was implemented as well but all three studies coming along and suggesting that this might have an impact.

About the same time, a few years later, some of you may be familiar with the Project Exile in Richmond. Here, the U.S. Attorney said, “I don't know how to address these problems of serious homicide and gun violence, but the one thing I can do as the U.S. Attorney is to prosecute people who are illegally possessing and using firearms to try to incapacitate the highest risk individuals who are shooting people on the streets and then try to deter others from illegally carrying those guns,” primarily focused on felons in possession of a firearm.

The first evidence that came out of Richmond was, again, this kind of simple pre/post comparison where homicides went down after this was implemented. The first, more formal evaluation raised some questions because it appeared that, well, the homicides are going down nationally at this time, so how do we know that Richmond isn't just following the national pattern.

Rick Rosenfeld later did a further evaluation looking at two years follow up and did this with a comparison of all other U.S. cities and found that, well, indeed, when you look at this, it appears that the decline in Richmond may have been due to this.

So you move from — if you take my word that in 1993, '94, we had very little evidence, by about 2000, there were these series of studies that had come out that at least pointed to some promising approaches to addressing homicide and gun violence. That provided the foundation for Project Safe Neighborhoods, a major Department of Justice initiative, in which every U.S. Attorney's office was to create a PSN task force and to do something, kind of building on these Boston Ceasefire, the Indianapolis as part of something called Strategic Approaches to Community Safety, or SACSI, and Project Exile.

There wasn't one single model that was followed or urged for adoption, but the model of the program was to use research and analysis to understand the local gun crime problems, so that you could then focus on those contexts that seemed to be driving gun crime, and then putting limited local and federal enforcement prosecution resources to focus specifically on those local hot spots. If there was a theoretical model, I guess I would argue it was to try to increase the credibility of a deterrent threat for illegal use of a gun in crime.

Now, the PSN evaluation has all the problems Stephanie talked about. It's a national program. So what do you compare it to? And we started off, I think, consistent with what Stephanie would have recommended. It was we'll do some case studies and so look at some of the places that have implemented the program. It appeared that two different models — one based on that Richmond approach and then the other that followed more like the Boston Ceasefire approach. In all 10 of these sites, we saw a decline in gun crime. In 2 of the 10, there were questions that you could raise in terms of the evaluation of that, but the evidence was suggestive.

One of the interesting things is, unlike a lot of prior research, the most rigorous design in Chicago also produced some of the strongest evidence. So you couldn't generalize from these case studies about PSN across the whole country, but at least it was suggestive that you might be having an impact on gun crime. So the next step in this was to try to distinguish by levels of dosage.

So the comparison becomes low-dosage, non-target cities with high dosage — or target cities and high-dosage locales. Again, I realize I'm moving through this pretty quickly, but happy to — actually, there's a presentation on Wednesday where we'll talk about this again.

But, to get to kind of the bottom line here, one of the ways of thinking about the dosage was to look at the level of federal prosecution, and so, in the cities that experienced high levels of federal prosecution of gun crime, you saw a very significant decline in violent crime. Percentage wise, it looks like this, and our kind of contrast was between these target cities, and high dosage sites had a 13 percent reduction. In the low-dosage, non-target cities in low dosage where, in effect, I would argue, PSN really never occurred, you had an 8 percent increase in violent crime. All this was presented at last year's NIJ conference, so let me tell you something new.

We've recently published, looking, controlling for a lot of the other factors that we think affect city levels of violent crime and find that these patterns are consistent, and perhaps most important is we now have some firearm homicide data, and the patterns are very similar to what you saw in violent crime. So, again, the target cities and the high-dosage environment, about a 10 percent decline in gun homicides compared to these low-dosage sites or these non-target cities that actually experienced an 11 or 14 percent increase. That 25 percent gap, I think, is pretty telling or suggestive that this approach can have an impact.

Now, a whole variety of limitations, I won't go into, since that's not nearly as much fun to talk about, but all the things Stephanie talked about, you could raise questions about this national evaluation.

Chicago Ceasefire, another promising approach, a public health approach where they used street workers to try to intervene in violent crime context. Again, a number of you may have heard about this. Wes Skogan and his colleagues have recently published an evaluation of the Chicago Ceasefire. It's complicated because, during this period, Chicago was experiencing a decline in violent crime, so teasing out whether it's Ceasefire or something else is difficult. But what they concluded, they looked at seven hot spot areas. They saw declines in six of those — where Ceasefire was occurring, the violent crime was down to six of those areas, and it appeared that at least in four of those six areas, that Ceasefire may have been associated with that decline.

To complicate the picture, we've recently completed an assessment of a similar program that was based on the Chicago model in Pittsburgh. The people that ran what's called “One Vision One Life” in Pittsburgh were actually trained in the Chicago model. We had three target areas, which are compared to both matched comparisons as well statistically constructed comparisons, and not only did we not see any impact but actually saw an increase in aggravated assaults and gun assaults, leaving us somewhat perplexed on what this all means.

It may be something about whether the Pittsburgh program was implemented with the degree of fidelity to the logic model and the appropriate levels of dosage. It may be something that has to do with the different gang structure in the two cities. It may be the Chicago results were due to that overall violence, or maybe it's something else. But I use this example because I think it, again, reinforces the point Stephanie made of the need to move from one place in one point in time, to looking at multiple interventions and try to — I think the next few years, we'll be trying to sort this out in terms of what do these Pittsburgh findings mean.

So now, 15 years, going from — I was trying to think about doing some kind of a pre and post slide here where if you asked me in 1994, everything would have been blank, but now I think we can point to a number of at least promising, if not evidence-based practices to inform how do we respond to gun crime.

I am going to make Phelan really nervous here because I got one minute to go through the rest of my slides, but let me just — so the point here would be, I think, clearly we've made a number of advances in these efforts. I want to point, go real quickly and could handle this in questions, but that Boston model has been applied to deal with drug markets. At this point, kind of a simple pre and post evaluation has been done.

There are a number of sites that are now using this kind of drug market intervention approach. There's some data that comes out of Rockford and Nashville that we've done that suggests, yeah, maybe this model is having an impact, but the point I wanted to make is I think this is really how we hope this evidence-based model will move forward, and that the high point gave us this promising evidence. A number of other sites have implemented this and seem to be having an impact, and now we see NIJ and BJA working together.

There's an RFP that may just close or is about to close — it is closed? OK. To do a very rigorous evaluation. So BJA is going to provide training to a group of sites in this model who will participate in this evaluation and then, using the most rigorous evaluation techniques, will apply them, and so we'll move — I think at the end of this process, we will know does this promising evidence hold out when you move to a more rigorous point.

So let me just conclude. I'll skip over restorative justice stuff. I knew I wouldn't get that in. But I think there are several points that I would hope to leave you with. One is to think about this as a building block kind of model, oftentimes moving from grounded experience, police-researcher or practitioner-researcher collaborations that look at promising practices, moving towards quasi-experiments and then, where possible, moving to the randomized controlled experiment.

I don't think it is an either/or choice, but we need to think about the strongest design that's feasible, given where we are in the state of knowledge at the time, given resource constraints and the nature of the intervention.

I would argue, again, linking theory to practice, and to conclude, I think one of the important points for those criminal justice executives and policymakers in the audience, I think one of the things that can really benefit the field is for you to insist on high-quality evaluations in that Anthony Braga has a commentary on Weisburd, in the same issue that Weisburd's article is in, and he talks about his experience of working with Chief Ed Davis, first in Lowell, now in Boston. This was my experience in Indianapolis. When the chief, when the mayor, when the juvenile court judge, when a prosecutor say, “I'll try this, but I want to know whether it really has an impact,” that puts us in a position to do these kind of challenging but very important evaluations that can give us that evidence base.

Thank you.

[Applause.]

Wyrick: All right. Thank you very much, Ed and Stephanie.

I want to reiterate where we started a little bit on this, and the work out of GAO that Stephanie talked about came to them from Congress. They were being approached by folks who were saying, “You know what, you really need to be committed to this top tier, this randomized controlled trials,” and I'm sure they had a very in-depth conversation with those staffers on the Hill and so on.

But, unfortunately, a lot of folks in those kind of positions, non-specialists, policymakers, decision-makers, they only bring away certain take away messages from conversations like that, and that take away message might just be randomized controlled trial good, everything else bad.

So some of those messages that folks may walk away with can be really detrimental for us because, as we just saw, as Ed, I think, very well demonstrated, that there's a progression, what he called a “building block model,” towards building evidence and towards even getting to the point where it would make any sense to try to attempt an RCT or some of these most rigorous designs.

So I want to talk to you a little about an initiative that we've launched out of the Office of Justice Programs. A little over a year ago, our new Assistant Attorney General, Laurie Robinson, came into her position, and she had 10 major goals for the Office of Justice Programs. And those goals, two of them, included a real emphasis on — one of them was a real emphasis on science, and the focus on science has been articulated from the President, through the Attorney General on down to the Office of Justice Programs.

And one of those goals was really to focus on data-driven strategies and evidence-based practices, and we took a look inside our organization to really figure how should we approach this, and through a fairly extensive process of talking with folks from across our organization, we developed what we call our “Evidence Integration Initiative,” and I've got the three primary goals of that initiative up on the board.

Essentially, we're looking to improve the quality and quantity of evidence that we generate, looking to improve the management of knowledge and the integration of evidence, to inform program and policy decisions and really looking to improve the translation of evidence into practice.

Now, there's a number of goals under these — I'm sorry — a number of objectives under these goals and a variety of activities that we're going forward with, but I'm going to focus on a couple of pieces of this that speak to this question of generating evidence and also the goal two here, the managing and synthesizing and integrating evidence and knowledge, because what we found is that, you know, at the Office of Justice Programs, we're not the office of replicating evidence-based programs. OK? So we don't just fund a list of programs, go replicate these things. We fund a lot of innovation, and we put out a lot of flexible funding streams that allows the state and local professionals to really determine how best to address their needs.

Criminal and juvenile justice are fields where there's a lot of local innovation. In fact, that's where most of our innovation, and it's not — much of this innovation doesn't come about through people who consider themselves program developers necessarily. They're simply people trying to solve problems at the local level, and they're professionals who have a lot of training and experience, but it may not be in research.

What we need to recognize is that our approach to addressing evidence should be cognizant of the fact that what we're trying to do is inform decision-making and get folks who are in those decision-making positions to integrate evidence into their process, and so, therefore, it has to be helpful for them. It can't paint them into a box, and it can't make them feel like their options are extremely limited.

So I want to talk a little bit, to start off with, some of our evidence generation piece and say right out of the gate that we're very committed to increasing our commitment to randomized controlled trials and randomized designs. For the reasons that we've already discussed, that I think Stephanie laid out very well, these are very strong designs. They have a long-lasting impact, and where you can use them, we should be.

So one of the things that we've done for years now is we've put priority in National Institute of Justice solicitations. We put priority on randomized evaluation designs, where possible.

So we'll have a solicitation. We'll say, you know, priority will be given to randomized applications. My own candid assessment on that is that it hasn't yielded a high number of randomized controlled trials that actually get to the funding point, point of funding, and that's because it's very challenging for a researcher out there to unilaterally put a randomized controlled trial into place in an applied setting. And so it's very expensive, it's very difficult, and our timelines, even if we give you 60 days to put that application together, that's extremely difficult to do. So we do get a number of applications. A lot of them don't make it through the very competitive review process.

What we're trying to do to up the number of randomized controlled trials that we actually field is, I think, demonstrated in this one solicitation that I've listed here, the evaluation of the multi site demonstration field experiment, what works in re entry. This closes tomorrow. So, if you haven't seen it, you know, you got about 24 hours to put your —

[Laughter.]

Wyrick: No. It does close tomorrow, but this solicitation, even if you're not applying for it, you might want to take a quick looking at it. It represents a deliberate effort on our part to work across NIJ and BJA to structure a field experiment out of the gate. So it allows us to address some of those challenges that we heard about in terms of being able to put a program together with distinct exposure to avoid contamination, to foster and nurture that randomization process that's so hard to really keep true in the field.

So we have a great deal of commitment to this, and our goal is to really put a series of randomized controlled trials in place, so that at any given year, we've got a number of them at different stages of development coming out of National Institute of Justice.

But I want to take sort of a big step back for a second because, when we went around our organization and started talking to people about evidence-based practice and evidence-based programs, we started hearing different things from folks in terms of what their priorities were, and it sort of informed our goals.

Well, one priority is we need to do better research; we need to do more research. Another priority is we need to really get that information into the hands of the practitioners in a way they can use them, but then there was this other priority about synthesizing and integrating information and evidence that was very important to people.

We do a lot of work that's cross cutting for our organizations. So we've got a juvenile justice office, we've got a — BJA is sort of our criminal justice office. We've got a victims' office, and we've got a — you know, NIJ, the research and evaluation arm and BJS, the statistics arm, and, in that sense, it makes us very well situated for addressing these questions of evidence, but, frankly, we have cross cutting issues where we're not all working off the same base of knowledge. The same thing's happening out in the field.

So we needed to step back and look at some basic definitions, and now we are — Department of Justice, I mean — at any time, we can go ahead and call up the National Academy of Sciences and get some of the best minds from across the country or even internationally to noodle on these questions.

We didn't do that. Instead, what we really tried to do is say, well, let's come up with some working definitions that really seem to make sense and could relate to our audience, our constituents. So we came up with one for evidence. Evidence is information about an object or question that's generated through systematic data collection, research or program evaluation using accepted scientific methods that are documented and replicable. It's still a little wordy, but the key pieces in there are systematic. It's documented. It's replicable. It's accepted scientific methods. OK?

So that doesn't say anything about whether the evidence is actually accurate. Right? We're taking our cues a little bit from the way evidence is used in the rest of the field of justice. It's not a foreign term to us. Right? We've got prosecutors, attorneys, investigators all over, work with evidence all the time. Some of it's good; some of it's bad. Evidence has different levels of reliability.

DNA evidence might be very strong evidence, but we don't say, “Well, you know, we're only going to look at DNA evidence.” We're not going to throw eyewitness evidence out the door completely just because we know there's biases and errors. OK? So we use evidence of different qualities to make life or death decisions every day all across our country, and we look at it on a case-by-case basis.

Similarly, that's how we feel evidence should be treated in this domain, but we break it out a little bit. So causal evidence is really what we've been talking about when we're talking about effectiveness, getting to effectiveness. Causal evidence provides information about the relationship between activities and interventions, activities or interventions and intended outcomes. So this is basically program evaluation, cause and effect, X leads to Y.

We distinguish that from what we call “descriptive evidence.” Maybe that's not the best name, but we're basically saying there's a whole other world of information out there that's collected through valid statistical or scientific methods and of varying qualities, but they're characterized individuals, groups, events, or processes. They may use quantitative or they use qualitative research methods.

Now I'll tell you a little of my background on this. For nine years, I was the gang program coordinator for the Office of Juvenile Justice and Delinquency Prevention. Now, if you really want to understand some things about gangs, you'd probably do well by looking at some of that qualitative research. There's been some great qualitative research on gangs that tells us about group processes and how the internal dynamics of those gangs play out. It doesn't tell us whether this program will or will not be effective, but it tells us some important information. But how do we incorporate that into our conversations about evidence?

So we've got another term that we're working with that we talk about “actionable evidence.” At what point do you have actionable evidence? Now, this is a very basic definition. It's of sufficient quantity or quality to influence the decision-making regarding a practice or policy.

But now we're talking about something different, and this comes back to me in terms of the types of issues that we face at the Office of Justice Programs, that people around the country are facing. Every once in a while, actually fairly regularly, we have people come to us from a city somewhere in the country or a county and they say — and it's the mayor and it's chief of police. They're coming to Washington to meet with a number of people, but they want to meet with us and they say, “We've got a bad problem, and we need to figure out some ways to address it.”

And the problems that they come up with are things that are usually pretty general, and they're going to get into a problem-solving kind of mode. And it's not going to be — it's rarely going to be something along the lines of what I'm looking for is a list of programs that I can pick from and just simply replicate. That's going to be a piece of it. That may be a piece of it, but they're usually thinking about multiple strategies. They might think about things that what are we going to do in the law enforcement, what are we going to do at the schools, because we're dealing with youth violence and that's a tricky issue. We've got truants, we've got kids that we're worried about who are deep into this, and we've got kids who we're worried about that are on the way, and we know there's stuff going on in the family. What do we have to say to these people?

And if we're just looking for randomized controlled trial evidence, then the answer is we're probably not going to have a whole lot to say to them. But there's so much more that we have, and in order for this field, I think, to really take steps forward in addition to focusing on those highest rigor studies, I think what we need to be focusing on as well is how can we use the wider range of evidence to help these folks.

So there are a number of factors that would probably be considered in terms of thinking about, well, whether is this evidence actionable. Obviously, the method design; what design do you have? What kind of findings are you getting from what kind of designs? The rigor of the particular study. Right? You can do a poor RCT. You can do a poorly executed randomized controlled trial, or you can do a well-executed one.

The point we heard that I think Ed McGarrell was just making, consistent results across multiple studies of different methods in different places by different investigators. So this question of multiple methods, we know that each method you're going to use has its own limitations inherent, but, as you get multiple methods of different kinds pointing in the same direction, you have a level of increased confidence. At what point does that confidence tip to the point where I'm willing to make a decision on that? Well, that might depend a lot on the importance and the size of the decision. Right?

Going outside of criminal and juvenile justice field, when we look at public health, we haven't seen — and I stole this line, but I think it's so good it's worth stealing. I'll credit David Altschuler, but we haven't seen a randomized controlled trial that says that smoking causes cancer or that it's bad for you. There are some things that we don't, for any number of reasons, conduct RCTs, but we know at this point that that's the result we're seeing. So there's a level; there's an accumulation of actionable evidence. On a smaller issue, you might be willing to make a move, a decision locally, based on a smaller piece of information.

So I want to give you a sense for how we can get to actionable evidence because it speaks a little bit to this question of, you know, we talk about how do we evaluate and how do we test the effectiveness of any particular programs, but I think what a lot of folks are also looking for is what should we develop in the first place. And whenever they were doing that deterrence work, whenever they were working up those different approaches to dealing with gun violence or any of these pieces, they were using information. They were using evidence at the outset to begin to develop those designs in the first place, and we've got to be in that game, and that game has to include — that game has to be recognized as part of the evidence-based work that we do.

So one of the things we're doing out of the Evidence Integration Initiative in our organization is we're testing out this idea of developing integration teams, evidence integration teams that are really going to draw together personnel and resources from across the Office of Justice Programs, to delve into the evidence and really synthesize and articulate, synthesize the evidence and try to articulate principles for practice and programs and policies.

This is not an enterprise that is entirely new. We've had any number of efforts to review the literature and make suggestions. One of the things we're doing here is also for the purpose of developing an understanding and an ability to use evidence within our own personnel and staff in the Office of Justice Programs because, if they're going to have any effect in being able to communicate the importance of that effort and the findings of that kind of effort to the constituencies that they serve, they need to understand it themselves a little bit.

So we're not just working with our researchers. We're not just working with our Ph.D.s, although we have a fair number of them. We have brought together program and policy people from across Office of Justice Programs to focus in on some target areas and dive into the research a little bit, and so they're going to look at some specific topics and start drawing together the evidence base.

I want to give you an example of this actionable evidence, what I think is actionable evidence based on a synthesis of existing — in this case, descriptive evidence. So, again, my background, I've done a lot of work on gangs, and this was something that we published out of the U.S. Attorney Bulletin in 2006. This is a graphic that was included in a chapter that I wrote, and, essentially, what this graphic does is convey an accumulation or a synthesis of descriptive evidence in a way that informs practice and policy. And I'll go through it pretty quickly here.

So the vertical axis is the share of illegal activity increasing as it goes up. The horizontal axis is relative share of the population, and what we're trying to talk about now is addressing gang problems within a community.

So we start from the understanding that we're talking about a community that has a significant amount of gang activity, and what we do is we present this general graphic to show that in those communities, you have a small portion of people at the top of this triangle, the group number one, small proportion of people who are involved in a lot of the serious violent offending. This comes from research that's been replicated in many cities, and the one that comes to my mind is out of Orange County, California. Eight percent of all of their offenders were responsible for over 50 percent of all their serious violent offenses. So that's one group that's out there in your community.

Below them, you've got another group that includes a larger number of active gang members and associates. Again, a whole body of research that gets into this question of once you enter a gang, your likelihood to offend, violent offending, property offending, weapons, drugs offending, all of that goes up. We know that from longitudinal research. We know that from qualitative research. We know it from survey and interview research with gang members themselves. And we know that they're not all up in that top group. So they're out there committing offenses.

Third group you've got is a high-risk group. Again, we get this from the longitudinal work. We get it from survey work, and it's the younger kids. And we know when they come to the attention of the system. They usually start coming to the attention of the system 8, 9, 10 years old. They start showing up with some delinquent acts, and they're high risk. We know what those risk factors are, and we know how risk factors work now. Again, all descriptive evidence, and it tells us something about this group of young people who are going to be the part from which the next cohort of gang members is most likely to come but not all of whom will go into the gang.

Finally, at the bottom of the triangle, you've got everyone else who lives in that community. It's the general population. They don't offend a whole lot, and they're a much larger group. So why is this actionable? Because all of those groups track or correlate directly with specific types of anti-gang strategies.

So, when I published this article in 2006, it was to the U.S. Attorney audience, and they had a vested interest in trying to understand gang prevention and how it relates to their overall efforts because they're a little more enforcement oriented.

We started getting invitations, “Come out and speak to us.” I've spoken all over the country essentially about this diagram, which is nothing more than a bunch of descriptive evidence synthesized into one sort of digestible graphic that puts together the relationship between different anti-gang strategies.

I've had people go beyond where I've said and say, “Well, really, this also speaks to how we should our investments as well.” So I don't think that's quite exactly correct, but I think there is something there because you got to put a lot of energy to that top group. But what it also says is how do you put together a multi-strategy, anti-gang program within your community in a way that's going to hit on all the people who are really involved.

So this is a way that I think we should also be looking at evidence, that I think most of the discussion really isn't going in this direction right now. We're going to try to see how far we can go with this because I think this also puts social science researchers in an important role within the field. The more we can start talking about actionable evidence, the work that we can do to help develop programs and help develop them through these building blocks, if you will, as Ed put it, this building block model of building evidence towards effectiveness.

So, with that, I'll leave it and we'll open up for questions. Thank you very much.

[Applause.]

Mark Nelson: Well, good afternoon. Welcome to the panel on "Making Sense of Backlogs." I'm Mark Nelson. I'm a DNA program manager at NIJ. I share those responsibilities with my partner, Min Nguyen. We have a distinguished panel here for you today. I'm gonna try to set the stage a little bit for why we're having this panel.

NIJ often receives requests from policymakers, press, media asking for what is the national backlog. And in the past, we've been relying on studies — national study has been done in 2003. The most recent national study was by BJS in 2005. That is now published. So in 2007, we initiated two surveys to take a look at this issue. I want to point out that both of these are still in peer review or going into peer review, and so all results that you hear of those two surveys will be preliminary in nature.

We want to address some of the issues with backlogs and dispel some of the myths. For example, I received a request from an individual who wanted to know why there was still a national backlog of DNA cases when we had funded a quarter of a million dollars to date at the President’s DNA Initiative, why were they still there. And I think folks have the misconception …

Can you — I guess I'm gonna have to lean forward. Is that better? OK.

There's a misconception of what a backlog really is, and I want to set the stage for you. If you have a laboratory that's done 50 percent more cases this year than it did last year, we'd all say that's really good. They've made real progress. They've increased their capacity. But if their demand has gone up by 50 percent, their backlog won't go away. It's only when the capacity increase is greater than the backlog that — or the demand that you’re going to have a decrease in the backlog.

So we're going to talk today. We're going to have three panelists. Our first is Kevin Strom. Kevin is a senior research scientist with the Crime, Violence, and Justice Program at RTI, and his interests are in law enforcement responses to the community violence, as well as forensic science effects on the criminal justice system. He's led numerous studies for the Department of Justice, including studies determining the nature of forensic case backlogs among the law enforcement agencies, forensic labs and medical examiners offices. Kevin will be speaking about the survey that RTI has done of law enforcement cases that have not been submitted to crime laboratories.

Our second panelist will be discussing the impact of increasing demands and a huge increase in his particular demand on his crime laboratory. Our second speaker will be Greg Matheson, who's the director of the Los Angeles Police Department Crime Lab. He's been with the lab as a criminalist supervisor and manager for 30 years. And I'm going to have to read this part: He's core qualified in toxicology, serology, crime scenes, explosives, flammable liquids and vehicle lamp filaments. Obviously, he's very well qualified. He served on the board of directors for the CAC, California Association of Crime Lab Directors, ASCLD, American Board of Criminalistics and is a member of the American Academy of Forensic Sciences.

Our third speaker will be speaking about case backlogs in crime laboratories, as well as an overview perspective. He's very well qualified to do that. It's Kevin Lothridge. He's the CEO of the NFSTC. And in 2007, NFSTC won a competitive process to become the NIJ Center for Forensic Excellence. Previous to being the CEO, Kevin has been a chemist, a chief chemist, lab director, and although he’s trained as a forensic chemist, he's acquired expertise in other areas, such as drug chemistry and fire debris analysis. And he's the past president of the American Society of Crime Lab Directors.

We're gonna hold all questions until the end of the panel, and I'll call on Kevin to start us off. Kevin Strom.

Kevin Strom: Thank you. Good afternoon. Today I'm going to be talking about a survey of law enforcement forensic evidence processing that RTI has conducted on behalf of NIJ. As Mark mentioned, these findings are preliminary. The final report is currently under peer review with NIJ.

And, just a quick overview of this. As many of you can imagine, this is not easy information to obtain. Law enforcement agencies — many don't, don't have record management systems that readily allow you to extract information on cases that contain forensic evidence but that were never went to the laboratory. And that was really our main goal here. So it was a challenging, challenging initiative, but I think we have some results that can, hopefully, move the field forward, especially as we learn about this issue.

Just a little bit about forensic backlogs in general. As many of you know, more evidence is being collected from a forensic perspective than can really be managed at forensic crime laboratories. And most of that information to this point has relied on the laboratory side of things to establish this backlog. In 2002, BJS conducted a census of crime laboratories and reported upwards of 260,000 backlog cases. An update in 2005 showed that that number in that time period had increased 24 percent, and those backlogs typically extend across all forms of evidence, including drug chemistry.

The sole initiative to look at sort of the other side of the fence of the backlog problem — and those are cases that really never left law enforcement but that contained some forensic evidence and could have potentially moved forward from an investigative perspective if they were analyzed — was conducted in 2002 in an NIJ national survey of state and local law enforcement agencies. And that survey estimated that there were 50,000, 52,000 unsolved homicides and 169,000 unsolved rapes that contained biological evidence that were not submitted for analysis to a crime laboratory. An additional 264,000 property cases were estimated to not have gone to the lab as well.

So in 2007, RTI was funded by NIJ to conduct the current survey. The current survey was different in that we were focused on all forms of forensic evidence, not just DNA. Our focus again, though, was on state and local law enforcement that investigated crimes as part of their regular processes. So we excluded, for example, sheriff's offices that only had jurisdiction over courts and jails. Primary objective was to estimate the number of unsolved violent homicide and rape cases and property cases that contained some form of evidence but that did not go to the crime laboratory, also the types of forensic evidence associated in these cases for violence, and the capabilities and procedures in law enforcement agencies for processing, submitting and retaining evidence.

It was a national survey, as I mentioned. We used a methodology similar to the LEMAS Survey, the BJS LEMAS Survey. Our final sample was over 3,000 state and local law enforcement agencies, and we used a multimode data collection approach, including Web mail, fax and telephone follow-up. The nature of the responses — the nature of the questions didn't really allow for conducting the survey on the phone very easily. So the phone was mainly used to follow up with agencies and discuss any problems they had with submitting the survey. Often the survey required coordination within the law enforcement agency from investigations, crime analysis, research and planning, and property and evidence.

We had a very distinguished panel of experts from forensic science, law enforcement, research, some of whom are in the room today, that assisted with the development of the survey instrument and even follow-up with specific agencies. And, of course, NIJ provided critical guidance and also helped obtaining letters of support, including a letter from the IACP.

In terms of response rates, overall, especially with a survey that was challenging for many law enforcement agencies, I think we did pretty well. We had a 73 percent response rate. Those responses were highest for large agencies, those with 100 or more sworn officers or those with 50 to 99 officers, and lowest for the very small agencies, which was not a surprise considering their often limited resources.

Responses were also highest for municipal police departments and lowest for state police agencies. In some cases state police agencies were challenged because of the decentralized nature of some of their units. We did get a fairly good response rate on the Web, which was promising, and by hard copy. Those were really the two primary methods used to respond. We also ran a help desk, so that allowed agencies to contact us if they had problems or to coordinate responses within the agency itself.

This is just a screen shot of the public version of the Web site which allowed for password control and the ability to track responses over time.

So in terms of the results, overall we … These are defined as cases, again, that were unsolved, so open cases that had not been closed by arrests or by exceptional means that contained some form of forensic evidence but that were never sent to the forensic lab for processing. And we asked about questions over the previous five years. The expert panel and others thought that asking within a longer period of time would be extremely challenging for law enforcement agencies to answer and to approximate.

So what did the results show? Overall, we estimated for homicides about nearly 4,000 homicides over this period that met that criteria. In other words, about 14 percent of unsolved cases were never sent to the crime lab for testing. For rapes, about 27,500 cases, or about 18 percent. And then, of course, for property crimes, because of the sheer volume, more than 5 million, so 23 percent of unsolved property cases with evidence were never submitted to the crime lab for analysis.

So what types of evidence were associated with these cases? Well, for violent crimes, homicide and rape, about a third involved DNA evidence, which suggests that with additional testing, some proportion of these cases could have moved forward, about 20 percent of all trace evidence or latent prints and 18 percent firearms or tool mark evidence.

In terms of agency characteristics, I think some could assume that this is a problem only specific to large agencies. And in fact, large agencies, those over 100 or more sworn, did account for a sizable proportion of the unsolved and unanalyzed homicide cases. But the story was a little bit different for rape. About 6 out of 10 originated with those largest agencies, but smaller agencies did account for a larger proportion of rape cases than for homicides. So I think it's important to understand this and design policies that don't just allow for resources and perhaps procedural changes to large agencies but also those smaller agencies.

By agency type, municipal police departments accounted for about 4 out of 5 and slightly lower percentages of unsolved rapes. And sheriff's departments accounted for about 18 percent of both. In terms of state police, about 1 in 10 of backlogged rape cases originated with state police agencies where they had primary jurisdiction.

So what were some common factors for not submitting evidence? We asked agencies to list all of these and list the most common or the primary inhibiting factor for not submitting evidence. More than half of agencies indicated they had not submitted evidence because no suspect had been identified. About 3 out of 10 were uncertain where to send the evidence, and some issues also pertained to prosecution, the agency looking to the prosecutor for guidance. About 14 percent indicated a suspect had been identified but not formally charged and 18 percent reported that the analysis had not been formally requested by the prosecutor.

Laboratory resource and timeliness issues were also cited. An inability of the laboratory to produce timely results, at least in the opinion of the agency was cited in 13 percent; insufficient funding for analysis in 11 percent; and the fact that the laboratory would not accept evidence due to backlog issues for 8 percent of the inhibiting factors.

We also asked about evidence retention. One issue was do agencies have a policy for retaining DNA evidence for closed cases. Less than half of agencies reported they had such a policy in place either because of a state, state statute or because of an agency-specific guideline. One in 5 reported they were unsure if their agency had such a policy. But, but when a policy did exist, it ultimately fell upon an investigating agency to store and maintain this evidence, 8 out of 10.

Law enforcement are also overwhelmingly responsible for retaining evidence for unsolved cases. They were responsible for, in more than 9 out of 10 cases, for storing this evidence on site in a storage location.

So what are some implications? One is that, obviously, these data indicate that these substantial forensic backlogs continue to exist in law enforcement agencies. Nearly 1 in 7 unsolved homicide cases, 1 in solve, unsolved rape cases, and 1 in 4 property cases were not submitted to a forensic laboratory for analysis.

And these backlogs are not limited to only large agencies, they also impact those that are smaller and even the very small agency.

One of the things that comes out of this study is the implication that more training, even though substantial training and funding has gone out to agencies to close cases and to move more forensic evidence through the system, more could be done to improve awareness among investigators, law enforcement officials about the need to rapidly move evidence, including DNA evidence, through the system, including enhanced policies that require submission under many circumstances.

Some U.S. law enforcement agencies continue to have the mindset that forensic evidence is beneficial primarily for prosecuting crimes and not as a primary means for developing new leads. And I think some of our presenters will talk a little bit more about that.

Another critical finding is that law enforcement information systems need to be enhanced so that they can systematically track and monitor forensic evidence in criminal cases. Many of the agencies struggled to respond to this survey for that very reason, and I think agencies have also struggled to really, over time, track how this issue was affecting their agency and what cases need to be moved forward in a timely manner. More than 4 in 10 agencies responding to the survey reported that they had no computerized system in place with this ability, and even those that reported they did, it's unclear really what the capacity of those systems was.

More guidelines and resources are also required for evidence processing and retaining evidence in agencies, but these policies must take note and count the resources available to law enforcement, both for evidence storage and for reviewing and analyzing cases.

As an aside, we also did ask about internal capacity for agencies for some nonlaboratory staff within law enforcement agencies for things like latent prints. About 4 in 10 reported they had staff that regularly conducted those activities, and about 20 percent of those reported those staff were currently experiencing a case backlog. So these things are not only affecting information going out of the law enforcement agency in the laboratories but also the ability of the agency to process evidence internally.

The other issue is the ability to improve, improve the ability for police agencies to track, to track and discard evidence that is no longer required and maintained by law. I think this was a critical issue. In many cases, agencies are retaining evidence for long periods of time because, really, they're unsure what they can safely discard of when, and for that reason many evidence storage rooms are filled with evidence that may not, may no longer need to be kept.

I guess a final implication is just the need to coordinate and develop a prioritization system for getting evidence efficiently through the system. You know, we've heard a lot of — or at least fairly new information about the potential for analyzing property crime cases, but you can see by the sheer volume, 5 million really backlogged property cases, that adding those into the system without really some type of prioritization of those could result in further problems. So I think communicating across law enforcement, prosecutors and laboratories but also prioritization of cases that need to move through more rapidly is a needed next step.

Thank you.

(Applause.)

Greg Matheson: Good afternoon. My part of this presentation of "Making Sense of the DNA Backlog" is to share with you the LAPD experience or what I also like to call it "or what happens when your backlog increases by 1,700 percent overnight."

(Laughter.)

Matheson: Before I get started on this, I do want to thank NIJ for the opportunity to share this with you, and I also want to thank the Marriott Hotel because, for all of you that may have issues with your budget and office supplies, they've been leaving pens around and notepads. It’s going to make our office supply budget go just a little bit further.

(Laughter.)

Matheson: So if you haven't caught onto that, grab them.

So. I want to do a little background on the city of Los Angeles, obviously located in the southern portion of southern California. We have an officially counted population of about 4.1 million. I anticipate it's probably a little bit higher than that. We cover 468 square miles. There's a potential for a lot of crime within that area with that many people.

The interesting thing is that our crime rate is actually the lowest it's been since the 1960s, and our request rate and our number of submissions has never been higher. So it kind of shows me that what we do as a community is becoming more important, and we're being involved more and more in the investigation of crimes.

A little on our laboratory. Our total staff right now is 171 people. That consists of 117 criminalists and other analysts, plus 54 support, and I consider support on both sides of the analysts. The supervisors and managers are support to the criminalists and the analysts, along with clerical and whatever technical support we have below 'em. Out of that 171 staff, as of today 60 are in our biology or serology DNA unit.

So I start off with our definition of a backlog. Historically and in almost all of our units still, it's just a request for analysis for which a report has not yet been issued. I figure as soon as a report is made, a request is made of our laboratory, it goes into the backlog because who knows when it's going to get done. Some are done the same day. Some may be sitting a year or more before it happens. And that's how we've always considered our backlog of cases that we had to work. And a lot is based on the decisions that are made on the backlog number.

To give you an idea of just backlog in general across the lab … The one thing I do want to point out is the serology DNA number does not include the increase of sexual assault cases, which is the main crux to this talk. But if you scan down those numbers — I mean, here we are talking about mainly DNA backlogs — there's a number there that jumps out pretty significantly of another unit that maybe could use a little bit of attention in dealing with our backlog, and that's our firearms analysis unit at 3,388 cases.

I was at a meeting not too, a while back, and we were talking about resources, and our assistant laboratory director that manages the firearms unit walked out of that meeting a little bit frustrated going, doesn't anybody care if anybody gets shot in this city? Because the resources are all going to DNA, and they obviously have a need, too. My answer to her is unfortunately right now, no, they don't. They want to know about the DNA. Now, if they were shot in association with a rape, there's a good chance that that firearms work might get done on that. But the reality is we're dealing with DNA and biological evidence.

This reminds me. I was going to give a disclaimer at the beginning regarding the numbers that are up here. I'm talking in broad terms. Every number I give up there is accurate at some point in time.

(Laughter.)

Matheson: But if you try and do the math, you know, if you remember a number before and say that doesn't add up, that's the way this works. They change constantly. It was an education process for our politicians — it sometimes took, sometimes didn't — wondering why are we giving them all these different numbers when they think they're asking the same question. Well, if they ask it today, it's going to be different than yesterday or the week before. So it's just the reality of the process we're dealing with.

So the crux to this mainly is the rape kit evidence. We have a combined kit in the county of Los Angeles. Both the city Los Angeles Police Department and the Sheriff's Department uses the same kit. A lot of the collection sites overlap. So it makes life easier for them, but obviously, the LAPD — the ones that occur in the city of Los Angeles come to us. The ones that occur everywhere else in the county go to the county lab.

Our storage conditions is we have, I think it's, nine, permanently built-in freezers that range anywhere from 15 by 30 up to about 60 by 30 that are in different locations, most of them in one, but there's a couple different locations. We also have an additional, I think it's, seven or eight freezer trucks that are leased. We store at this point every piece of biological evidence that comes into the possession of the laboratory in the freezer, and we've been doing that since about 2002.

So the pictures that I have here, and some you'll see later, are pulled directly out of the media because they've come in, and everybody wants to take a picture of all the unanalyzed rape kits that are in all of our freezers. Well, they're shooting everything, obviously, not just the rape kits. But it makes for a good graphic for them.

So we had a change in our serology/DNA backlog definition. This was a policy change or a political change. We talked a little bit about it at the end of the last session, but our new definition, when it comes to rape cases, is all sexual assault kits that are collected by the department for which a final report has not yet been issued. The detectives no longer have any discretion as to whether or not they request a kit to be made. They prioritize 'em for us, so we know what the important ones or the ones that require a faster turnaround time is, but they no longer make requests. Every kit that exists in our backlog, every kit that is collected within the city limits will become part of our backlog.

The only exception to that — and this is still up in the air — are kits that are collected where it's determined not to be associated with a crime. We're arguing the fact that if we can't load it into CODIS, why do we want to analyze it? There's that one has yet to be completely decided, but that's currently what we're standing by.

So the serology/DNA backlog — oh, there it is, prior — again, this is about two years ago, but prior to our rape kit backlog definition change, we counted a backlog of 444 cases. That was both the rape kits and other biological evidence analysis that was requested by the detectives. After the rape kit analysis — or the backlog definition change, overnight we are sitting at about 7,500 or about a 1,700 percent increase. I hope I did the math right, but the numbers sound really scary. I was going to say good, but it was an amazing change, and we had to give some serious thought to how we were going to deal with this significant change in our backlog.

So why did the backlog definition change? Victim advocates groups got involved. Among those, primarily were the Santa Monica Rape Treatment Center or (inaudible) very, very strong in our city, very influential when it comes to this type of both collection and analysis. The Human Rights Watch got involved; I'll be talking a little bit more about them. The Hollywood Chapter of NOW was the most recent addition to this group; now they're very, very involved. The National Association for Women. Obviously, the media jumped in; that's how the victim advocates get their point across or were able to get it across. And politicians. This is not necessarily in any order because they kept … You know, it would come up in a variety of different ways.

The Human Rights Watch in March 2009 came out with a report they called "Testing Justice: The Rape Kit Backlog in Los Angeles City and County," a very comprehensive report. They were working on it for easily more than a year. We became very well acquainted with many of the members associated, as did the L.A. County Sheriff's Department. And then they submitted this significant report. It's very, very long. It deals with the issue.

One of the things that I want to point out, we had a member of the Human Rights Watch speak recently at a California Association of Crime Laboratory Directors meeting. And the point that they wanted to make very clear is they're just starting in Los Angeles. Basically she looked around the room at all the different lab directors from California and said we will be coming to your lab next. I anticipate they will be going nationwide at some point because this is a significant issue to them. So be prepared. They are, as a rule, very fair about what they do, and we had a pretty good working relationship with them.

But one of the things … Don't you hate slides that just have huge amounts of writing on it? Don't worry about it. The point I wanted to make is in some respects they really get it. They weren't just attacking the lab for not getting the work done. They were very concerned about the fact that this is a combined issue. "For rape victims," I'm going to read a couple parts of it.

"For rape victims to have access to justice, policymakers and law enforcement officials in Los Angeles County will need to test every booked rape kit. But their responsibility doesn't end there. Law enforcement, in collaboration with rape treatment providers, with the support of elected officials, will need to create systems to ensure that every reported rape case is thoroughly investigated and, when appropriate, leads to the arrest of those responsible." They get that it's a whole system problem, and the crime lab is just one little piece of it.

"The remedy will require a comprehensive plan that's made known to the public, compliance with existing laws, and swift and efficient action. This is a necessary part of the core governmental obligation to protect victims of sexual violence and promote public safety."

The media, obviously, also gets involved a lot locally. They started off with a number of articles saying about how far we were behind in processing the cases. I love this one, "LAPD Rape Kit Debacle Continues." But then it starts shifting a little bit and saying that, OK, we're going to clear them all by 2010. That was, I think, a really good projection until kind of the economy fell out of things and the money that's going to be available to do it may or may not be there. We'll see. But we were shooting to have the backlog eliminated by mid-2010.

And then finally, "A Solution for LAPD's Rape Kit Backlog Announced." We have been working our way through this, and they don't always just write nasty articles about us. So occasionally there are good things.

The politicians. The head of our Public Safety Committee, City Councilman Jack Weiss, was a strong proponent of this. He goes back in his efforts to get us resources many, many, many years, long before Human Rights Watch came into play, long before the Santa Monica Rape Treatment Center. And in fact, over the last couple of years, he's given us $350,000 out of his private office account or discretionary fund, or whatever the council members have, to work on some of the rape kits.

Our city controller, Laura Chick, also audited the whole issue and came up with a not-so-flattering report. She calls for the city to end rape kit backlog once and for all. Again, an awful lot of it points at the laboratory, but she was pretty … pointed her finger a lot at both the department as a whole and the city as a whole for not providing the support, the council for not providing the support in the past.

I think one of the interesting things about this is because, you know, we all tend to take things personally, and they say nasty things about a job we should be doing, and we haven't done it. The day after her report came out, I was sitting in the airport getting ready to fly out to Sacramento, and I got a call from her on my cell phone. And we had met in the past. I had no clue how she got my cell phone number, but she did. Basically not apologizing but saying, I hope you're not taking this personally. I really want to help you out. It was just nice having her understand that.

So why weren't the cases requested originally? Well, the detective mind set regarding sexual assault investigations changed over the years. I mean, I've been … Talking about the dinosaurs, I mean, I was doing serology work back in the early 80s and late 70s, and at that point no hospital wanted to take a rape victim. They would sit in the emergency room for hours on end waiting to be, have their, their evidence collected. It was a nightmare. Detectives didn't want to handle it. It was relegated to a small part of the department.

Obviously, things have changed. The SAR programs, or the SANE programs, the evidence being collected is significantly better than it used to be, and the detectives are coming around understanding that this is an important case that needs to be investigated. When the California Cold Hit Program came out, which was in about 2001, 2002 and I met with a bunch of detectives at that point, some of the less enlightened ones said, well, if you get hits, we're going to have to do more work. You know, what are you doing this for? Obviously, that's changing somewhat, but this explains why they weren't requesting them.

They also had the mindset regarding comparison versus, you know, answering it. People did not … well, prior to CODIS becoming the level that it is now, we would discourage a request to be made for a kit unless they had a suspect in custody, unless we had something to compare it to because it was still more of a prosecutorial tool or confirming what they already knew. Now it's a investigative tool.

Detectives would self-triage the cases. They wouldn't make a request because they knew we had limited resources, and they wanted to make sure that the really important one we'd get around to. So they just didn't request a lot of them.

And obviously, our laboratory resources, even if they did request them, at the time we couldn't have done the analysis.

So … oh well. In late December of 2008, the part of our department that we were moved to, which was the Detective Bureau, decided they wanted to know exactly how many kits were in the freezers. Up to that point, we were calculating them by a means that left a lot to be desired. So the chief of detectives said we're going to find out exactly how many kits are in the freezer, and he ended up getting 50 detectives over about a two-week period — they spent in the neighborhood of about 2,000 hours — with parkas on and notepads looking at every item of evidence in those dozen-plus freezers that we have to determine how many rape kits or sexual assault kits were in the freezer.

What they found was there were 11,077 rape kits that were currently in the freezer; 48 percent of them had already been analyzed. I thought that was really good. And 52 percent had not been analyzed. So now we had a firm number. That 7,000-plus was calculated. We now know that we had just under 5,200 kits that actually had not been analyzed.

Of the 5,200 that were not analyzed, you can see how they broke down. One of the reasons that some weren't analyzed, 23 percent of them were cleared by arrest. Why request the analysis if it's already been determined who did the crime? Thirty-six percent were cleared "other," or most of those were D.A. rejects. They take and file, you know, take 'em to file. The D.A. would reject them. Why do the analysis at that point? "Investigation continued" was a big chunk, and then 770 were not eligible for CODIS upload, which was great because we immediately dropped our backlog by that 770.

Let's look at the "investigation continued." We had 70 percent were known suspects that they were still looking for, they were doing the investigation on.

But the important one is this middle one, 29 percent stranger crimes. We had, I believe it was, three detectives who were doing nothing but looking up those 5,000-plus cases to find out what the status of them, and they were saying out of these 402, they're looking at this going, why didn't the detective request it? So that really was a situation where the detective fell down on the ball. So really, from our standpoint, I think we really kind of dropped the ball on only 402 out of the 5,000-plus. Those are the ones where we should have been being hammered for because those should have been collected.

But there are reasons to do those other ones. Maybe a D.A. reject wouldn't be a D.A. reject if we analyzed it, uploaded it, and found it hit to another case. And there's a lot of reasons out there to analyze kits, and we aren't currently doing it.

So how can the laboratory respond to a big change in the backlog definition?

You know, the first thing is you get defensive. It's not our fault. You know, we only did what we were being asked for, you know. Go away. Don't bother us with this sort of stuff.

The next thing is argue the necessity of the case. Well, why do all of them? You don't need to. It isn't needed for investigation. You know, we can come up with a lot of reasons why we don't want to do this work.

Finally, you might want to embrace the opportunity, and I'll get into how that's worked for us.

I went through three of those. I mean, I think that my position on analyzing these kits has changed significantly over the last two to three years because I've become educated on what all is involved in the cases and what can come out of it.

So backlog versus resources. How do you determine your resource needs within it? You know, we now have a major difference in the backlog, but let's take a look at it. There's different ways of doing it. We used to always do it based on the request for work. You knew approximately how many requests you had coming into the laboratory. If you weren't getting 'em done, you'd go to the city council or whoever it was, ask for more resources based on what you were being asked to do. That's how we always did it before.

You could do it based on what you think you'll get in the budget. I mean, there were times when we knew that the city budget was going to be poor, so we didn't ask for a lot, even though we needed it. The problem with that is when the budget gets good again, they say, well, why didn't you ask for it before? If you didn't ask for it before, you don't really need it now, so I'm not going to give it to you. So you always ask for it.

Finally, you do it based on potential workload. Had we been asking for resources in the past when there was a policy change, and our backlog went from 444 up to 7,000-plus, you know, maybe we would have said our potential backlog or our potential workload is 7,000 cases, and maybe 10 years ago we would have started asking for more resources based on that. It probably wouldn't have gone anywhere, but who knows?

So finally, we know we got to do it. We've got these kits. We have to get it analyzed. So I had to develop a backlog-elimination plan. I'll go through the overview of it real quick and then go into specifics.

First, identify the workload. Backlog is any case coming in that hasn't been analyzed. So you got to take into account your new including, new cases.

Determine your capacity, how many you can get out internally and externally. We did a lot of outsourcing, by the way. We do DNA in-house, but we do a lot of outsourcing.

Identify the funding that's available for it.

Identify other resource needs, such as we rely on our property division to move the evidence around. We rely on detectives to do what they're supposed to do. Our success is based on a lot of other areas, and they're prepared to constantly revise the plan.

So when we're talking about backlog, ours was determined by the hand-count that the detectives did. However, we got new cases coming in all the time. That's going to change depending on the crime rate trends, the number of cases that are being reported. As we heard earlier, I mean, as we know, many of them are not reported. Well, what if all of a sudden, because we're getting more hits, and we're doing the work, everybody starts reporting or more people start reporting? That's going to change our new cases, which is going to throw off the plan.

Technology changes. Sometimes we'll get a new technology that allows us to analyze kits out to a week later or something. That's going to change the number of cases coming in.

And obviously, political changes will change the amount of cases that you have.

So just to give you an idea of what our historical average, if were basing it on the requests that we receive for analysis, you can see that from 2004 through roughly 2006, it stays constant and then it starts rising. So basically we've seen a 63 percent increase in requests for analysis in five years. Ignore the fact that we're being told now to do all the kits, requested or not. Just our requests went up that much. So if you're planning on figuring out how or the resources you need, you have to take that into consideration.

You've got to determine the capacity internal. You have to look at the individuals, how much they can do. Look at average unit productivity levels and figure out how many cases you can get out the door. The number of analysts you have may or may not – or it may change. As you'll see, ours changed significantly.

Technology changes. They become more efficient usually. You can get more cases out the door. Or a new technology comes along and all of a sudden everybody is less efficient. You know, all these things have to be considered and taken into effect or you throw a lot of disclaimers into your plan so they know that it's not necessarily going to be accurate.

Like I mentioned, we do a lot of outsourcing. The capacity of the different contract laboratories we use are changing constantly depending on other contracts they have, what else is happening in the rest of the country.

So looking at internal staff capacity, in fiscal years '07-'08, we received 13, 15 new positions for the unit alone. In fiscal year '08-'09, we received 16 more positions. This is all coming in because of – OK, where is it? There. In the next fiscal year, which is amazing considering what the whole fiscal situation is, we're supposed to get 20 more criminalists and a lab tech. So in the City of Los Angeles in the next fiscal year, there's only two job classes that are being allowed to hire: criminalists associated or staff associated with DNA and police officers. That's throughout the whole city.

Laboratory growth due to the rape kit. We went from 140 in 2007 to 171. We're going to go to 192 people in the lab next year. In the serology/DNA unit, we've gone from 28 to 81 in the last five years. Significant growth in internal capacity.

Funding for this project, backlog elimination. We will have received about $5 million in general funds – federal grants, about $2 million. Thank you, NIJ. Donations, which is an interesting funding source when it comes to police work. So far, we've had about $1 million and it's continuing to come in. The donations have come from the Los Angeles Police Foundation, the Hollywood NOW is out there and other civic groups.

Regarding the Police Foundation, they have a Web site up and circled up there is a button that says "donate now." So if somebody is on their Web site, they receive donations from individuals, you know, $20, $30, $40, up to large groups that have given us – one group has given us a half a million dollars towards this one topic, elimination of our backlog of sexual assault cases. That goes to embracing the opportunities.

We could continue to argue this forever, but we've received a lot of resources because of the bad press that this situation has received. I'm sure the city of L.A. isn't any different than most of your jurisdictions. The reality is they work on crisis, and this was a crisis that became a political issue. And we are embracing it for everything we can to grow the laboratory to meet the needs.

Identifying other resource needs. Like I mentioned, we rely on a variety of both internal and external groups.

You need a grant coordinator to handle all the grants coming in.

Contracts. We are dealing with four companies right now and the contracts change on a regular basis.

And I know none of you have problems with your IT group. You get always what you want right away. Yeah. They're interesting. They have held us up a couple of times.

Reconsider, re-evaluate, revise, revise, revise the plan constantly. Workload changes. Capacity changes. Funding could increase or decrease. It has gone up. It's gone down. They find more money. They tell us to increase our amount. The next day they figure, "oops, we didn't mean that. It's less." The number is all over the board. So you have to set up a way of dealing with every change that comes along.

Another little bit, but this is just an example of a portion of the spreadsheet that I used to both predict when we'd get done. We don't have a laser pointer. But if you look up there, the left-hand column is we know where we started at a certain point in time. We know how much money we had. We know what our backlog was, and you work your way down. By adding in the number of new cases that you estimate will come in the door each month, the amount of money that's going to be used on other types of contracting, the amount of kits that you expect to send out — all of these are estimations at the beginning of this program — you can figure out approximately how long it will take, given the cost. Any of those factors can be changed, and it will adjust the top number. You know when you're going to run out of money by, if you insert higher numbers of kits that are going out the door. You know roughly when the backlog is going to be eliminated.

On our current plan, our current spending, we're looking to be done with the backlog of the 4,000-plus kits by the end of the next fiscal year. But it's been a great tool for both predicting and then tracking whether or not you're keeping up with your plan.

Current status. We have a backlog of 4,786 kits. Since we started dealing with that outsourcing, which was only last September, October, we've outsourced almost 2,700 kits. This is where the math doesn't work, if you remember some of the other numbers, because our backlog … You know, you'd think if you've got a fixed point in time and you counted every kit that's in there, that number should stay the same, go down as you get them done. They find more. I mean, it's just … it was an interesting process.

Like I mentioned, you have to take into account the new rape kits that are coming in the door. So far in 2009, we've been averaging about 26 per week. That's the number of sexual assault kits that are collected associated with crimes in the city of Los Angeles.

Then the next question is — we get this a lot. You're asking for all these people to eliminate your backlog. When the backlog is gone, you know, gone, what are you going to do with them? Well, I can think of something. We might use them for quicker turnaround, more cold cases completed. Maybe we can find more of our detectives that committed crimes 20 years ago.

(Laughter.)

Matheson: For those of you who have heard about that case.

Obviously, we're going to be analyzing every new sexual assault that comes in the door, and we want to do it without outsourcing. We've been outsourcing DNA now for 21, 22 years and would really like to be able to do them in house.

Obviously, we also want to move on to doing more property crimes. We were involved in the NIJ program on that, on the burglaries and DNA. Amazingly effective tool when you're dealing with property crimes like that.

Finally, I find it interesting that … Keep in mind — all of our caseload is request driven except for the sexual assault kits. So everything, we know what our backlog is when it comes to all of the rest of 'em because they're the requests that we have received for analysis.

But what if suddenly somebody comes along, like they've asked, what about all the rest of this biological evidence you have in your freezers associated with homicides? Shouldn't that all be analyzed too? Well, they recently went through and determined that right now there are 42,927 items that are frozen storage that are associated with homicide evidence. We all know all of that is not probative or informative when it comes to the investigation of a crime, but you never know when somebody is going to come along and say, do it anyway. And boy, look at the resources we can get out of that if they force us to consider doing all of the homicide evidence.

And that's it. Thank you very much.

(Applause.)

Kevin Lothridge: If you don't have enough pens, we have NamUs pens right outside the door. So we'll help your budgets also.

Before I get started, I'd like to thank everybody in here for filling out the survey that we did, and we're in the draft phase of the first report to go into NIJ. As in all things, no good deed goes unpunished. I think the reason we got the work is we're really good naggers. We, we kind of call everybody up. And it's kind of the same as Kevin was saying. You know, you have to work together to get this data, and I think we've been doing it for a long time.

Those of you who are old like Greg in the room remember that we did workload surveys for ASCLD years and years and years ago, and they kind of dropped us. And now we're doing all different kinds of data collection. But I would like to thank everybody because it's not easy. If you look at what Greg had to do to get the data that he was required because it was driven, those are some of the same questions people in your agencies ask you every day. And it's not just about one type of evidence. It's about all types of evidence.

And as we go through here, I'm going to talk to you in generalities about the study we did because it is not final. It's preliminary.

I'm also going to apply a personal story at the end and really kind of try to tell a story. I think we do a really good job. We have numbers. We have charts. We have slides up there with lots of things that people can't read. But it's very difficult for people that aren't in the laboratory on a day-to-day basis to make some understanding of that information. They think it should just be done.

Greg made a good point. I think Kevin made a good point. When you take a measurement at one point in time, it's kind of like the wood pile. You think if you take pieces of wood out, it goes away at some point in time, where our wood pile just keeps getting bigger and bigger. It changes because of things that happen. So it is something that's important to remember.

So what's the issue? We know that there's been tons and tons of public attention on the funding that's been provided to the DNA backlog. I think, Mark, you said somebody called you and wanted to know why it wasn't done. Well, there's lots of reasons for that.

And we also have to look at what are the future demands, not just in one section but all the sections. If you look at what Kevin put up there, it looks like we're already 50 percent in the hole in capacity across the board for different types of analysis.

The considerable attention we've paid to this — it's been great. However, there's still evidence awaiting and having delays. I mean, following up Greg, I can't say it any better than he did. He's actually got the numbers. He's in the trenches, and he's actually in the freezer in the parka it looks like.

Why does this backlog exist when we have put capacity to it? It's because it's always changing. There are resource implications, new technology, new rules, new laws, new collection types. And really, the programs that have been developed have tried to look at the DNA backlog and tried to make efficiencies and enhance capacity. But as that's gone on, the desire, the need and the requests for analysis continues to increase.

The study? We did this at the same time that RTI did this for the law enforcement evidence. It's to provide an update on the nature and extent of backlogs in the nation's publicly funded state and local crime laboratories providing DNA analysis.

I was fortunate enough to work with the BJS studies in 2002 and 2005 where, as many of you know, I called you until you filled it out. And we really did the same thing here. We're really trying to provide information not on just the size of the backlog, but also what are the levels of demand. Is the capacity truly there that we think is there? And what's the reliance on funding and, in this case, federal funding?

I think I've not seen anybody present it as well as Greg just did as far as what resources have been put in from the general funds to the federal funds, and I think a novel approach is the donations. That's actually pretty amazing.

And this is still a work in progress. As with all OJP reports, it must undergo peer review. Once the peer review is final, the report will be delivered to NIJ.

What was the methodology? We actually went and looked at all publicly funded crime laboratories and the laboratories accredited by ASCLD and Forensic Quality Services International, the same ones surveyed through the BJA Census of Publicly Funded Crime Labs in 2005. And the final pool of respondents was limited to those laboratories that were accredited and operating DNA analysis programs at that time. We did not look at any federal laboratories or military laboratories providing DNA analysis just to make sure there was no confusion of what we were talking about.

The survey itself was an online form, which we really thought was a good tool. We tried to make it as easy to fill out as possible, but also hoping that the questions would be answerable by the data systems that everybody collects their data in. That's not easy. A question is not always answered the same way. We don't define backlogs the same way. We don't define DNA cases the same way. We don't define forensic biology/serology screening cases the same way. And as I go forward, we'll talk a little bit about that.

We did follow-up communication with the key people in there and did the standard, typical survey model.

Response rate? I have to say this. Each laboratory was asked to fill out the survey individually. However, for many cases, the state lab headquarter laboratory filled out one for the combined state system. Every state laboratory provided a response and a large percentage of local laboratories responded. We were very pleased with the response. It does take time, though. People were like how can it take so long to do a survey. Well, when you ask people and you give them a deadline and the deadline slides and you call them again, that happens. And you have to provide data analysis, and you have to really do a job that nobody really wants to do because it's just another extra task on your plate.

What are the outcomes? Backlogs still exist. That was no surprise. The varying definition of backlog is troubling. I think Greg just did a great job in explaining how their backlog definition changed because it really was those cases that had no report. It wasn't all the cases that most people have. Sometimes you provide services for folks that don't know where their case is. They think they've sent it to the lab. It may not have been sent to the lab. There are some real issues there.

Demand and capacity. I think this is really interesting. The preliminary data shows that almost a doubling of DNA requests between 2005 and 2007. Now, that is actually going along with what Greg said, and not only that. With the low crime rate, your requests are actually increasing. Output has increased to meet the demand at the time but not to reduce the backlog. So if you look at that, the expansion of programs and things are working. It's just that the benchmark keeps changing.

And demand for DNA analysis continues to outpace available capacity. Again, a lot of this is intuitive information. We had to do it with statistics, but I'm not a stats guy. It's pretty simple. If you can only do X number and you get Y, there's going to be a backlog.

And the federal funding. Laboratories reported there will be an increase in backlog cases if federal funding was not available. I think that that's a fair statement. I think Greg would have been in the hole $2 million even with his $1 million worth of donations.

And a large percent of the funding actually was going towards training and equipment. Training has increased greatly. I hate to use Greg as an example, but he had the data just up there. When you add that many people, you got to train them, and the number's gonna increase. The 20 more for next year has another impact on getting more cases out the door as you're doing the training.

The story behind the backlog. I think you have to put these things in proper context. Forensic science in general lacks the capacity to perform cases as they arrive at the lab. Kevin said that the law enforcement agencies had well over 50 percent more items that were not submitted to the laboratory. It's pretty easy to figure out that the capacity is just not enough to even do what we have, let alone the ones that aren't submitted.

Many times they're held for a period of time before they get to the lab so they immediately become a rush case and a backlog occurs. And so they become backlogged.

Here's an example. It's been a while since I ran a crime lab, but I can tell you we did controlled substances and fire debris analysis, mainly chemistry. But in 1985 when crack cocaine hit the streets of Pinellas County, Florida, nobody even thought what was going to happen with that. It really blew up overnight. Our lab moved in 1986 to a new building, so it took time away, and we had cases that were on-hand being held in the locker.

In '87, the lab staff takes other jobs reducing the staff from five to three. So we actually had almost a 50 percent decrease in case-working people.

Increase in law enforcement investigations in the street-dealing of crack cocaine, without consulting the laboratory who was going to receive all those samples, caused a six-month case backlog into 1988. Judges in Pinellas County — if you go query the St. Pete Times, you can look at this. They were kicking the people out the door. They said the lab was woefully understaffed and that this was just a terrible, terrible thing. We needed to get these cases adjudicated.

So we add some resources, like everybody does, hire four new analysts, train them. The backlog returns to 30 days. Total time, even being predictive in something as relatively simple as controlled substance analysis, four years. That's in one small jurisdiction where the median age is now about 53 years old. Back then it was about 63 years old. And that's still a lot of drain.

And when you look at this, you have to put this stuff in proper context. Everybody wants to do every case they possibly can. They want to provide the timely necessary service. At some point in time, the requests outweigh the capacity and these cases are going to be backlogged.

If any of you were in the session earlier, it's actually the NFSTC booth outside the door here, and we have the NamUs program. And when we talk about looking at other backlogs, we do have a huge backlog of missing and unidentified persons cases. Again, it's one of those where we can get the data, get it in, and it's a capacity thing also.

Thanks.

(Applause.)

Nelson: OK, do we have some microphones available for the audience since they may want to ask questions? Do we have any questions, first?

Questioner: I have one question. (Inaudible) one of the things that has bothered me for years, and it has been reiterated many times, and that is there is no universal definitions of backlog. And recently NIJ has made an attempt to do (inaudible) to define what a backlog is. Can any one of you address that and maybe give us some ideas as to how to (inaudible) universal definition back home (inaudible)?

Nelson: OK. For those of you on this side of the room, the question was we don't have a universal definition of a backlog. Does anybody want to take a crack at what a universal backlog might be? How do you define it?

Lothridge: Well, you have to define the terminology first so that you can actually apply it to the backlog, and because we don't say what a case is, what an item is, how things are counted, until we say if it's possible. Now there have been many years — Greg and I were talking to each other just now saying this is something that everybody has asked this exact question. Until you can define the terms that say these are identifiable units, the backlog is going to be what each and every person reports as their own backlog. So you're not going to have a total number as of date certain. You have to get the terminology down right. You have to say what an item is, what a case is, and identify where they're at. I just don't think it can be answered right at the moment.

Matheson: Obviously, I talked about a backlog definition change that we had. I agree. I mean, getting some sort of standardization is very difficult. I would love for somebody to come along and say from now on, you have to call it X, Y and Z. Unfortunately, you know, coming from an agency that has a heck of an ego, they definitely like to call things the way they want to call it unless they're being mandated to do it. You have to figure out a way to get along with other people. So coming up with a standard one is going to be difficult.

But to answer your question, number one, I think it has to be request-based. I mean, obviously, we don't know what exists out there in most areas, and that does actually pertain or go along with our current situation because another way of saying that all of, all of a sudden our backlog became all the kits in the freezer. Another way of saying it is overnight all of those were requested to be analyzed. So if you start on the basis that it's the cases that have been requested to analyze, then decide at what point it no longer is a backlog, which should be at its final state of the analysis, whatever that happens to be.

The final thing is do you count it from the request is made or do you count, give yourself a 30-day or a 60-day? That's something within our laboratory we're actually looking at because many people don't count a backlog the moment a request is made because if it's going to be completed in a timely manner, they don't want that on their books. So a lot of places when it comes to biology, they don't consider it a backlog until it's like 60 days old. And we're considering that.

In our case in narcotics, it becomes a backlog immediately, but as opposed to a lot of other agencies, we have to get the narcotics done within about 48 hours. So we don't carry much of a backlog in that.

Nelson: Does anybody know whether the Foresight Study is addressing this issue? Yes, George?

George: As a participant in the Foresight Study, I can tell you that we've spent days coming up with standardized definitions for what does it take, you know, what is the service request, what is the item to be tested, and what is the backlog. And it's not to say that if it meets any one laboratory's internal definitions, but it allowed a consensus definition that could be used across the North American continent actually.

Nelson: Thank you, George. Tim?

Questioner: This is a question for Kevin Strom. According to IACP, I believe they say there's about 42,000 law enforcement agencies in the United States, and you surveyed about 3,100. Does that mean the true backlog is about 15 or 15 times higher than what you're recording?

Strom: I don't know about the 42,000 in the United States. That may include, you know, many types of specialized police. General estimates are about 17,000 to 18,000 state and local law enforcement agencies, and we surveyed a representative sample of those, which was about 3,000 agencies, and then applied statistical weights to give us a national average.

Questioner: So your numbers were based on what you think is the national average?

Strom: It would be a national total, correct, for state and local agencies.

Nelson: Pat?

Questioner: A question for Greg. Now that you've removed the discretion for sex crimes, why is it not reasonable to then move forward on attempted homicides or homicides and also remove that discretion (inaudible)?

Matheson: My personal opinion? I mean, one of the reasons that the discretion was removed from sexual assault cases had to do with, number one, it was a very easily defined piece of evidence that we could locate within our freezers. I mean, there's some practicality involved in this. It doesn't necessarily involve sexual assault cases at this point where there is no rape kit collected from it. The issue was the sexual assault kits because you could identify those and count them. It also had a very strong and well-organized group of people that are interested in seeing these happen. So the likelihood of another type of analysis getting the same kind of attention from external forces I think is fairly small.

From my own viewpoint, I think it makes less sense because you do need the detective's input to determine whether or not a piece of evidence is informative or probative to the investigation. And not that it isn't important to have their information in the sexual assault kit, but anybody in the room that has ever worked sexual assault investigations, there are so many unknowns, and the victims are reluctant to give information many times, that it almost pays to do the work and then figure out later on what the events were from the evidence as opposed to the other way around.

Nelson: Joe?

Questioner: Kevin, I was curious about your statement. You said that 29 percent of the agencies (inaudible). What does that mean? (Inaudible).

Strom: You know, we didn't really ask for clarification on that particular response. It could have been, in some cases it was for the very small agencies. For example, on the issue of no suspect identified in the case, I think 70 percent of the agencies that responded to that question were very small. So sometimes it's a learning curve at the smaller scale and that could be an issue with that response as well.

Nelson: Go ahead. Lytton?

Questioner: I have to say, as a CODIS state administrator, that I'm confident that the Los Angeles Police Department, local DNA system administrator, would be ensuring that only forensic (inaudible) profiles are coming into our state DNA (inaudible) from this massive undertaking on your part. And I'm serious about that. I know what's going on with you guys, and I trust that. But we've got to understand that we cannot just put any profile that we pull out of a piece of evidence into CODIS. It's got to be (inaudible) forensic (inaudible) profile, and there's a technical definition for what that is and how you figure that out.

Part of it is eliminating other persons, not just consensual partners and so on and so forth.

Matheson: That has been one of our major concerns throughout this whole area, and if you remember — I am going to go back — the review that our detective bureau did of all the kits in the freezer, one of the things that they did was they looked up every single case, either pulled the case package or looked in the automated system, to determine what the status of it is and where the evidence came from and associated pieces of information. That's why we found out that out of that batch, 700-plus were considered to be unfounded, and those won't be analyzed and won't be uploaded. So it definitely was a concern of ours.

And actually, that's one of those areas where we very much appreciate the fact that there are rules out there. I know that at least one of the strong victim advocate groups feel that we shouldn't take that into consideration, but having the law behind us, we can tell them, no, we have to take that into consideration.

Nelson: Barry?

Questioner: (Inaudible.) (Inaudible) a number of our other clients, the smaller police departments throughout the county, were satisfied and happy with that because they (inaudible) property crimes and (inaudible).

One of the solutions that our liaison has been expressing (inaudible) was to allow individual police departments or clusters of police departments to pay for AVLS to work on (inaudible), which creates another whole series of issues for us (inaudible). It's difficult to just turn out fully trained DNA examiners. It's a one- or two-year process (inaudible). We're still struggling with that particular question.

Nelson: So that I understand correctly, are you saying that the local police departments are going to pay to put an analyst in your laboratory to work their cases under your supervision and control?

Questioner: They would be county employees, but funded through municipalities in our jurisdiction.

Nelson: That's an interesting concept.

Yes, sir?

Questioner: Greg, I guess one of my questions — because you spoke very eloquently to the fact that the political pressures that (inaudible) bringing pressures to bear on you. The concept is — and you mentioned this very briefly — that they understand this is all progressive; you're only one small stop. And so you're moving with this, from your widget cost into someone else's widget cost. And when they see the pressures that are going to be applied by them, my point would be in (inaudible) to the prosecutors. Are they going to be prosecuting all of these crimes that you're now working? Because they're going to have to have the same kind of uploading of new prosecutors that you've had (inaudible).

Matheson: You know, I have no idea whether or not, you know, where they're going to go with it, but they will have the same groups watching them that watched us, and maybe they'll be able to get additional resources out of it. Along with the growth in our laboratory, our rape special section or our cold case group in the police department right now has more detectives assigned to just the investigation of these type of cases than anywhere else in the country. So they have received additional resources for exactly that same thing. So I don't know. We'll see.

One of the things that I didn't mention is Joe Peterson from Cal State L.A. is going to be doing a very exhaustive review of the experience that we have, what prosecutions come out of it, how many hits we have, and try and correlate it to the type of evidence that was collected or the type of situations surrounding the crime. So hopefully, by the time we are done with this, and he's done with his study, it will allow other laboratories or other jurisdictions to decide whether or not there's value in analyzing all of the kits or maybe just the kits that fall into a certain category.

We're going to kind of wait and see. In the meantime, we'll be done with our backlog and other people, hopefully, will be able to make some more intelligent decisions than — or more informed decisions — excuse me — than was made in our jurisdiction.

Nelson: Yes, ma'am?

Questioner: I heard something surprising. I was at a (inaudible) training last week. In Massachusetts (inaudible) the vaginal, the genital, (inaudible) rectal (inaudible). And we assume that just testing the vaginal swab, if we got a positive result there, it was a rape case that is done.

And the D.A.s were saying they actually like all of these laws tested, because if they can find, they can bring different charges depending on where they're getting positive results. And they want to stack up as many charges as they can, and some of the charges will get knocked off, and they're hoping that they'll get a sentence for one, a sentence for the other (inaudible).

Does this come into if you have to do four swabs from four different cases, that's four cases off your backlog versus four swabs and that's all in one case?

Matheson: OK, when it comes to the backlog or the previously unrequested or whatever the backlog is, unknown or stranger rapes basically, our goal is to identify that person as quickly as possible. So in other words, the process that we're doing to analyze these kits, the stranger ones, is to have them start working their way through the items until they find foreign DNA from the victim and stop at that point. That then can be uploaded, if it meets the rest of the criteria in the CODIS.

If we get a hit and they move on to prosecution, then it's entirely likely that that kit will then become a new request. It will come back to the laboratory, and all the rest of the items will be analyzed to determine whether or not there's additional charges. But the first round through is justifying something foreign and get that uploaded into CODIS.

Nelson: Any other questions?

(No response.)

Nelson: Join me in thanking our panelists today. I think they've done a great job.

(Applause.)