Meet the OJP Science Directors: Nancy La Vigne and Alex Piquero Discuss the Future of Research and Statistics at the 2023 NIJ Research Conference

DR. NANCY LA VIGNE: Good afternoon, everybody. Thanks for joining us, especially all of our students. We value you so much, and we've had a lot of opportunity to be out and about across the country, meeting with a lot of different types of audiences, practitioners, community members, policymakers, and yes, students. And with all due respect to the practitioners, community members and policymakers, students are our favorite audience, because as seasoned as we are in our careers, we remember very well what it was like to be a student. We remember like it was yesterday, when we were students attending the NIJ conferences of the past, and getting to listen to PIs who were published and articles that were assigned to us in grad school. It was just such an amazing experience. We reflected recently on how we would have loved to have an opportunity to talk to the then directors of NIJ and BJS. So, we want to prioritize your questions, but I also see there's other members of the audience here, so thanks for joining. We are doing something we have done once before where we were asked to do a video on what we know about the topic of reentry. And we just sat down and talked. There was no script. It actually went very well.

DR. ALEX PIQUERO: No talking points.

NANCY LA VIGNE: It was kind of interesting.

ALEX PIQUERO: And at the end of the day, and it hardly had to be edited. No lighting, makeup, or anything like that.

NANCY LA VIGNE: Well, a little makeup. So, we promised that we would talk about the future of research and statistics. But it occurs to me Alex that it's hard to talk about the future without kind of looking at the past and then thinking I've had the pleasure of hearing your story about what got you into the field and how that frames your perspective today. And I thought maybe you could share that.

ALEX PIQUERO: Nancy and I went to graduate school about the exact time. I went to a better school. I went to University of Maryland. And...

NANCY LA VIGNE: Rutgers rules.

ALEX PIQUERO: No, no clapping. But prior to that, all of us had these different life trajectories of how we ended up on the stage and I'm sure I could speak for Nancy that when we were 10, 14, 18 years old, the last thing we would've thought about, we'd be sitting on stage being appointed by the President of the United States, to lead two scientific agencies, which is an incredible honor that we were able to do this. But I'm a child of two Cuban political refugees, who came to the United States with nothing. Literally nothing. And I remember as a kid growing up, my mom and dad always said, "Hard work. Education. Family. Loyalty."

And all through high school, went to college, and the only reason why I became a criminal justice major was that one of my classes didn't make. There was no dropout on the computer back then. A little older. So, you wait in line, wait in line, I got to the front of the queue, and the only class that was open, Tuesday, Thursday at 11:00 because I didn't want an 8:00 class, was Introduction to Criminal Justice. And the faculty member made the topic come alive. And totally changed my entire life. One class, that absolute serendipity of being in a classroom with that professor. And then I started doing research, and I got the bug for collecting data, and in analyzing data. Then the person who became my mentor said, "You should go to graduate school." And so, it happened to be taking one class that totally changed my life. So, these random shocks that occur in your life. And then meeting Nancy over the course of my career, and the suggestions she has made over the course of my career.

Each of us have come to this place and followed different career trajectories, but we have the exact same mission about the importance of science, the importance of evidence, and the importance of statistics to improve the human condition, and that's what we in the business are doing. And I think that there's not a day that goes by that we probably don't talk multiple times a day. But the point is, what can we do better today? What can we do better tomorrow for our agencies, so that they are the agencies you need them to be for the work you need from us? So, your story is somewhat different.

NANCY LA VIGNE: It's somewhat different. We both ended up with the letters after names. But my mom was first-generation, Italian American. She was a real tiger mom. It was all about education, work hard, work harder, so not dissimilar from you. But I actually was a government major and econ minor in undergrad. I had the opportunity to come to DC for my junior year and work on the Hill. And I just got enamored with the whole thing about policy and changing policy. So, after I graduated, I went and got my terminal master's at the LBJ School of Public Affairs and thought that that was it. I mean, I knew I wanted to be in crime policy, but I didn't think I was going to go the academic route because I knew I didn't want to be an academic. And that all changed one day. Did I ever tell you my "Little Lady" story?

ALEX PIQUERO: No. I know a little bit of that, but I think it's important one for people to hear.

NANCY LA VIGNE: I call it the "Little Lady" story. So, I was working for a sentencing commission in Texas, and I was the Research Director, which was a lofty title. But essentially, my job was to analyze data that someone else had collected and look at issues that might drive changes in sentencing guidelines because at the time that there was, I think judges had discretion of five to 99 years for first-degree felony. I wanted to look at whether judges were making decisions based on race or ethnicity. And in a very public forum, somebody told me that that wasn't possible, the ends were too small. He said, "You think you know statistics, but you only went to the LBJ School, and you only got a Master's. So, you really don't know, and you can't do this." I got angry for a minute and then I'm like, "I guess I'm getting my PhD now," because I wanted to be relevant and heard. I knew darn well that the ends were big enough to analyze differences and disparities by judge. But I felt like I had to get the degree to prove it. So, I never really desired to go, with all due respect, into the academic realm. I graduated and my first job out was actually at none other than NIJ.

ALEX PIQUERO: The academic realm is interesting because being an academic for 25 years, we try to produce ourselves. It's really this bizarre thing. We don't train people to do the kinds of work of not just writing the great PDF article that's going to get published in CPP or whatever, which is important and needed, because that's how you build a body of evidence. But more importantly, how you get that information to people in ways that are relevant to them for their particular purpose.

That last part I think is really important. I think Nancy's career and the different roles she's had at NIJ, the pioneer of the Crime Mapping Research Center, her work at Urban, and the Council of Criminal Justice, she's always had her hand in some aspect of policy, not just one of those. Nancy is a really good example of a jack of all trades. Done a little of this, done a little of this, and done a little of this. I kind of like generalists. I think generalists are a good thing in our business.

NANCY LA VIGNE: Well, it's true. And you too. You absolutely have areas that you're very well known. When I knew you were going to be appointed, I'm like, "That is a stroke of brilliance," because I don't know anybody else in the field who has published using more different BJS datasets on more different topics than Dr. Alex Piquero. But that's not rewarded in academia. So, I'm curious about how you got so successful, not doing it the normal way. But I also want to say that I too have dabbled, and I'm a life-long learner, right? I always want to learn new things. And by dabbling, I can now connect the dots between reentry and policing and correctional environments and filling the blank in a way that academic environments don't necessarily reward.

ALEX PIQUERO: That's right. They're not rewarded, and they don't train people how to do that. And so, I had to do it backwards on my own. There's nothing wrong with writing another article. That's really kind of fun. But at some point, it's like, "Okay. So, what do I do with this?" And I fortunately got asked to be on several National Academy of Science panels. One of them was on evaluating the research portfolio of the National Institute of Justice. And another one was on modernizing crime statistics. And over that period of time, being around a different series of academics, and policymakers, and my work with the McArthur Foundation on our research study, actually brought me to go, "Okay. So, yeah, this is great, but now what?" Because all these people talking to me are spewing stuff that is just not based on anything. At least I know something, which is a little bit better than nothing. But some people don't know anything, and that's not good. And that's not good for anybody because parents read the newspapers, people watch the news, and if they're getting information from one source who doesn't understand or has a horse in the race then that's shame on us for not going into that terrain.

Now, when you go into that terrain, and you work with policymakers, and you work with the media, you have to be willing to take the hits. But your eyes on the prize, and the prize is, “How can we take that information and give it to people in ways that matters for them?” I think it dawned on me one day when I was sitting at an NAS panel, and I said, "Different people need different data and different ways for different reasons." So, when I talked to my mom, my mom doesn't care about the standard error and the likelihood function, or the particular model that was a negative binomial. It's irrelevant to her. “What did you do? What did you find?” And most importantly, “Why does it matter?” And what's helped me in my career is working with police chiefs. It's actually sitting down and saying, "I'm not going to parachute in and jet out. What can I do to help you? And if there's nothing now, fine. Here's my number, call me if you ever need anything," or if you need an answer to something, you build those relationships and then you see them seeking out the work.

And then when I worked with mayors, there are mayors who don't know what they don't know. We don't know what we don't know. And so, when they seek us out, our job I think, is to provide them what they need for their particular purposes, recognizing that you may not always change the policy today, but it might change three years from now or five years from now.

NANCY LA VIGNE: So, one of the things that's on my mind just thinking about the last day and almost two, there's a theme that's run throughout the plenaries and actually a lot of the panels I've sat in on, and that is the importance of relationships and authentic relationships. I was thinking about how the conversation on inclusive research talked about trust or distrust. And then later, the conversation on de-escalation training also talked about trust or lack of trust with researchers. It almost like you could've swapped out the words and the meaning is the same. You know this better because until a year ago, your whole professional life has been in the academic environment. This isn't trained. It's not rewarded.

ALEX PIQUERO: No.

NANCY LA VIGNE: You're not incentivized. You don't have the time to do it. Here I am, Director of NIJ, trying to use this platform to do what I can to transform the field into the way I think we should be doing research if we really want to be making a difference. And I walk into this with the assumption that we're all in the space because we do, right? How do we do that when the academic environment is the way it is? I mean, I'm really concerned.

ALEX PIQUERO: Yeah, so am I. The reward structure doesn't necessarily need to be completely overhauled because research universities like to count. That's what they do. And that's what they're going to do. What some universities are doing and some things I started to put into place in different leadership positions was, if people want to engage with policymakers or write an op-ed — and we've done two of those, which is a really rewarding part, so far, of nine months working together is —they should at least be recognized, "Good job," because what universities in theory should do, private and public, is not just produce knowledge and not just disseminate knowledge, but use that knowledge to improve the world around them. Whether it's environment, whether it's food science, whether it's criminology, political science, like whatever it is. That's the job, is to not just create basic knowledge, but to use that knowledge to make the world a better place.

So, the more the fields start to change, I think the more we can say, "Yes, we need to recognize that," but not punish the people who don't want to do that. We have to use it to incentivize people and show them, "Hey, this is a good thing to do, and this is why it's good to do that." But that takes sea change, that takes generations. And fortunately, I've trained 20-some-odd Ph.D. students, tell them, "This is why you have to do this." No, "have to" is bad word. “This is why it's a good thing to do this.”

NANCY LA VIGNE: But it's not like a pat on the back is better than what happens now. We've had these conversations over the years when we go to ASC, to these roundtables around criminologists and the media that are attended by six people, we're always there.

ALEX PIQUERO: Yeah...

NANCY LA VIGNE: And...

DR. ALEX PIQUERO: ...two and three and four panelists.

NANCY LA VIGNE: Yeah. And it's like, "Well, that's not incentivized, nobody cares. A pat on the back is fine, but that's not going to get me tenure."

ALEX PIQUERO: That's right.

NANCY LA VIGNE: Right? So, I think that we have to blow up academia.

ALEX PIQUERO: Oh, well...

NANCY LA VIGNE: Just blow it up.

ALEX PIQUERO: I'm not willing to go to there yet but...

NANCY LA VIGNE: Create a new model.

ALEX PIQUERO: ...I'm close...

NANCY LA VIGNE: This is being recorded so, you know. He's a better politician than I am.

ALEX PIQUERO: She's working on it. No, your point is well taken, and I think that the way we were trained is very different from the way we should be training people now.

NANCY LA VIGNE: Yes.

ALEX PIQUERO: And it's not just training our own students. I think Nancy and I; we believe in people. People are what we recruit, but people are a legacy. People are policy. People are NIJ, people are BJS. And people are police departments and...

NANCY LA VIGNE: And our grad students...

ALEX PIQUERO: And our grad students.

NANCY LA VIGNE: ...that we're supporting.

ALEX PIQUERO: And kudos to her for putting the dollars into bringing you all together. But there's also work to be done. And you'll be graded. There's always a grade. We count --we count and letter, everything. But to do that, you have to get people to understand that we are all always learning, and we don't always know the answer when we start off our career. I didn't have the self-esteem to start doing policy-relevant work and talking to a mayor because I didn't have the credentials to do that early in my career. So, I had to build that up. I had to spend the time; demonstrate I do know something. Then, okay, well because people will do background on you and then you have to have some sense of working with them and not burning them, because the relationship is something that will last a long time.

My first job was at Temple University and Jack Greene was on the faculty of Temple. And one of the first meetings I ever did, Jack said, "Let's go to the Roundhouse." The Roundhouse is what the Philly PD is known as. So, we went there and Chief Timoney was the chief at Philadelphia PD at that time. And so, we went in there and it was months of talking and discussing about revamping their entire IAD system. To have a police chief say, "You can come in and help us understand our Internal Affairs Division and make suggestions for us." So, we put together a report and we met with them before we did anything with those results. That’s how you build relationships. And we don't teach people how to build relationships.

NANCY LA VIGNE: No. Well, if you're anyone from the School of Social Work?

We do teach people how to do that.

ALEX PIQUERO: In criminology, we don't necessarily teach people how to build relationships.

NANCY LA VIGNE: In criminology programs, we do not.

ALEX PIQUERO: I think that's something that over the course of our two careers is Nancy went one route, I went a different route, but we've always been preaching and focused on the same kinds of things, is how do we get the work into people's hands to make decisions better for all of us? And you don't always win. You don't always win in that, but that's the goal.

NANCY LA VIGNE: So, I’d like to pivot a little bit and ask you a tough question that I think would be hard for me to answer, so please forgive. I'm just thinking about you leading the Chief Statistical Agency for DOJ. You're actually the Chief Statistical Officer for all of DOJ, am I correct?

ALEX PIQUERO: That would be correct.

NANCY LA VIGNE: And there's always a tension in where you invest your resources and unfortunately our resources aren't what they could or should be. And we're hoping that will change, but that's outside of our control largely. So, you have these statistical series that are so important to keep doing because that longitudinal data is what tells us what's shifting and changing and why, right?

ALEX PIQUERO: Yeah.

NANCY LA VIGNE: As a consumer of BJS data myself, I'm like, "When is that next series posting?" Like, "Why is it taking so long?" I know you're fixing that. But then there's new inquiries.

And with the same amount of resources, how do you make those tough decisions about whether to launch a new series or add to the existing ones?

ALEX PIQUERO: Yeah. So, preserving some of the classic data collections that we do is critically important. We need to run the NCVS. We can't have a world where we don't run the crime victimization survey. We have to run the LEMAS survey. We have to do jails in Indian country. We have to do these things because we need that longitudinal thing, but we also need other things that we haven't done. And so fortunately BJS, before I got there, started to do, and plan out certain new data collections such as maternal health in prison, that's kind of important. We don't have, data on that at the scope and scale we have. My Principal Deputy Director, Dr. Kevin Scott is here in the room. Everybody's pointing at him. He's fantastic, even though he went to Ohio State.

NANCY LA VIGNE: Oh.

ALEX PIQUERO: One of the things that what I did randomly in a meeting, no one knew this, I said to my unit chiefs, "Come up with an idea of a new data collection that we're not doing right now that we should be doing." The statisticians should come up with ideas. So, we've actually started to put things into place. I can't really discuss right now because you know how that is. About being very responsive to new ideas and newer data collections that we can do much quicker than the usual process, because when we do an original data collection, it's not doing a SurveyMonkey. That isn’t how it works for us. We don't survey 300 people online in 10 minutes.

NANCY LA VIGNE: Your job would not be very impressive if that were the case.

ALEX PIQUERO: Yeah. And it's not a two-week IRB process either at BJS. It's a little bit time-consuming. We got to do a little bit of both. But one of the things that I think we're doing right now is, we are laying the foundation for things that we haven't done before that whenever I leave the office, people are going to go, "Thank you for doing that," because it's the kind of stuff I've always wanted BJS to do. And so, what's also happening now is that other parts of the government are coming to us to do co-projects that are...

NANCY LA VIGNE: So, you get their money to do the things to add to...

.

ALEX PIQUERO: They get a little bit of their money. I mean, there's only so much money. But they're cool things that you go, "Oh, we need to know the answer to that."

So that's the exciting part because you want to do new stuff, but you also want the team at BJS to create what that's going to be those new legacies going forward. But you do the same thing too. Right. You got to do some of the same things NIJ does, but then you have the ability to go, "You know what? I want to invest in this big new thing that you just dropped last week in Las Vegas."

NANCY LA VIGNE: Well, then that our Associate Attorney General dropped yesterday as well. So, two new things in case people are wondering. Last week we announced that we're releasing a new prize challenge. It's a new way of inviting innovation in the field, to welcome people to develop new methods to better measure community sentiment at a very micro geographic level that doesn't underrepresent people who reside in the most heavily policed high-crime, largely community of color street segments. Because people talk about community and it's like, community is an entire city, community is a census tract. It's just not precise enough for really understanding what people are experiencing when it comes to policing and whether changes in police practices or moving the needle.

Then the other exciting announcement yesterday was the partnership with BJA, where we released complimentary solicitations on the issues of correctional culture and climate, which is an issue near and dear to my heart. And that’s where BJA is supporting, inviting proposals to provide technical assistance to agencies that want to transform their correctional cultures and climates, and NIJ's companion pieces inviting research and evaluation on those topics, so yes. But those aren't very big, and I don't have enough money. And so, like what we discussed, I don't know how many times and how many different venues that might have involved a beverage.

We talk a lot. Our days or months are numbered. We serve at the pleasure of this current President. And we are always thinking about, "What are we leaving behind? What are our legacies?" And so as much as it's really fun, frankly, to be in a position to say, "This is where we need to invest in research," we were we've talked a lot about focusing on people because that's what lives on. I know this because I was at NIJ a long time ago and established this thing called the Crime Mapping Research Center and then I left. And then they changed the name and it got smaller and smaller, and then it went poof, and it's not there anymore. Now those investments live on, because the grants were awarded and the research was done, and we've built a lot upon that. But you look back and you say like, "Well, should I invest in programs? Should I create new centers?" I'm not so sure because a new political comes along and they're like, “Yeah, no, that was the old.” It's just like police chiefs, you see it all the time. But investing in people, now that makes a difference.

ALEX PIQUERO: Yeah, and that's what the reward is. Our former colleagues or students or coworkers ask us for advice. And our job, our best ability is our availability. And so mentoring is the most rewarding part of my academic career and certainly our professional careers because you're investing in people and then you're going to see their successes. Then they're going to do the same thing to their students and their colleagues.

And the team I've been able to fortunately put in — well, they said yes to come to BJS, which is good. But the team is such that the values they have will transcend whoever is in that position.

So, it's not about a particular data collection, it's about people who have the right values about what a federal statistical agency should be doing, what those safeguards are about and on the importance of getting information out to people as quickly as we possibly can. But as Rob Sampson said yesterday, "It's got to be right." That's what BJS prides itself on is that there's a lot of people in the crime data space. There's only one official federal statistical agency in charge of that.

NANCY LA VIGNE: And you need to be that definitive source.

ALEX PIQUERO: Yeah. And the people who I'm recruiting, the people who want to come to BJS, they have the same vision about what the role of that organization is.

Not about a data set, not about a topic, but it's the vision.

NANCY LA VIGNE: And similar at NIJ. We've had the benefit of being able to hire some new people. Some of them are in this room, who are embracing this evidence-to-action mindset. But I want to say too that a lot of existing NIJ staff are really excited and on the bandwagon. So, it's not like you need outside people to help transform an organization. And we're also investing heavily through grants programs that I have that you don't. Sorry....

ALEX PIQUERO: Sorry, not sorry.

NANCY LA VIGNE: Our graduate research fellowship program that we brought back this year after it was not offered last year. How many graduate research fellowships fellows do we have here? A couple? Thank you. That's a really important program. We have our research assistantship program. We usually bring on two or three people a year. They're doctoral students that work part-time with us. And it's a great way to be exposed to NIJ and what we do. In case any of you are contemplating a non-academic career. And we're hiring. Both of our agencies are hiring. So, there's a lot of opportunities there as well.

ALEX PIQUERO: Yeah. I think what's really cool about what Nancy's doing is stuff that we actually started putting in the pipeline is mentoring that next generation of people. Not necessarily to not make them go into academia.

NANCY LA VIGNE: But it's okay if they don't.

ALEX PIQUERO: Or not. But we’re really in the position right now and we're recruiting more than a dozen people in BJS. When I got to BJS in August of 2022, I met with every single person, whether in person or virtually. I said, "What do you want to do in your career? And how can I help you get to that point? Whether you stay at BJS or you go to a police department, you go to NIJ, whatever you do in your career, how do we set you up in a position to succeed in that direction?" We have a program that BJS started last year called Digital Fellows. Basically, they work with a BJS statistician, and they produce a report. One of those reports not only was published, but it's also one of the coolest data visualizations we have. This is a statistician and then a student. And the visualization is cool because what you can do is you actually go to the county level in your state and say, "How many victims per 1,000 people are there and how many victim service providers are there in that same county?" Why that's cool is you can see where those are concordant and more importantly, where they're not concordant. That's a great thing because we're exposing new people to BJS, they're learning about the tools and the data we have, and they're producing something that's going to live forever. Forever. And that's cool for them, and it's great for us.

NANCY LA VIGNE: Yeah. It's a win-win for sure.

ALEX PIQUERO: And it's a win-win for sure. So that's part of the job.

NANCY LA VIGNE: So, we promised opportunity for Q and A, and so I wanted to leave some time for that. And I think we have someone with a wandering mic or two. And we agreed we'd want to prioritize questions from our students in the first couple rows. I think we have one right here.

.

ALEX PIQUERO: So, if you introduce yourself, your name and where you're at currently.

MONICA: Hi, my name is Monica. I'm a Ph.D. student in the Chemistry Department at the University of Albany in Musah Lab. You were talking about some of the issues with academia and I think one of the main things that we see is that we put in all this effort to do this research but there's not a lot of collaboration or it can be very difficult to do so. And throughout my years in this program, I've developed a relationship with the Director of the New York State Crime Lab, Dr. Wickenheiser and he's all onboard for us doing collaborations. However, the main issue is that they have so many backlogs that the crime analysts don't even have the time to work with us. And that's an issue. There're security reasons where it may take a really long time to even get access to the labs or even not at all.

Another issue is that we have all the instrumentation and the equipment in our lab, and they can't just bring evidence over or we can't just bring all of our equipment over there.

Another issue with academia, even if you have people who are willing to collaborate with you, even though they can't, there are some agencies that I've had colleagues who have told me that they've done research specifically for a certain agency. And the agency just didn't care, and they did their own in-house research instead, because they didn't want to use outside research. So I think that's another issue and so I was wondering what could be done to circumvent this, if maybe there could be some sort of program that allows graduate students to do research at even just a state crime lab or if there's some sort of program that could be started there where you don't have the crime analysts that are working with the evidence or maybe there's like another section where these people can work with us.

NANCY LA VIGNE: I think I get the gist. Let me just say, I know nothing about crime labs or working in them. So, my answer isn't going to be very specific. It's more based on just experience and observation. I believe that good mentors always make the time, and that I don't care how busy they are, they always make the time. I consider myself an example of that. I always make the time. So, there are people out there who can be mentors to you.

You stated a lot of barriers that I really don't have the expertise to be like, "Well, just fix this and then the other," but I do recognize that crime labs are overburdened, and I think maybe this is something that BJA can help with because they are more working in training in TTA and getting resources to state and local labs to alleviate that burden. But I don't know that that all of a sudden open the doors to collaboration. I think you sat in on —the couple different conversations about trust-building. And that example you gave about, we did the research, but they didn't want that research. Well, I don't know the scenario but maybe you should talk to them before you do the research to see if they want it and why they don't want it and make it more collaborative from the beginning.

.

ALEX PIQUERO: Sometimes when you invest working with agencies, you're going to do stuff for them just to do it for them, without a product for you. But that's part of the relationship-building. And if they do anything with that, you can't control that. But they don't have the skillset that you might have. So that's important for them. Whether they may not use it today, could use it a year from now, or a new chief could come in, "Oh, I didn't know we had this. Oh, how great." So, you never know when those things will come to life.

KIM WILLIAMS: I'm Kim Williams. I'm a Researcher at Evident Change and recent graduate of Georgia State University.

NANCY LA VIGNE: Congratulations.

KIM WILLIAMS: Thank you. I've heard a lot of talk about participatory research. So, my question kind of surrounds that. With that initiative, how will participatory research be reflective in the RFP and the peer review process? Because that is currently more supportive of traditional research and as well as the individuals who are doing the review, or because we tend to use more seasoned researchers for that process.

NANCY LA VIGNE: Who are trained more traditionally. Yes.

KIM WILLIAMS: Yes.

NANCY LA VIGNE: I hear you. Just to be clear and I do think language is important, I'm not advocating for participatory research. I'm not. But I am talking about inclusive research because I want to bring the entire field along. The way I describe inclusive research is that it exists on a continuum. On the far end is full-on community-based participatory research. But I don't want to let the people who are doing largely quantitative research off the hook for also being intentional about how they include the people who are closest to the issue or a problem that's being studied at the beginning, middle, and end, even if it is largely quantitative. I think that's important to clarify, because I think there's a lot of different research questions and different methods and teams are appropriate to answer different questions at different times. —

But I do think NIJ could and should support more community-based participatory research. When I was a grantee on that side or an applicant side of it, I used to call the "boilerplate" of all of our solicitations to reflect the priorities that I put forth as NIJ Director, chief among them inclusive research. You will see language already in everything for FY23 and most of it is on the street already that says priority will be given to proposals that embrace an inclusive research frame. Proposals that take a racial equity lens, which is another thing I talk about. Proposals that demonstrate interdisciplinary research teams. And proposals that are very intentional and dedicating strategies and resources towards dissemination in the interest of evidence-to-action. We also have language in our solicitations that prioritize proposals that actively and meaningfully partner with community-based organizations and folks that have lived experience on the topic, that is the nature of the proposal, to give those proposals a leg up. As long as that's a meaningful share of the budget and not just some kind of box-checking exercise where the applicant gets a letter of support and thinks that that's going to give them a leg up.

CAMPBELL CLARK: Hi, my name is Campbell Clark. I'm a graduate student from Oklahoma State University. First off, I'd like to just thank the NIJ and everyone who's come, just granting us ability as students to come here, as well as the excellent staff and generous attendees who I managed to connect with. I'd like to kind of hone in on like the specific audience in which you're publishing towards as well as the intended effects of hopefully doing good with research, which was mentioned. So how is it that as a researcher one can adapt to gain the trust of non-receptive audiences? Considering the modern challenges you have like biases, you have increasingly shorter attention spans of certain audiences, you have disinformation, misinformation, as well as political and ideological polarity.

ALEX PIQUERO: It's yes to everything. And here's why it's yes to everything. One of the things that's been very important to me in my academic career that I've morphed into BJS is the ability to get out and engage with stakeholders. And that's in our statutory language, but I define stakeholders as anybody. So that would include the media. That would include social media. That would include academic audiences. That would include practitioner audiences. We've given talks to practitioner audiences, to professional organizations. And I say this all the time, I'll play to a bar of five people or a stadium of fifty thousand people.

NANCY LA VIGNE: Is that the Beyonce reference?

ALEX PIQUERO: Yeah. Yes. Some of you who saw that yesterday.

NANCY LA VIGNE: Couldn't resist.

ALEX PIQUERO: Anytime I have the ability to talk about the people at BJS and the work we do to anybody, because that's five more people or 10 more people who will learn about the work. But what I also did at BJS is I realized that we need to be where people are at, because that's where the world is at. So, immediately when I got there, I wanted to create a new series where it was designed, "designed," there's a reason why I said designed, to be a one page. And you see a title, and you see a figure, and you know the answer to the question literally in 10 seconds. We had a naming contest, because I think it should be about the people who come up with a name. We're statisticians. we came with up a name. They started weighting names and the standard deviation. We came up with a name. It's called Just the Stats, and forcedly of course. It wasn't copyrighted. And the first one we produced was on carjacking. And it was fortuitous that it was on carjacking because in February ‘22, Senator Durbin wrote a letter to various federal agencies. He said, "Why don't we have data on carjacking?" Well, the FBI doesn't produce data on carjacking, but the NCVS has a way to do that. So, boom, there we were responsive, and we had over 2,000 people immediately sign up for this new package of how we deliver the stuff. And we now have done more of those and they're just so poignant because it hits a different audience. It's a specific question on a timely thing in a way for people to understand.

NANCY LA VIGNE: Can I answer your question in a different way because what I heard was how do I as someone who isn't academic and producing round research, how do I get that research into the hands of people who might be skeptical or not be patient, and your answer was accurate, but I'm just thinking from your perspective. And again, I think this relays the fact that I never thought I was going to go into a Ph.D. program, so I was trained in public policy, and one of the things they teach you in policy school is how to write a decision memo. It's like the bottom line first and then it's just really succinct. So I took that training with me to my doctoral studies and beyond, and then at the Urban Institute, I had a lot of time to think about how I've effectively communicate, sometimes very complex research projects, and essentially what we landed with is what I do call it a pyramid approach.

In the base of pyramid is write the final technical report that you deliver to NIJ. And then one up is your journal articles or series of journal articles. So, who are those reaching? More researchers, right? It's really important to think about the exact summary in that report and how can you bullet out key findings. Then the next level up is the abstract, because sometimes people don't read anything but the abstract. And we spend a lot of time rewriting abstracts because a lot of people pulled the abstract from their research proposal, even at the end where they had findings, the abstract was still all about the methods and what they were going to do. We had to rewrite all the abstracts and talk about the research question why it was important and what was found. Then at the very top of the pyramid are things like postings on LinkedIn and tweets. And all of that is important if you really want to reach these different audiences.

Now, how you get over the divisiveness and the skeptics? I don't know. I think this conference is a really interesting one because we've brought together a lot of different people with a lot of different perspectives. We have a panel on policing, and we have police abolitionists and everyone in between. I kind of feel like people are being respectful and listening to each other, maybe they're all going to different panels, I don't know. But I think we need more of that and more understanding. One of the reasons we are researchers is because we believe in the truth. Sometimes I have research findings that surprise me, but I keep an open mind.

ALEX PIQUERO: I’ll add one more thing to that now that you’ve interpreted the question the right way. It's pretty common in this work relationship. So, I do a lot of op-eds, and so that, again, that reaches a different audience. The universities that I've always worked at had done press releases. That reaches a lot of big audiences. Sometimes you have to be very pushy and get your stuff out there. I don't have a problem with being pushy, so I'll talk to anybody about the work I do. What you do is sometimes you establish a good relationship with reporters. This isn't about doing favors, but it's about building relationships. Sometimes they'll call on background. You'll spend two hours; they won’t quote you. But you've built a relationship where they're seeking information from you. That's an investment in something. But then you say to them, "Hey, I have a new study coming out, you might be interested in this." And that helps.

So, I had a piece come out in New York Times on a study I did about Juvenile Fines that hit above the fold New York Times and it changed policy in lots of counties. When one of those hits, then it's a cascading effect. But you can't control when your study comes out or when the news release comes out because there's a lot of other things going on. So as many different audiences and as many different ways as you can humanly get that stuff out there because you never know who's paying attention, and you just don't know.

NANCY LA VIGNE: We have a lot of questions.

ANTHONY AZARI: Yeah. I have a really quick question. My name is Anthony Azari from the School of Criminal Justice in Rutgers, Newark.

NANCY LA VIGNE: Rutgers. Woo-hoo.

ANTHONY AZARI: My question is kind of broad and has already been touched on and I've seen the hands of social workers in the audience. That's reassuring at a criminal justice conference, and same thing with chemistry, and so forth. My question centers around this aspect of the constant change that is occurring within the field of criminal justice. We're seeing things like "defund the police." We see advocates like Angela Davis out there, making her work known on how we need to reform the entirety of a system. I was just wondering if you had any perspectives as agency heads yourself as to how should we approach this as new researchers in the field where someone might say’ “You're conducting this evaluation on this police department and, yeah, our group over here, has this completely different perspective on what they're doing,” or something like that. So, any thoughts on how to approach this push within the space. And it's not just that, it's everything within criminal justice, from court reform to back-end sentencing policies. I mean, it's the whole nine yards.

NANCY LA VIGNE: That's a really good question. It's one I wrestle with a lot. As much as I like to be forward-looking, I'm old. I've been around for a while. I don't want to say a more traditional mindset, but one where I've seen how change occurs over time, and a lot of it is incremental. But that doesn't mean that we need to be incremental. I think that the field is an expansive space where everyone can find their own place. And I think that there's a role for people who are pushing on the margins, and there's a role for people who want to make change from within agencies. What I'd like to see is that everyone respect those choices rather than judge them.

ALEX PIQUERO: I still read as much as I possibly can read right now. I'm always learning. It's one more thing to read because you got to expand your knowledge base. And I've always believed that the creation of knowledge is putting together a piece of the jigsaw puzzle and you're building out over time. When I got to the University of Miami three years ago, one of the first things I did, this was during the beginning of COVID, is I contacted the prosecutor's office, the public defender, the police chiefs, and other CJ actors in Miami. I said, "I just want to meet you and see what you all are doing, and what you need to do." Because I wasn't picking sides. I'm trying to help everybody create something better. It's not like updating your iPhone software tonight. Well, sometimes it takes a long time. It takes time, it takes investments. But in some cities, when you get those people — they're not always on different sides, they have different roles. I think they're all focused on the same thing. How do we improve public safety when you're weighing decisions of people, and communities, and we're going to make errors. We're going to make errors. We're human beings. We're making predictions about people. And that's the thing that we focus on, but we also are right a lot of times. We got to remember that we are right a lot of times. But I do believe that at least in my experience working with police chiefs, and public defenders, and state attorneys, they really are committed to try to do the best job they can with limited information, and oftentimes, like that [snaps fingers]. So, I'm hopeful.

NANCY LA VIGNE: Other questions?

FRANKIE: Hi. My name is Frankie and I’m a Ph.D. student at Forensic Genetics in Indiana University in the lab of Dr. Susan Walsh. I have a question about your opinion on law enforcement using outside companies and the data that is based on science but not peer-reviewed, because I find that very detrimental to the research community. It becomes like our research isn't as important because law enforcement is already using it, however it's not peer reviewed. I feel that that also puts in a lot of bias if research isn't peer-reviewed, which I think it's detrimental to law enforcement because I feel like that's also kind of the topic of this conference. I was just wondering what your opinion would be on law enforcement doing that.

NANCY LA VIGNE: I'm not sure I know the specific reference, but I will say that I am a firm believer in peer review. That said, I think that we often go to the same people as peer reviewers and we need to diversify our pool of peer reviewers, so that we're really bringing in different perspectives while ensuring the rigor. While I'm at it, because we've talked about the rigor tension too. We're aware that a lot of people who are entering this field as new doctoral students are entering it because they have a passion for change, and they identify with the advocacy community, which is fine. But we have to make sure that the rigor is not compromised in the process. What it is to be an academic researcher and explore questions that you care deeply about the outcomes for? And what happens, how do we ensure that we are being objective while also being inclusive? These are real tensions that I think that your generation are going to need to wrestle with. I don't have the answers, but I want to name that because I know it's a big issue.

ALEX PIQUERO: You can't always control where people are going to get their information. We've seen a lot of that as your colleague in front of you said that a little while ago. But I liken crime and justice to public health and all of us in this room at one point of our lives, if not by now, currently, or in the future, will take some sort of medication. Probably based on peer-reviewed science of some capacity. There's a reason why we rely on peer review research to give us something which is better than nothing. So, like Nancy, I will hang my hat on the peer review.

LESTER KERN: My name is Lester Kern. I'm a Ph.D. student at the University of Chicago School of Social Work. First off, thank you all for having us. But my question, I do research on mental health emergencies and responses. In my work with patrol officers, emergency departments, and advocacy groups, there seems to be this prevailing desire for officers [INDISTINCT] to cover their butts, specifically the cases that are vaguely mental health related or occurring at an address that they associate with somebody that has a mental health issue. So much so that the officers would arrange for involuntary transports, so it's kind of coerced involuntary transport, that on the surface looks voluntary and it looks great. But once you dig into it, it has some more disturbing implications. That’s a lot of effects. You are maybe manipulating or violating rights of somebody, placing unnecessary strain on partners. Now even more so than that, but relevant to this question, it clouds the data that we're looking at when we start thinking about outcomes and processes that we want to see.

The cases that I'm thinking of, they're interesting and they have some local novelty but they're not necessarily unique. Thinking of how that kind of misrepresenting of processes and outcomes can frequently happen and scale up and then show up in the data that undergirds a lot of our assumptions and initiatives, should we blow up our confidence in the data or even blow up what we consider be trustworthy data? And then how can we avoid having situations where the big date initiatives that we buy into seem too big to fail even if they're based on inaccurate data?

NANCY LA VIGNE: Wow, that's a beautifully crafted question, especially the last sentence around the big data. I think that we, in this field, take a lot of data for granted. Maybe not the data produced by the Bureau of Justice Statistics, but we believe on its face that it's measuring what it's measuring and that's just not true. So many articles are published around data sets where we're not interrogating the data. And honestly, I've done it myself. I'm still learning. And I think in this topic around police response to issues of people experiencing crises or mental health related calls, we don't really even know the nature of those calls because of how they're coming in and being received by the call taker. And the biases the call-taker might have in what they're hearing, what they think they're hearing and do they make it a priority 2, when it's really a priority 4, and then the officer responds and thinks that it's more of a high-threat situation than it is and that leads the officer to do things differently. Then is that data ever corrected?

This is how nuanced it is, right? The answer? I think we need to do more work to see what those data sources are and how valid they are. Who's collecting? Who's coding? I would love to see more of research where people are on the ground and doing observational studies. There were so many observational studies of policing decades ago, and I think we need to bring them back so that we can really see what's happening on the ground.

ALEX PIQUERO: I think it's a great question. Having spent time with police departments and seeing the calls for service dispatch occur, so someone calls the dispatcher, the dispatcher tries to get as much information as they possibly can from the other human calling them, so there is a big who-knows-what-they're-going-to-get. Then the dispatcher calls the police and then it goes from there. There's a lot of missing data that gets even to the whatever the officer gets, and then when the officer shows up, who knows what has transpired in that period of time. One problem we have, and I think you've nailed this right on, is we don't have a really good understanding of calls per service data in the United States. And we are exploring actually how to do that in BJS right now because it's not real simple, because we have 18,000 police departments and they have very different ways of collecting data.

And in fact, fortuitously — I’m glad they pointed this out — I have the unique honor to be a co-chair of a working group in the federal government, with the Office of Science and Technology Policy and the Domestic Policy Council. Nancy was on this as well, as well as Dr. Shelley Hyland and Dr. Kevin Scott, who are part of this team, and we just produced reports to the President of the United States. Never thought that that would happen in my life. About the nature of police statistics in the United States, what does it look like? What's needed?

NANCY LA VIGNE: With an equity lens, right?

ALEX PIQUERO: With a very much equity lens in terms of disaggregated data.

NANCY LA VIGNE: And public access to the...

ALEX PIQUERO: And public access to data and it’s transparent for people to see and to use, that's easy for them to download and to look at and to understand. With a lot of policing data in the United States, because of the lack of mandatory data reporting, we don't have a lot of information on race, ethnicity of arrests or calls per service. In a lot of jurisdictions, they don't collect data on Hispanics, so you have a huge data missingness in the United States. So, what we've tried to do is say, "Here's what the landscape looks like. Here's what's needed. Here's how you should produce it, and here's why it matters." So, the "why it matters" is to have more equitable outcomes. Then you work backwards, to get those outcomes. What kind of data do you need? How do you we have to collect those data? And we live in the world of thousands of different data collection systems, and we need to rein that in to create the kinds of outcomes we deserve to have.

NANCY LA VIGNE: Our time is up. Did we leave any questions unanswered? We're going to take two and then we're going to answer both.

BALAJI SUNDARAM: My name is Balaji Sundaram and I'm from University of California, Santa Cruz. It seems like I'm a STEM minority here. There is a lot of good talks on social science and criminal justice. Because I'm working on DNA sequencing, I must ask this question. I was funded by the GRF, so thank you for the funding. When we wrote the grant three years ago, we were a little bit hesitant to even use the words “genetic genealogy” and “public data base search of DNA profiles.” But now I think the dust has settled and it seems like genetic genealogy and searching the public database seems accepted as a norm. But it's still lacking. It's still not admissible evidence. So, I'm actually wondering what is needed to move the needle of taking the genetic genealogy as primary evidence or admissible evidence and how we can push into an action?

NANCY LA VIGNE: Okay.

BALAJI SUNDARAM: My second part is I see lot of private companies working on and actually making the secondary evidence for the law enforcement agencies so I'm wondering what the NIJ sees as a future? Are there any plans to enable the law enforcement agencies, like the government agencies, to do their technologies by themselves? Or do we still see that private entities are going to play a major role going forward?

ANNA NEWELL: My question's pretty different but it is kind of similar to Frankie. To Frankie's question. My name's Anna Newell. —I just finished my first year at Penn State in the Criminology Ph.D. program. I think anecdotally among academics, there is concern that research that comes from governmental groups or NGOs or think tanks can be a bit biased. Or when reports and recommendations are made for policy, there is cherry-picking of research. I think it's a tough issue because when we talk about research-informed policy, research can produce mixed findings. It can also change over time. The same study conducted 10 years apart could be totally different findings. So, my question is, how can we connect academics and government groups and government agents in a way that can promote, selecting the highest quality research to consider in developing research-informed policy?

And for the record, I do think events like this are very important for that. I'm very thankful for the opportunity to present a poster here in a less traditional academic setting. But I'm curious what other steps you think can be taken both by academics and government agents.

NANCY LA VIGNE: (answering question from Balaji Sundaram) So absolutely not my area, so I don't want to make up an answer. The answer I can give you though NIJ used to have a whole robust funding stream for our forensic investigative sciences research, and that went away in 2018. And now we have to beg, and borrow, and steal to get money from another agency to support that work. I'm hoping for a dedicated line item for that research so that we can explore a lot of those issues that you've raised. We'd really love to support more research in that area. I'm sorry, I can't say more. And also, I've been talking to our forensic and investigative sciences staff about how the few offerings that we have at this conference that were from that office are not enough, and that when we do this conference again, we want to really have a more robust representation.

Now, how do we get the quality right? You talked issues of quality and how we get people better understanding of it.

I don't know if you know Crimesolutions.gov. Do you know CrimeSolutions? That is the thing that was developed with the best ever intentions about applying a very rigorous lens to all research no matter whether it gets published in an academic journal or through a think tank, or it's a government publication. It's assessed to meet a certain level of rigor. And then if it meets that threshold, it's then assessed for whatever the thing that was evaluated led to its intended outcomes. And it's this robust treasure trove that needs to be overhauled in meaningful ways in my opinion and it's something that we'll be doing at NIJ. Because I don't feel like it's known. I don't feel like it's really used. And in a way, maybe it's somehow part of the presentation. Maybe it's something a little bit about some of the criteria employed. But we are going to be launching an effort to explore all the different ways that we can make that a better resource for the field because I think that addresses a lot of your questions. And then in the interest of evidence-to-action, we get that where we needed to be but how are we getting it into the hands of the people who can use it to inform their decisions about what programs to invest in, what policies to change.

ALEX PIQUERO: Your question, I would say, I still struggle with now. I read GAO. I read reports from Vera. I read reports from academics., I read working papers. And our job is to synthesize all of the knowledge. And there's no scorecard that says, "Okay. You can only count four studies from here and three studies from here." Our job is to go to the mountain, read, continue to read, and synthesize, and...

NANCY LA VIGNE: But you have to assess if it's not well-documented and they can't show the rigor, then it's like, “Hmm, maybe you don't mention it or it's a footnote—” right?

ALEX PIQUERO: And it's not always one preferred methodology is the gold standard. That we hear about that a lot and obviously some methods might produce more reliable estimates or a high power, or lower power. And qualitative research can't be compared to RCTs easily enough. So, you have to put all that together, hope that the producer of the document provided the necessary level of detail to say, "How did you code this study and in a meta-analysis? How did you collect this sample? What did you do for non-responses?" All of those details sometimes aren't in the articles because they're whacked or they're on an online website. That's when the consumer sometimes is like, "Oh, my gosh. Am I getting everything we can get?"

If you don't know, you have to go get the information. But amass, synthesize, and always think and ask more questions.