Webinar Transcript: NIJ FY 24 Research and Evaluation on the Impact of Technologies for Forensic Science Applications

This webinar provided information about the "FY 24 Research and Evaluation on the Impact of Technologies for Forensic Science Applications" solicitation. NIJ seeks applications for funding for proposals to study how adopted technologies with a forensic science application impact criminal justice systems, how the implementation of forensic laboratory programs or practices broadly effect outcomes, and how changes in policies adapt the use of forensic evidence. NIJ seeks rigorous evaluations of the implementation of technologies, methods, or workflow processes to understand the impact on backlog reduction, capacity enhancement, cost relative to benefit, workforce development, or case closure efficiency.

Transcript

STACY LEE: Good afternoon, everyone, and thanks for joining us today for the Fiscal Year 2024 Research and Evaluation on the Impact of Technologies for Forensic Science Applications Solicitation Webinar. It is my pleasure to introduce Danielle McLeod-Henning, a Physical Scientist with the National Institute of Justice.

DANIELLE MCLEOD-HENNING: Thank you, Stacy. Hello. And thank you all for attending today's webinar on the FY24 Research and Evaluation on the Impact of Technologies for Forensic Science Applications. As Stacy mentioned, my name is Danielle McLeod-Henning and I am a Physical Scientist and Program Manager with NIJ's Office of Investigative and Forensic Sciences. And now I would like to turn the mic over to NIJ Director, Dr. Nancy La Vigne, for opening remarks.

DR. NANCY LA VIGNE: Thank you so much, Danielle. And welcome, everybody. I'm delighted to be here to help kick off this webinar on Research and Evaluation on the Impact of Technologies for Forensic Science Applications. I am the Director of the National Institute of Justice. I've been in this position for coming up on two years - it will be two years in May. There's so much that I love about this job. And one of the many great things in this role is working with team members here at NIJ. I think one of the most exciting aspects of the work we do is identifying where the needs and gaps are in the field and how we can fill them through solicitations (requests for research proposals). 

This particular topic is one I care deeply about, not just because I've been having such a fun time learning about NIJ’s Forensic Sciences portfolio since my arrival here. I am not a forensic scientist by training. I'm a social scientist. So it's been fun digging into all aspects of forensic sciences and learning from the wonderful experts we have here. But it's also the case that I really love seeing interdisciplinary research and funding connections across disciplines and across topics in the criminal justice realm. 

One gap that I've noticed is one that NIJ used to fill quite robustly, and I know this because I used to work at NIJ years ago - many years ago. Back then, we funded a lot of research that looked at technology implementation, including implementation of forensic science applications. Since then, much has changed. Our budget has shrunk in forensics, unfortunately. We keep fighting to get increases each year. Our staff has diminished as well. Along the way, we've become a little siloed. It happens in a lot of different organizations. We've had to make tough choices about what research to invest in. 

So we've been investing a lot in the development of new forensic science methodologies and technologies, but what was lost after all these years was funding research and evaluation on the implementation of new technologies and methodologies, or even existing ones used in new ways, to help discern how, in a real world field setting, agencies and entities are experiencing those changes in practices, new methods, new tools - what's working well and what could be improved, and, importantly, are they having the intended impact. So that's what this webinar is all about and that's what the solicitation is all about. We're looking at all kinds of different types of interventions or applications and looking at how they might influence a lot of different types of outcomes, and the outcomes are going to be different depending on what the application is. It could be improving efficiency, addressing backlogs and enhancing capacity, improving accuracy. There's a lot of different outcomes. So that's what we'd like to share more about with you today. 

But before I pass it back to my colleagues, I did want to just emphasize a little bit some overarching themes of the priorities that we baked into NIJ's solicitations for research this year. They really track with priorities that were brought to NIJ when I came on board as director. The first is that we really want to encourage research proposals that are inclusive in nature. What I mean by that is that it includes some component of the research or the research process that engages with the people who are closest to the issue or problem under study. In the case of forensics, this would be people who are perhaps in public crime labs or otherwise consumers of forensic science in the real world. They could also include people who, for example, may have been wrongfully convicted. I'm kind of making this up as I go, but the point here is that, at a minimum, we should be bringing findings back to the people who are closest to the issue. In this context, bringing findings back, if we're evaluating an application, would also mean back to the people who are part of that evaluation, the people whose job it is to use the tool or methodology or technology. 

The second priority, and it's closely related to inclusive research, is that research should be approached through a racial equity lens, or an equity lens, writ large. There's a lot of different ways we can think about equity, but this is to encourage that whatever study methodology we use or we support, we consider biases - biases that are infused into the data and methodologies that are employed and the outcomes that may end up unintentionally yielding racially-disparate impacts, or maybe disparate impacts based on gender and so forth. So it's important that proposals take time to explain about how they're going to anticipate and try to mitigate these types of biases. 

I talked a little bit about the types of evaluations we're seeking in this particular solicitation. One thing that we always emphasize is that we're not looking solely for impact evaluations. So we don't want to see proposals that only look at whether or not a certain thing, say, increased lab efficiency, is effective. That's fine and good, and we definitely want to see outcomes like that, but we also want to see what's called a process or an implementation evaluation. That's critical because we need to know whether staff are trained appropriately to use the new tool or methodology or technology, that they're following the procedures that will ensure that it's used properly, to ensure that the right people know about the processes. All this important information around implementation fidelity needs to be documented along the way. 

And finally, I already mentioned my love of interdisciplinary research teams, and I certainly feel like those are what's called for in this particular solicitation, because no one person has expertise (well, maybe somebody does—we’ll call them a unicorn) in forensic science as well as in the social science evaluation realm. It’s a pretty unique combination. So we encourage you to reach out and identify partners that fill all the needs of creating a winning proposal and a robust and impactful evaluation. 

So with that, I'm going to turn it back to the dynamic Danielle duo. I couldn't resist naming them that. We've got two Danielles. I'm sure they'll share that they come from different two sides of the research house here at NIJ. They'll walk you through this solicitation in more detail. And before I do that, I do want to thank Stacy Lee, our host, for all she's done, along with her colleagues, to plan and execute this webinar. And of course, thanks to all of our attendees for your interest in this program. We look forward to seeing your applications. Thank you.

DANIELLE MCLEOD-HENNING: Thank you so much, Nancy. Before we proceed into the body of the webinar, I would like to first introduce my colleague and co-presenter today. Danielle of the dynamic duo Danielles.

DANIELLE CRIMMINS: Yes. Hello. Thank you again all for joining us today. My name is also Danielle and I am a Social Science Research Analyst in the Office of Criminal Justice Systems here at NIJ.

DANIELLE MCLEOD-HENNING: Thank you so much. So without further ado, let's go ahead and get into the meat and potatoes of this particular webinar, which is on NIJ's funding opportunity in Research and Evaluation on the Impact of Technologies for Forensic Science Applications. 

So in today's webinar, we already received our opening remarks from Dr. Nancy La Vigne. We are going to go into some specific information on this solicitation. The solicitation's goals and objectives, priority areas, and address the application submission, and application review process. 

NIJ, under this funding opportunity, is seeking applications to study how adopted technologies with a forensic science application impact criminal justice systems, how the implementation of forensic laboratory programs or practices broadly affect outcomes, and how changes in policies adapt the use of forensic evidence. 

NIJ is interested in research and evaluation projects, analyzing the implementation of technologies, methods, or workflows across the forensic science enterprise. Under this funding opportunity, we're defining the forensic science enterprise to include crime laboratories, medical examiner/coroner offices, law enforcement agencies, officers of the court, and other entities that collect, process, or consume forensic evidence. In looking at technologies, workflows, or methodologies, we want to better understand the impact on forensic evidence backlog reduction; capacity enhancement in forensic evidence analysis and interpretation; cost effectiveness or cost-benefit analysis of the implementation of the technology, method, or workflow; workforce development regarding education, training, and retention; and/or case closure efficiency. Proposals submitted to this funding opportunity should examine the impact of technological advances on the criminal justice system within the forensic science enterprise that will increase the body of knowledge to guide and inform forensic science policy and practice. Applicants should describe how they propose to evaluate the technology, method, or workflow, including how the evaluation will address implementation and outcomes. Although evaluations may be single-site, NIJ encourages applicants to partner with other sites to simultaneously and robustly evaluate the technology, method, or workflow at each site. 

Now we'll move on to what will not be considered or not be reviewed in application submissions. The following are not within the scope of this solicitation and will not be reviewed: proposals that do not contain a research component; proposals that do not respond to the specific goals of this solicitation; proposals that do not clearly address criminal justice concerns in the United States; applications primarily to purchase equipment, materials, or supplies; proposals primarily to provide training; proposals that provide direct services; or proposals to evaluate technologies, methods, or workflows that are not within the scope of the forensic science enterprise. 

 This solicitation funding opportunity has the following goals and objectives. The goal of this solicitation is to generate evidence-based knowledge on the implementation of technology, methods, or workflow processes within the forensic enterprise and how those technologies impact criminal justice systems. The objective of this solicitation is to evaluate technologies, methods, or workflow processes within the forensic science enterprise to measure the impact on backlog reduction, capacity enhancement, cost-effectiveness, workforce development, or case closure efficiency. 

Some of the priority areas that you may notice within the solicitation that NIJ may consider in appropriate circumstances, and give special consideration in award decisions, are for “new investigators” and applications from minority-serving institutions. And note that these are two of many factors that NIJ considers in making funding decisions. And now I will turn it over to my colleague, Danielle Crimmins.

DANIELLE CRIMMINS: Thank you, Danielle. I am now going to go through the application submission process and talk a little bit about the review process. So as described in the solicitation, on page 18, there are several application elements that should be submitted and there are a few critical elements that must be submitted in order to have the application move forward to the next round of review. The critical elements include a Program Narrative, a Budget Web-Based Form, this includes budget details and also provides an opportunity for you to provide a narrative about the budget that explains the details. Also, a CV, resumes, and/or biosketches for senior or key personnel within the application. There are also two required forms which are required and are standard for federal assistance. This includes SF-424, which is an Application for Federal Assistance, and then the Disclosure of Lobbying Activities, the SF-LLL. 

As described in the "How to Apply" section on page 15 of this solicitation, applying  is a two-step process. The first process is registering with Grants.gov and the deadline for the Grants.gov application is April 29th. And in this part of the application, you need to submit the SF-424 and also the SF-LLL only. 

The next step is submitting the full application through JustGrants, and the deadline for this is May 6th, 2024. We have a great resource page for applicants and awardees. There's a link on the screen, as well as a screenshot of what this page looks like, and you can also find a link to this page on the announcement page on NIJ's website and also within the solicitation. This is where you can read in detail regarding NIJ's requirements for Human Subject Protection, privacy and more. 

Now I'm going to talk a bit about the application review process. Once your applications are submitted, we begin the review. First, we determine whether the application meets Basic Minimum Requirements, or BMR. During this phase, we make sure the Program Narrative, Budget, and CVs have been submitted. We also determine if the application is responsive to the solicitation. 

Next, all applications that meet BMR and are responsive to the solicitation are moved to the External Merit Review phase. During this time, external subject matter experts review the application and assess the application against the Merit Review Criteria. Next, we hold collaboration sessions with the reviewers to discuss the criteria and scores. The applications, next, are reviewed internally by NIJ science staff and leadership, and all funding decisions are made at the discretion of the NIJ Director. You can find more details about this on page 31 and 34 of the solicitation. 

We wanted to provide a few general tips for applicants. Your proposal should be responsive to the funding opportunity as previously mentioned. The application should be well-written, innovative, timely, rigorous and well-designed, feasible, and impactful. We encourage you to read the solicitation carefully and fully. A link to the solicitation has also been added to the chat. In addition to the page that I referenced earlier, we also have a few additional support options which are listed here for your review. And in addition, we have a few recommended resources. And now I will ask Danielle to join me, and Nancy as well, if you'd like, in the questions and answers portion.

DANIELLE MCLEOD-HENNING: Thank you so much, Danielle. We will open this up now for Q&A.

DANIELLE CRIMMINS: The first question, "Can federal government employees compete for NIJ grants as principal investigators?"

DANIELLE MCLEOD-HENNING: Yes, they can. Federal government agencies are eligible entities to apply. If an award were to be made to a federal government entity, then the award would not be through a grant but would be through an interagency agreement between NIJ and the awarded agency.

DANIELLE CRIMMINS: Thank you. The next question. "Would evaluation of technologies that support effective prioritization of CSI-type resources response to crime scenes or serious crimes be within scope?"

DANIELLE MCLEOD-HENNING: I would encourage you to read the solicitation carefully On the surface, it sounds like that would be a responsive project for this funding opportunity. But, again, I would encourage you to read through the specific information section of the solicitation in its entirety.

DANIELLE CRIMMINS: We have one more in the chat. "If an award is made, what is the timeline for finding out decisions and when funding is made available?"

DANIELLE MCLEOD-HENNING: So funding decisions are made by September 30th. By September 30th of 2024, funding announcements will have been made. We recommend a start date of January 1 of the following year, so that would be a January 1, 2025 start date. The reason that we recommend that is once an award is made, there are certain special conditions that may withhold funds, so that gives our Office of Grants Management time to work with the awardee to remove those special conditions so that the project can begin groundbreaking on January 1.

DANIELLE CRIMMINS: Thanks. So, we do have a few more questions coming in. "Can you remind us of the deadline?"

DANIELLE MCLEOD-HENNING: Yes. The initial deadline to submit the SF-424 and the SF-LLL into Grants.gov is April 29, 2024, and then the due date for the full application into JustGrants is May 6, 2024.

DANIELLE CRIMMINS: Thank you. "What is the anticipated award amount range per award?"

DANIELLE MCLEOD-HENNING: There is a maximum of $1 million that is being appropriated for this particular solicitation. We anticipate one to two awards being made under this solicitation.

DANIELLE CRIMMINS: "Does NIJ provide any options for assistance in a connection between a practitioner with a researcher?"

DANIELLE MCLEOD-HENNING: Yes. At nij.gov, we do have a page about connecting forensic science practitioners with researchers. Our Forensic Technology Center of Excellence at RTI is also a resource and they also have a page that seeks to help create those partnerships between practitioners and researchers. So there are some resources out there on both nij.gov, and forensiccoe.org, which is NIJ's Forensic Technology Center of Excellence website.

DANIELLE CRIMMINS: Thank you. "Does the solicitation require a focus on technology-based improvement?" Second part of the question, "Could a proposal focus be on better work processes that, for example, address backlog?" "Are changes to public/victim perceptions of justice due to use of a given forensic technology acceptable outcomes to be evaluated?"

DANIELLE MCLEOD-HENNING: I think that we would need to see a full proposal. But on the surface, it does appear to be responsive.

DR. NANCY LA VIGNE: I would second that.

DANIELLE MCLEOD-HENNING: I know that we've received a lot of questions asking about the scope of particular projects, and I will say that we will have to see a full proposal before we could actually assess it on its own merit and assess its responsiveness. But, initially, what seems to be occurring is it seems that everything that has been proposed appears to be responsive. But I just wanted to caveat that a full review of the proposal is necessary.

DANIELLE CRIMMINS: Thank you. "What is the length time window of the projects?"

DANIELLE MCLEOD-HENNING: A project cannot exceed five years in length. Typically, projects, I would say the average at least on our Forensic Science R&D side, usually run about two to three years.

DANIELLE CRIMMINS: "Can you give an estimate of what percentage of funding goes to PUI institutions versus R1 institutions for federal/state labs?" Not sure if we'll have that data offhand.

DR. NANCY LA VIGNE: Can I ask a follow-up question of the person who posted that question? Are you trying to discern whether we give priority to one over another or that one would be more competitive than another? If so, no, we do not have a priority.

DANIELLE CRIMMINS: If I see a follow-up from the individual who asked the question, I will circle back to that. The next question, "Can you give an estimate of common average award sizes for this solicitation?"

DANIELLE MCLEOD-HENNING: This is the first year that this solicitation is being offered, so we don't have that history. However, like I stated earlier, $1 million is being dedicated to this funding opportunity.

DANIELLE CRIMMINS: Somewhat similar, "What is the average number of applicants?" As Danielle said, this is the first time we have offered this exact option. 

"Are follow-up meetings with NIJ required to monitor the progress of awards?"

DANIELLE MCLEOD-HENNING: Yes. NIJ and, overall, our overhead umbrella agency, which is the Office of Justice Programs, which is this justice grant-making agency under the U.S. Department of Justice, we do have a monitoring plan through our Office of Grants Management that, yes, awards are monitored and annual desk reviews are done on awards.

DANIELLE CRIMMINS: "Can we revisit the slide and/or probably verbally discuss the priorities given?"

DANIELLE MCLEOD-HENNING: Yes. So priorities under this solicitation that may be given further consideration are for new investigators that are submitting or for applications from minority-serving institutions. And that is found on page 12, I believe, of the solicitation.

DANIELLE CRIMMINS: Thank you. We have a question from a PhD student. "Wondering if there is a channel for me to participate?" I think they might want to turn them towards the GRF solicitation that is still open.

DANIELLE MCLEOD-HENNING: Yes. GRF is a funding opportunity for PhD students and we would encourage you to look at that funding opportunity.

DANIELLE CRIMMINS: "Does a winning proposal have to have a collaboration with RTI-type agencies or can a sole university apply on its own?"

DANIELLE MCLEOD-HENNING: A sole university can apply on its own.

DANIELLE CRIMMINS: We did hear back on the previous question. They were wondering if R1s have priority over others. And so, we have answered that fully. 

"Do labs need to be accredited to participate?"

DANIELLE MCLEOD-HENNING: No, they do not. That is not part of the eligibility under this funding opportunity.

DANIELLE CRIMMINS: "Is there a requirement to test new technologies on adjudicated samples or are mock evidence samples suitable?"

DANIELLE MCLEOD-HENNING: Yes, mock evidence samples are suitable.

DANIELLE CRIMMINS: There is one more question. "Are fundamental studies that do not provide in-field testing competitive?"

DANIELLE MCLEOD-HENNING: Again, I think that we would have to assess the proposal as submitted. I'm not sure that I fully understand the "do not provide in-field testing".

DR. NANCY LA VIGNE: I have to say probably not, based on what was proposed. We really want evaluations of real-world application. So, we do a lot already in the way of testing in a laboratory setting or in other ways.

DANIELLE MCLEOD-HENNING: Yes. And we do have a robust number of awards that are looking into fundamental studies on accuracy and reliability of forensic science disciplines. I would encourage you to look at our current solicitation, Research and Development in Forensic Science for Criminal Justice Purposes. That is our large forensic science R&D solicitation that is currently open.

DANIELLE CRIMMINS: Thank you. Related to funding priorities, "How do we define new investigator?"

DANIELLE MCLEOD-HENNING: There are two separate criteria, if you look on page 12 of the solicitation. One would be early career investigator who has not competed successfully for NIJ funding other than GRF, and the other would be an experienced investigator who has not, within the past 10 years, received NIJ funding. But, again, you can look at the criteria starting on page 12 to get further details on that.

DANIELLE CRIMMINS: Danielle, I do think you touched on this, but a few more questions regarding the number of awards. "Is there only going to be one or will there be multiple?"

DANIELLE MCLEOD-HENNING: That depends on the applications that we receive and what the funding requests are, but we will have up to a million dollars slated for this solicitation. So that could be one, that could be two.

DR. NANCY LA VIGNE: Yes. Or more.

DANIELLE MCLEOD-HENNING: Or more.

DR. NANCY LA VIGNE: It just depends on the scope of the applications that rise to the top. My preference is that we fund more than one, maybe a few.

DANIELLE CRIMMINS: Thank you. We have a follow-up question to the new investigator status. "Is that 10 years since received or 10 years since the end of past grants?"

DANIELLE MCLEOD-HENNING: Ten years since awarded.

DANIELLE CRIMMINS: "Do crime scene investigation techs fit within scope?"

DANIELLE MCLEOD-HENNING: All forensic science disciplines are within scope. 

DANIELLE CRIMMINS: All right. I think that is all the questions.

STACY LEE: On behalf of the National Institute of Justice, thank you for joining today's webinar. This will end our presentation.

DANIELLE MCLEOD-HENNING: Thank you all very much.