Webinar Transcript: NIJ FY 2024 Research and Evaluation on Youth Justice Topics

This webinar will provide an overview of the NIJ FY 2024 Research and Evaluation on Youth Justice Topics solicitation. In collaboration with the Office of Juvenile Justice and Delinquency Prevention, NIJ seeks applications for research and evaluation projects that inform policy and practice in the field of youth justice and delinquency prevention.

Specifically, this solicitation seeks proposals for studies that advance knowledge and understanding in the following two categories:

1. Youth Justice Reinvestment Studies which evaluate the effectiveness, including cost-effectiveness, of youth justice system reforms and subsequent reinvestments into programs that serve youth in their communities (prioritizing collaborative partnerships between community-based organizations and youth justice systems).
2. Prevention and Intervention Program Effectiveness Studies which evaluate the effectiveness of school and community-based (a) delinquency prevention programs, (b) intervention programs for youth engaged in delinquent behavior, and (c) intervention programs for youth who have been victimized by abuse and/or experienced trauma.

Transcript

STACY LEE: Hello, everyone, and thanks for joining us for the Fiscal Year 2024 Research and Evaluation on Youth Justice Topics Solicitation Webinar. It is my pleasure to introduce Barbara Tatem Kelley, a Senior Social Science Analyst at the National Institute of Justice.

BARBARA TATEM KELLEY: Welcome everyone and good afternoon. Thanks for joining our webinar. My name is Barbara Tatem Kelley and I'm one of the social science analysts here at the National Institute of Justice, which we generally refer to as NIJ, working on the youth justice portfolio. I'm joined today by Ben Adams who leads the Office of Crime Prevention and Youth Justice at NIJ. For those of you who are not familiar with NIJ, we are the research, development, and evaluation agency of the US Department of Justice. Today's webinar is focused on the fiscal year 2024 Solicitation--Research and Evaluation on Youth Justice Topics. This solicitation was released March 7th, closes June 4th, and there's also a May 21st Grants.gov deadline. We'll go over with these two deadlines shortly. 

But first, I would like to take this opportunity to introduce the Director of the National Institute of Justice, Dr. Nancy La Vigne. Dr. La Vigne is a strong advocate for research which addresses the concerns of children and youth including child safety and protection, delinquency prevention, and youth justice. Director La Vigne collaborates closely with our counterpart at the Office of Juvenile Justice and Delinquency Prevention (OJJDP), Administrator Liz Ryan, to identify key topics which merit the focused lens of NIJ research and evaluation efforts. And without further ado, I'll turn it over to you, Director La Vigne. Thank you.

DR. NANCY LA VIGNE: Thank you very much, Barbara. And good afternoon, everybody, or I guess it might still be morning for a few of you on the West Coast. Thank you so much for joining this webinar on this important topic, youth justice and related issues. I am Nancy La Vigne. I am Director of the National Institute of Justice, and I'm coming up on my two-year anniversary in May. One of the many joys of this position is being able to seed and fund research on topics that are important to people, to families, to communities, and especially to kids. They are our future and they're so very important to us. Unfortunately, there are a lot of ways that kids end up in the juvenile justice system or at risk of being system-involved. 

And Barbara mentioned OJJDP Administrator Liz Ryan, who likes to say one of her biggest priorities is treating kids as kids. I support that goal and I'm really privileged and honored to have her as a partner at the Office of Justice Programs (OJP), and to have a long-standing partnership that predates both of us between our two agencies to collaborate on research investments that are about the well-being of our country's children. We do that in a variety of ways. We're in close contact and collaboration. I'm sure you know OJJDP is focused primarily on training and technical assistance and support around issues of children in the justice system, whereas we're the research arm. So they have their ear to the ground around a lot of the needs of the field and the needs of children. And of course, we try to have our ears to the ground in terms of where the gaps are in knowledge and how they can best be met. 

We follow what's called the “Listen, Learn, and Inform” model. We're not just collaborating and listening to each other in our two agencies. We spend a lot of time listening to the field through formal convenings, through roundtables, listening sessions, conference participation, and the like. So we're really intent on knowing where researchers and practitioners and families and community members and child advocates see the needs and then we try to best fill them, through seeding research. But then we disseminate and share as widely as possible to help promote change. 

As Barbara will be sharing, this solicitation this year is focused very much on treating kids as kids. One section is particularly focused on how to reduce the use of confinement of youth in facilities, and to reinvest in community-based alternatives. We also have a very heavy focus on program evaluation. One of the things we hear from the field is that they want to know what works. So we have a heavy emphasis on requesting proposals for evaluation across prevention and intervention activities, as well as focusing on child victims in the context of community and schools. 

You'll be hearing more about that from Barbara, but I did want to share a little bit about some overarching themes and priorities that we have here at NIJ that I brought to the Institute when I started here. I'll just go over them very briefly because they all align with this solicitation and what we're seeking. First of all, we highly encourage and will prioritize proposals that embody what we call inclusive research. So that's research that takes the time to engage with the people who are closest to the issue or problem under study. Now, that doesn't mean that it has to be full-on qualitative research, although it could be. Community-based participatory research is a really excellent model for inclusive research. But it could also be a highly-quantitative undertaking that still builds in activities along the way that solicit input and feedback from the people who are closest to the issue or problem. It could be people with practitioner expertise, and it absolutely should be people with lived experience or their advocates. That's one key priority. 

Another priority that we're always careful to emphasize is approaching research through what we call a racial equity lens - or really an equity lens writ large. There are a lot of different ways we can think about equity in terms of race, ethnicity, identity, gender, and sexual orientation. There are a lot of different ways that we can think about these issues. But what we're inviting our applicants and our funded grantees to do is really be very intentional about issues of race and other issues of equity or biases. Also be mindful that sometimes they’re baked into our data sources. Sometimes they influence our methodologies based on how we're trained. It's important to think about whatever topic we're studying, even if it's about a program that seems to be, say, race-neutral, to look at outcomes. Who has access to these programs based on their race, their ethnicity, their identity? Who's more likely to be able to follow through on program participation and content? Looking at these issues can help us develop implications from the research that cannot just yield the overall goal of keeping kids out of facilities but also ensure that these goals are equally beneficial to all. 

Another priority that we lift up is a focus on interdisciplinary research. We've funded a lot of research teams that span various academic disciplines and perspectives. We find that those teams tend to have the most innovative, the most robust methodologies, have findings that really have strong implications throughout the field, and really create a very robust team to tackle any number of research projects. So we really want to encourage you to think about getting outside of your silos and teaming up with people in other academic disciplines or other areas that touch upon the topic that you’re addressing. 

I mentioned that one big part of this solicitation is on program evaluation, so I did want to note something that's very important to us at NIJ. That is, we absolutely want to seek proposals that show rigorous evaluation methodologies. They don't have to be randomized controlled trials. There are other ways to do that rigorously, but comparison groups of some kind are a must. But we don't want you to stop there. It's really important that you also include an implementation or process evaluation in your proposal. What we mean by that is taking the time to measure the inputs and to assess the degree to which the program was being implemented as it was intended. Did it reach the right target population? Was the amount of programming delivered as intended? Did it unfold as planned? This is really important, and we also welcome proposals that use an action research model. As you're learning these things about implementation fidelity and you're finding that fidelity isn't as strong as it could be, we encourage a relationship with the program operators where you're feeding that information back as you go rather than waiting until the end. That in turn creates for a stronger program during the period of evaluation so we can learn from the best possible implementation of the program. 

The final priority I want to speak to today is what we call “evidence to action”. And that is the focus on not just translation of research findings so they're understandable to others, although that's important and we encourage that, but also really making sure that the findings are shared in a lot of creative ways that meet the people we're trying to influence where they are. We do a program evaluation. We learn that a certain program is effective. How do we get that word out to the right people and who are the right people? The right people are people who are thinking about developing programs in our own jurisdictions or perhaps people who are thinking about funding those programs. So we're really interested in this notion of getting from the evidence to action through creative dissemination activities. And that's why we've built into all of our solicitations a priority incentive for you all to dedicate at least 15% of your budget to creative dissemination activities that go beyond the journal articles, beyond the executive summary, to other activities — conference presentations, engagement with practitioners, engagement with the community. It's all going to depend on the topic, making sure that you're building in those activities and that you're well-budgeted to engage in those activities. I'll stop there. I feel like I've already taken up too much of your time - I know you're really here to hear more about the content of the solicitation. 

I believe Barbara's going to make this point but let me reinforce it here. There's one more thing that I wanted to draw your attention to, and that is that I recently delivered what's called a “Dear Colleague” letter[A1]  to the field. This letter is not just from me. I mean, my name is on the end of it, but it really represents the collective wisdom, experience, and observations of NIJ's science staff, and what makes for a strong proposal, and some of the areas of weakness that we routinely see. Many of which, if not all, are easily avoided. I highly encourage you to look at that letter and follow it to a T. It's in our interest to see the strongest possible proposals that we can see. Thank you so much, and I will turn it back to Barbara.

BARBARA TATEM KELLEY: Thank you, Dr. La Vigne. I really appreciate your sharing your insights with our audience. The purpose of our webinar is to provide an overview of NIJ's fiscal year 2024 solicitation--Research and Evaluation of Youth Justice Topics. We will be discussing the application and review process. Additionally, we'll go over some common issues and critiques that we see during the peer review process and provide some tips, and then we'll answer some frequently asked questions. Finally, we'll identify places to access additional resources before we transition to Questions and Answers. 

In collaboration with our Office of Juvenile Justice and Delinquency Prevention, we are seeking proposals for rigorous research and evaluation projects that will inform policy and practice in the field of youth justice and delinquency prevention. NIJ has an emphasis, as Dr. La Vigne mentioned, on actionable research. Specifically, this solicitation seeks proposals for studies that advance knowledge and understanding in the following two categories:

• Category 1: Youth Justice Reinvestment Studies
• Category 2: Prevention and Intervention Program Effectiveness Studies

In fiscal year 2024, any applications that are submitted to proposed research outside of these two specified categories will not be considered. Each year and in previous years, the solicitation title has often been NIJ Research on Juvenile Justice Topics. We specify categories under consideration for that particular year. 

Category 1: Youth Justice Reinvestment Studies. Research indicates that confining youth in secure residential facilities, such as detention and correctional centers, can and indeed does negatively impact normal adolescent development. Removal of youth from the community results in the associated loss of opportunities for positive engagement with their family, pro-social peers, schools, and potential employers. In addition, confined youth are at a heightened risk for criminogenic influences, victimization, and suicide. Many state and local juvenile justice agencies are striving to deliver services to youth in community settings while still protecting public safety and improving youth outcomes. When cost savings are realized with declining placement of youth and secure facilities and the closure of such facilities, these savings may then be reinvested in the delivery of treatment and services in the community. Under Category 1, we invite applicants to propose independent and rigorous evaluations of youth justice system reforms and reinvestment efforts. Applicants are encouraged to assess not only negative outcomes such as escalating violence and a higher level of recidivism, but also to assess positive adolescent development outcomes such as educational achievement, association with pro-social peers, and behavioral and mental health positive outcomes. Category 1 proposals must include all three components of a process evaluation, outcome evaluation, and cost assessment. 

Category 2: Prevention and Intervention Program Effectiveness Studies. Although there is an impressive body of research on youth-focused prevention and intervention programs, many contemporary programs in use at the state or local levels have not been subjected to scientifically rigorous evaluation yet. So Category 2 will support field-initiated innovative and methodologically sound evaluations of contemporary community and school-based prevention and intervention programs. Applicants may choose to focus on three priority areas. First, prevention of the onset of delinquency. Second, intervention in the lives of youth engaged in delinquent behavior to support desistance. And third, intervention and programs for youth who have been victimized by abuse and/or experienced trauma. Category 2 applicants are required to include both of the following two components, the process evaluation and the outcome evaluation, and they're further encouraged to assess the cost associated with program delivery. 

May applicants apply under both categories? That’s a question that will be commonly asked when we have a solicitation with specified categories. You may submit proposals to both of these categories, but each proposal must be in a separate application.

Please identify the category to which a proposal is submitted on the title page of the proposal narrative. We go through a process of sorting out the applications and generally try to have our peer review panels focus on all related applications. So having that category identified right on your title page of the proposal narrative is very helpful. 

Now, what will we not fund this year? We won't fund unresponsive applications, those not directly addressing one of the two specified fiscal year 2024 categories. We also may not use funds to support the purchase of equipment, materials, or supplies, or the delivery of direct services. However, a budget may include equipment, materials, supplies, et cetera if these items are necessary for the actual conduct of the research development, demonstration, evaluation, or analysis. 

I wanted to point out to you that there is also an open solicitation that NIJ has posted entitled NIJ FY24 Research and Evaluation on School Safety. There is one area of that particular solicitation that might be of interest to some of the people attending this webinar. If you are planning to focus on the evaluation of impact and effectiveness of school safety approaches implemented specifically under the Stop School Violence Act, then I would recommend you consider the relevance of the goals and objectives of this other solicitation as well and try to determine which solicitation is the best fit for your proposal. Please refer to:  NIJ FY24 Research and Evaluation on School Safety,

If your application is funded, there are several required deliverables. You will need to comply with the standard grant reporting requirements. including submission of  the semi-annual progress reports and also the quarterly financial reports. Additionally, any applicant that is awarded a grant under this solicitation will be expected to submit a final research report at the end of the award period and provide a draft of that report 90 days before the end of the award project period. Award recipients will also be expected to submit to the National Archive of Criminal Justice Data (NACJD) all data sets that result in the whole or in part from the work funded by the award, along with associated files and any documentation necessary for future efforts by others to reproduce the project’s findings and/or to extend the scientific value of the dataset through secondary analysis. In addition to these deliverables, NIJ expects scholarly products to result from each award under the solicitation. These can take the form of one or more peer reviewed scientific journal articles or other appropriate products as listed in the solicitation. NIJ expects that there will be an equal effort to make the research findings accessible to practitioner and policymaker audiences, not only researchers. This is in keeping with NIJ’s emphasis on evidence to action. 

There are key elements of an application that must be submitted for the application to even be considered. If even one of these elements is missing, the application will not move forward to the review process. Each of the following elements must be included in the application to meet the basic minimum requirements and advance to peer review and receive consideration for funding:

• Standard Forms SF-424 and SF-LLL (which you will have to submit into the Grants.gov system)
• Program Narrative
• Budget Web-based Form that includes budget details and narrative
• Curriculum Vitae (CVs) or Resume for key personnel (For the purposes of this solicitation, key personnel means the principal investigator and any and all co-principal investigators.) 

In addition to the minimum requirements, there are many other requirements that need to accompany an application for full consideration. I've listed a few here, such as the project abstract, letters of support, tools/ instruments, questionnaires, tables, but more importantly, I flagged the page where the full explanation and the application checklists can be found in the solicitation that begins on page 20. 

There's a two-step submission protocol that requires submitting different information in two different systems, each of which has its own deadline;

• First, you will need to submit an SF-424 and SF-LLL. Those are standard application forms and notification of lobbying activities in Grants.gov. The deadline is May 21st. The time of that deadline 11:59 p.m. Eastern Time. 
• Second, you will submit your full application including all the applicable attachments in JustGrants. So this is the closing deadline for the full application on June 4th. Please note, this application deadline is a bit earlier at 8:59 p.m. Eastern Time. 

A couple of important additional notes: 

• Applicants must register with Grants.gov and JustGrants prior to actually submitting an application. Processing delays of up to several weeks can sometimes occur when you're registering. That's why we strongly encourage applicants to register several weeks before the application submission deadline. Even if you're not ready to submit your application yet, please register early in these systems. 
• If you do not submit your forms in Grants.gov by the deadline, your application in JustGrants will not be accepted. 
• When submitting your application, we're urging applicants to submit at least 72 hours prior to the deadline time to allow time for the applicant to receive a validation message or a rejection notification from Grants.gov. And this way, you'll be able to correct in a timely fashion any problems that may have caused a rejection notification. This will also allow you to verify all materials were correctly uploaded in JustGrants. And please do go into JustGrants before the deadline passes and make sure all the attachments you intended to attach are uploaded in JustGrants. 
• We recommend that you label your documents and attachments appropriately with the file name indicating what the document is. For instance, call a program narrative a program narrative, call a timeline a timeline, and upload the document to the corresponding section in JustGrants. 
• Our deadlines are hard. I know that can be very difficult for people when sometimes life happens. But basically in the spirit of fair and open competition, we are very strict with our deadlines. In the past, if you come in after the 8:59p.m. Eastern Time deadline in JustGrants, the application is rejected. 

Applicants report missing deadlines because they have used third party software to submit their application and the third party failed to properly submit their application on time. These applications cannot move forward for consideration. 

We also often get questions about whether an applicant can submit an additional piece of the application that was missing from the initial submission or not received in time for the deadline, such as a letter of support. After the deadline, these requests are not accepted. 

Finally, we get requests for exceptions or extensions. We are unable to grant these unless an applicant can prove — and let me stress prove — that there was a technical error in the NIJ system or software while the applicant was submitting the application that caused a missed deadline. NIJ cannot accept applications after the deadline because this is a competitive process. We are unable to grant applicants additional time as it would be unfair to those applicants that submit their application in full and on time. In order to prevent some of these issues that we discussed from arising, applicants are encouraged to contact the OJP Response Center, Grants.gov, or JustGrants support. We've listed the full numbers and email addresses here (Slide 12). This contact information can also be found in the solicitation itself. Please document and retain any emails that you have in correspondence with these organizations about technical problems should you later need to appeal the application not formally being accepted in the system. 

Application Review Process. After the application closes, all applications are screened for basic minimum review (BMR) requirements. During the BMR process, we assess the application to make sure that it's responsive to the solicitation. That is, you are proposing a research project that responds to at least one of the two specified categories, and it is being submitted by an eligible application organization. For instance, is it a university or research organization? We don't accept applications from individuals. During the BMR, we also check to make sure that the application includes the critical elements, those again being the required forms, the program narrative, the web-based form including the budget detail and budget narrative, and the CVs or resumes of all key personnel. 

As a reminder, there are other attachments that should be included in the application such as the human subjects and privacy documentation (which is required for all projects, whether human subjects are involved or not), the data archiving plan, letters of support, and the project timeline. While these are not critical elements that would cause an application to fail the BMR process, when you don't include these items (among others as they are applicable to your application), this may result in a less favorable review or a delay in releasing funds if an award is made.

Once your application passes BMR, it moves onto the External Peer Review Process. These external peer review panels are made up of researchers and practitioners in the field with expertise relevant to the particular solicitation. The peer reviewers score the applications and discuss them, providing NIJ with both the merits and weaknesses of the proposed research. The review criteria can be found in the solicitation starting on page 31. NIJ then uses these external peer review scores and comments to inform the conduct an internal review of the application. The internal review includes a team of science staff and leadership at NIJ in addition to any other relevant experts who may be able to weigh-in on the merits of the application. After reviewing the applications themselves, taking into consideration the scores and comments provided the external peer review panel as well as the budget, NIJ science staff then make funding recommendations to present to the NIJ Director. The NIJ Director then decides which applicants will be awarded funds. It should be noted that all final funding decisions are made at the NIJ Director's discretion. 

I cannot strongly encourage you enough to read and take to heart the insights the NIJ Director Nancy La Vigne shared in her Dear Colleague letter that was posted last month:

I will walk through some, but not all, of the points made in that letter. Many of these points have relevance for all of our solicitations, not just this single solicitation. NIJ favors proposals that meet our funding priorities, which are those that:

• Propose an inclusive research design.
• Address issues of race, gender, ethnicity, and other potential disparities in data sources, research methods, and outcomes. 
• Propose a multi-disciplinary research team.
• Ensure rigorous measurement of implementation fidelity.
• Allocate ample resources for the translation and dissemination of research findings. 

To be successful, an applicants must also achieve two key objectives. First, they must propose rigorous research that develops needed knowledge or tools to address the major challenges of safety and justice in the United States. Second and equally important, applicants must demonstrate that resulting products have a potential to address those challenges--either directly through policy and practice improvements or indirectly by advancing the body of knowledge. 

Other commonly missed opportunities in submitting a strong application include:

• Failure to describe how the applicant will ensure research independence and integrity of the evaluation findings. This is particularly an issue when the project team includes program staff or investigators who have been directly involved in the design and development of the approach being evaluated. We are looking for what would make an objective reader put credence in the findings rather than question the findings based on the lack of objectivity. 
• Insufficient discussion of the potential risks and harms to individuals or groups associated with the use or misuse of a proposed research practice or technology. 
• Undeveloped dissemination strategies that fail to make the case that the resulting research information will lead to actual changes in the field. 
• Timelines, staffing plans, and/or budgets that are not aligned directly with the proposed work.
• Inadequate specification of research questions and/or failure to connect research questions to the research design and analysis plan.
• Failure to demonstrate the most rigorous feasible research and analysis methods have been proposed. In other words, make sure your science is fully and feasibly developed. 

Common critiques raised during peer review are what I'm going to flag now, and we see these raised during the peer review process across hundreds of applications submitted to NIJ each year. In the statement of the problem, we see that sometimes the statements fail to identify gaps in the current literature or understanding of current research. The literature review is sometimes insufficient and the scope for the proposed research is extremely limited or conversely too ambitious to be achieved. We also see applicants dedicate too much text or too much real estate in the 30-page limit, to argue for the necessity of the proposed research at the expense of clearly articulating the innovation rigor and feasibility of the project design. Bear in mind that NIJ solicitations weigh the selection criteria. And for this solicitation, your statement of the problem is weighted at 15% of the total score whereas the program design and implementation section is set at 45% of the total score. 

Other common critiques raised during the peer review process focus on the research design section. Sometimes we see that the research questions are not derived from the literature review or are inadequately specified. We see problems where: the research design is not well-articulated and clearly laid out; the research design or methods do not flow logically from the problem statement, research questions, and literature review; or where the design and methods are too ambitious and too complex to be feasible. Alternatively, we see applicants fail to demonstrate the proposed research and analysis plan are the most rigorous approach that is feasible. In terms of additional critiques on the research design, we sometimes see the sample, such as the sample size, not being supported by a power analysis or a sampling strategy being flawed or too ambitious. We often see problems with the quantitative analysis being vague and unclear, or the data collection and analysis plan being confusing. If complex predictors, mediators, moderators, or other outcomes are proposed for a study, their measurement must be well-described. If there are limitations in the available secondary data, the application must acknowledge the limitations and describe migration efforts and potential impact on the interpretation of the result. If multiple secondary data are proposed for compilation for analysis, the applicant must describe how the data sets will be matched, in terms of the unit of analysis and the specific variables common in each data set. 

Other common weaknesses include: failure to identify potential research challenges and strategies to mitigate them; and failure to demonstrate access and the ability to utilize proposed data sources (such as the ability to merge multiple administrative data sources). Applications also suffer when the timelines, staffing plans, or budgets do not support the work proposed. There are special considerations to be addressed from proposing a program evaluation. Applicants must demonstrate the projects are sufficiently developed and well-positioned to support the proposed evaluation. For example, before a rigorous outcome evaluation may be conducted, there must be clearly designed conceptual framework that is a logic model. NIJ wants to build understanding not only of whether a program is effective but also what elements within or external to the program design or implementation contribute to a success or failure. Bottom line, the innovation, rigor, and feasibility of the research design to answer clearly articulated research questions needs to be fully demonstrated including consideration of risk and appropriate mitigation strategies. 

Common critiques raised about capabilities and competencies section. The Principal Investigator sometimes does not demonstrate familiarity or proficiency with the proposed quantitative analysis. When an applicant organization or key research staff have direct involvement in the program they propose to evaluate, peer reviewers question the perceived objectivity of the findings. 

In the potential impact area, we see instances where the dissemination plan lacks specificity or is not innovative, or there is no plan to reach non-academic audiences. It's important to make a case for how the research will meaningfully contribute to safety and justice and how the dissemination plan will make the resulting research information actionable. Specifically, Dr. La Vigne's Dear Colleague Letter notes underdeveloped dissemination strategies that fail to make the case that the resulting research information will lead to actionable changes in the field is a major concern. 

Now, I have a few tips. Overall, I think one of the key takeaways from these critiques is that the research proposals need to be well-written, feasible, impactful, timely, innovative, and clear. Applicants should demonstrate an understanding of the current needs, the existing literature, and the work that NIJ has already funded. The application itself should be easy to read and explicitly with no mystery around what is being proposed or how it will be achieved. The research design should be as rigorous as possible and the sampling strategy should be backed with demonstrated relationships or letters of support that will make it attainable. Secondary mitigation plans should be in place and articulated. The application should articulate the extent and importance of the project impact on the field. That is applicants should ensure that the findings from the proposed research, if awarded, will have a potential for high external validity beyond the focus of the study itself. Applicants should also demonstrate consideration of ensuring research independence. In particular, the applicant must describe how they will ensure research independence and integrity in evaluation findings when the project team includes program staff. 

I also wanted to highlight some frequently asked questions we receive. “What is the award amount and period of performance?” The award amount actually depends, in a large part, on the quantity and quality of the applications we receive. Each solicitation has a maximum amount to be awarded under each category. There's no set number of awards to be made under each of the two categories specified in the Research and Evaluation on Youth Justice Topics Solicitation, but no application under a category can exceed the total dollar amount available for that category. The period of performance may be up to five years in duration, but, of course, this may be shorter depending upon the nature of the proposed research. 

“When does NIJ release notify applicants about awards or non-awards?” Award announcements are usually made by October 1. Non-award announcements are usually made by the end of the year and non-award applicants are sent their peer review comments. There have been some years in the recent past where award and non-award announcements were delayed, but we don't anticipate this being an issue moving forward. 

“What about foreign entities?” Basically, this is a kind of complex subject. I'll try to cover it as quickly as I can but you may have questions you want to refer directly to the OJP Response Center. “Are US citizens working outside of the US eligible, either as a PI or co-PI? Can any grant funds be paid to institutions outside of the US?” Our solicitation this year states that foreign governments, foreign organizations, foreign colleges, and foreign universities are not eligible to apply. However, the situation with subrecipients and subawards may be different. We can get to subawards in a minute. First of all, the initial question, we encourage everyone to review the Code of Federal Regulations, particularly 2 CFR Part 200, which covers various matters regarding foreign organizations or entities, costs incurred in foreign countries, and several other matters that may arise in connection with non-US-based entities or individuals that may be funded under a federal grant award. The DOJ Grants Financial Guide also covers policies with regard to foreign travel costs under an award. Generally speaking, a US citizen who's not otherwise debarred or suspended from participating under a federal award, speaking as a general matter, could be funded under the award even if working outside of the US. However, more information about the work being done, by whom, and in what capacity is unique to every situation. So if you have a specific question, please submit it through the OJP Response Center so it can be routed to the appropriate person. 

“Can any funds be paid to institutions outside of the US?” NIJ solicitation language barring applications from foreign entities would not mean that a US-based applicant could not propose a foreign subrecipient or submit a proposal under which a US citizen could perform work outside of the US. So foreign entities cannot apply as a direct applicant, subrecipient arrangements could be made. All in all, we encourage applicants to familiarize themselves with the Code of Federal Regulations mentioned and exercise the appropriate cautions when partnering with foreign entities to conduct research. Data archiving, human subjects, and privacy considerations are all major contributions to whether a partnership with a foreign entity would be feasible under an OJP or NIJ award. Again, if you have specific questions, please send them through the OJP Response Center. 

Lastly, we receive a lot of questions around submitting forms, what constitutes a new investigator, and other questions related to privacy and human subject concerns. For all of these, I encourage you to please submit the questions through the OJP Response Center listed on the solicitation and displayed on Slide 21. Your questions will be routed to the most appropriate person who can answer the question. 

To ensure costs are allowable, we encourage applicants to review the Funding Resource Center for additional information and helpful guidance. We also encourage applicants to review the Department of Justice Grants Financial Guide and take the online training, which is actually a requirement of all funded grantees. Finally, project descriptions and an overview of the portfolio are available on the NIJ website. Review this information for an idea of the type of projects and awardees that NIJ has funded in the past and also to view what funding is currently posted. 

Questions and Answers

Well, we’re ready for our Q&A now. And, at this point, we will be joined by Ben Adams, who will also help to field these questions. Welcome, Ben.

BEN ADAMS: Thank you, Barbara. I’ll start with the first question. “Was it stated that funding will not cover the delivery of direct services, even if those services are being evaluated in the research?” 

I’ll just reiterate that applications proposing to primarily provide direct services will not be funded. Any budgets that include funds related to service delivery must demonstrate the necessity, emphasis on necessity, for conducting research and evaluation. So, for instance, if a subrecipient is a service provider, the applicant must demonstrate that the funds will be used to support research, development, and support activities. Think about things like research site coordination, scheduling of data collection activities, research related support activities, and project stipends for research participants or incentives. 

Another question here. “Will you be resharing here or via email links to all documents associated with the webinar?” 

Yes. We mentioned that the slides and transcript from this webinar will be posted within the next few weeks, including responses to the questions asked during the webinar. 

A question that I think I'll pass to Barbara. “Will NIJ consider prevention of status offenses and inequities in court petitions for status offenses as a primary outcome or must the outcome be criminal activity?”

BARBARA TATEM KELLEY: Well, that's kind of an interesting question that I haven't given much thought to before. We speak to how we're looking at delinquency prevention, and I think the terminology is probably broad enough that status offenses, even though they don't qualify as delinquent acts, would be an area that we would also want to prevent. But I do feel the need to defer to you, Ben, on this question. Sorry to throw it back to you.

BEN ADAMS: Well, I would just add that we don't specify outcomes per se within the solicitation other than to encourage the inclusion of both positive youth developmental outcomes as well as any justice-related outcomes. I would encourage the applicant to consider the outcomes most relevant to answering the research questions that are proposed with the application. So referring back to the solicitation there.

BARBARA TATEM KELLEY: Also, the solicitation speaks to positive youth outcomes where attending school and educational achievement would kind of coincide with a positive outcome you might want to look at, which often occurs in the absence of the status offense of truancy.

BEN ADAMS: Right. Sort of an administrative question here. “Must the application be submitted on each site, Grants.gov and JustGrants?” 

Here, again, I'd refer back to the solicitation itself, pages six and eighteen. The SF-424 and SF-LLL are what must be submitted to Grants.gov by 11:59 on May 21. The full application is what must be submitted to JustGrants by 8:59 on June 4. Again, you can find that information in the solicitation. 

There's a question here about letters of support. “What should be the source of a letter, such as from an institute, community organization, or a mentor?” 

I would point you to page 28 of the solicitation. Applicants are requested to include, for each named supporting entity, a signed letter of support that outlines the reasons for supporting the project and the scope of work that they are committing to if they're involved as a subrecipient or in a consultative role. So the elements, again, of those letters are described on page 28 of the solicitation. Importantly, if they're participating by providing data, it must include an acknowledgement that the de-identified project data will be archived with the National Archive of Criminal Justice Data at the conclusion of the project. 

There's a question here about whether or not there's a specific NIJ contact person who would be able to discuss potential fit for proposed research. Because NIJ issues competitive solicitations and the solicitation is open, we do not provide direct individual consultation on research ideas to ensure that this remains a competitive review process. So, again, we'd encourage you to refer back to the solicitation. In your application, make a clear justification for why your proposal is good fit for the goals and objectives that are laid out in the solicitation categories. 

All right. “Will qualitative projects be considered?” I think you spoke to this, Barbara, do you want to say anything else with regard to qualitative projects?

BARBARA TATEM KELLEY: I think you just have to, in keeping with the solicitation, justify that the method that you're proposing is the most appropriate method to answer the specific research questions you're proposing. There is nothing that says you have to do both qualitative and quantitative research, but you have to consider how responsive you are to the solicitation whenever you propose a methodology.

BEN ADAMS: And the only thing I would add is please also be attentive to the guidance provided under the categories for the elements that we are encouraging or requesting be included as part of the evaluation process. I think Barbara spoke to process outcome and, as appropriate, cost analysis being included. 

There’s a question, “If you are uncertain whether your project is a better fit for school safety or youth justice programs, is there a program officer we could discuss this with further?”

I'd refer back to my prior response and encourage you, as Barbara did in her presentation, to identify (based on the solicitation content) the solicitation and category under the solicitation where your proposal fits best. We are not able to have individual conversations about perspective proposals with applicants during the competitive period. 

Let's see. “Category 2 includes the study of intervention effectiveness. Do diversion programs qualify for this category?” Barbara, do you want to speak to the content on that?

BARBARA TATEM KELLEY: Category 2 is really pretty wide open for field-initiated areas to be addressed. For instance, under Category 2, we're looking at intervention programs which focus on youth who have been victimized by abuse and/or experienced trauma which might include instances where diversion approaches are serving youth who have experienced abuse or trauma. There might also be instances in which diversion approaches might be incorporated into programs for prevention of the onset of delinquency and for intervention with youth who engaged in delinquent behavior. Please remember to be responsive to the solicitation language for the category you propose to address.

BEN ADAMS: Indeed, that particular Category 2 is relatively open for field-initiated proposals with respect to prevention, intervention, and our response related to delinquency and victim services. 

“Does the solicitation include proposing a rigorous evaluation of an OJJDP model program that has promising evidence but does not yet have full evidence?” 

There's no specification with regard to that or exclusion criteria related to that. We did talk about building the evidence base for contemporary program models. So, again, you would need to propose and justify why this particular evaluation would fill a research gap and address the needs that are prioritized within the solicitation. 

There's a question about the funding amount. “Applicants can request funds for up to five years. Am I correct that there will be one budget period for awardees for the appropriate length of time for the entire project, not $600,000 per year?” 

Again, the proposal budget should be commensurate with the research proposed, not based on simply the availability of funds that are listed in the solicitation. So budgets should match the research that's proposed, and the budgets provided should be for the full performance period that is proposed.

BEN ADAMS: There's a question here. “Would Category 2 consider an evaluation study of an existing community-driven violence prevention model currently being implemented?” 

Here, again, I would say that that Category 2 is fairly open. However, I would also encourage you to visit the NIJ forthcoming funding page because we do obviously have a major OJP initiative around the community violence intervention and prevention work, and that solicitation has also included funding opportunities in the past for research and evaluation related to those prevention models. 

All right. I think we hit on most of the questions. We are also right at 2:30, so I want to be respectful of everyone's time. And thank you all for joining us. And, again, the transcript and slides will be posted. And we are happy to answer any other follow-up questions through the OJP Response Center. I will pass it back to Stacy to close out the webinar.

STACY LEE: On behalf of the National Institute of Justice, thank you for joining today's webinar. This will end our presentation.