Research and Evaluation on Domestic Radicalization to Violent Extremism

Ms. Giovacchini: Good afternoon everyone and welcome to today’s webinar Research and Evaluation on Domestic Radicalization and Violent Extremism Prevention and Intervention Demonstration Programs, which is hosted by the National Institute of Justice. Your presenters today will be John Picarelli, Director of Violence and Victimization Research Division at NIJ and Phelan Wyrick, Director of Crime and Crime Prevention Research Division at NIJ.

At this time, I would like to introduce you to John Picarelli, Director of Violence and Victimization Research Division at NIJ.

Dr. Picarelli: Good afternoon everyone and thanks for joining us today. What I’d would like to do with my colleague Phelan Wyrick is walk you through a quick slide deck to give you a little more information about the solicitation this year. As you can see here on slide two, what we like to do is go over some of the background for the program and some of the goals for the solicitation this year. Then, we’ll go discuss some of the expectations and requirements for applications, some recommendations for how to prepare your application, the review process that we use and a quick breeze of the application checklist. Everything we’re going to discuss today is available either within the solicitation itself or on NIJ’s website, which is www.nij.gov.

So, let's start with a little background and where want to start is with a definition. For those of you who may not have applied to NIJ before, this program does focus on radicalization to violent extremism as it occurs here in the United States, ergo domestic. For the purpose of the solicitation, radicalization is defined as the process by which individuals enter into violent extremism. Violent extremists are those individuals who support or commit ideologically motivated violence to further political, social, or religious goals. Further, for the purposes of this solicitation, domestic radicalization does limit applicants to focus on radicalization as it occurs in the United States, regardless of where violent extremism that ensues occurs. We are, therefore, in plain English, we want to understand more how the radicalization process happens here in the United States, even if someone were to travel overseas or to another country in order to engage in violent extremism. While applicants are encouraged to use comparative approaches in their applications, applications, and this is critical, applications must focus their findings on how radicalization occurs within the United States. The one take away from this initial definitional journey is that we are looking at all forms of violent extremism as it occurs here in the United States. We will look at whether it is antigovernment, ethnic supremacists, anti-capitalist, religiously motivated, or any other form of  violent extremism and, more importantly, the radicalization to that violent extremism. That is our focus. And if you review the applications that we have funded over  the last four years, you will see this spread across many of our funded projects. So speaking of the Domestic Radicalization of Violent Extremism portfolio is entering its fifth year of existence. We have funded to date 23 separate projects. All of those can be found again on NIJ's websites including a brief summary of each application and I would strongly urge you to review these and become familiar with  them before you apply. We also fit into the broader effort in the federal level to prevent radicalization of Violent Extremism in the United States. The best documents that capture what that is at the federal level are the National Strategy for Empowering Local Partners to Prevent Violent Extremism, and its accompanying Strategic Implementation Plan. There are links to both documents found within the solicitation, or you can Google those titles and find those documents.

So moving on then. This solicitation really does focus on how to effectively prevent and intervene into radicalization as it’s occurring here in the United States. In the previous four years of this program's existence, most of what NIJ has funded has focused on two questions. The primary drivers of radicalization to violent extremism, and how those drivers vary across cohorts such as age or socioeconomic categories and so forth. And the other critical question has been to compare how radicalization of violent extremism occurs and how it may or may not be an analogous to other forms of violence. That is included mass casualty attacks, gang violence, trafficking of human beings and so forth. When you review what we have funded to date, you will notice that much of it has been in the research category. We have also funded some evaluations and I’ll touch on those in a minute. The third question that has been driving this portfolio since its creation has really been what programmatic interventions can prevent or reduce radicalization, violent extremism, including disengagement from violent extremism, de-radicalization and desistence. This year, what we’re doing is shifting from a balance between a call for research and for evaluation and putting much more of an emphasis on evaluation itself. So, coming back to the solicitation's goals and objectives, the overall purpose of this year's solicitation is to identify programs that will reduce the likelihood that acts of violent extremism will occur. And there, in  the final bullet you can see all  the ways in which we are looking to do that, the primary mechanism for that .

And so the aims of the solicitation are twofold. One is to support the replication and/or evaluation of existing programs that focus on preventing and intervening in the process of radicalization of violent extremism. This is something that we have looked for in the past and indeed we have funded some evaluations. We have evaluations of the work program in Montgomery County, Impact Safe Spaces Initiative, a program that is being run by an NGO called Life After Hate which has involved former violent extremists and engaging them to hopefully get people to disengage from violent extremism. We have funded a few other evaluations up to this date. So in this first aim of the solicitation, we want to continue to build this. We want to see applications that identify existing community-based prevention, intervention, engagement efforts and to evaluate them. But we are also cognizant that there are areas where there may be opportunities to create new programs. And so the second aim of  the solicitation is designed to support the development and evaluation of community-based demonstration programs where no existing programs may exist. We’ll talk a lot more about what entails a demonstration program as we continue here. Before we get to that, I just want to touch on what we mean by community-level programs. We are looking at primary prevention strategies that focus on reducing the likelihood of radicalization in communities through anti-violence messaging, education, and numerous other activities. We’re also looking at secondary prevention strategies that are directed at individuals who have been identified as being at higher risk for becoming radicalized or many other problems in communities. And then of course we are looking at intervention strategies that are intended to aid the disengagement of radicalized individuals, de radicalize those who have adopted extremist ideologies, but maybe or not are engaged in or carrying out acts of violence. So one thing I just want to flag here for you is that much of this is aimed at the what is often referred to pre-criminal space. But we are also considering disengagement and de-radicalization programs for this solicitation.

Here on the next slide is more emphasis on what we mean by prevention and intervention areas. We feel that specific activities may vary across applications but you should include many of the bullets you see here on this slide. And  here is a more detailed list of these potential activities in the solicitation. Developing a better understanding of  the ways which radicalization occurs, building partnerships, leveraging and coordinating with existing resources, developing methods for individual level risk assessment and intervention, providing professional development opportunities, education and information, enhancing communication under the rubric of engagement and so on and so forth. We understand that not every program, be it an existing program or one that has been developed for a demonstration project, will include each and every one of  these, but when we are looking at applications coming back, these are the sorts of building blocks that we think applicants should be addressing. And one, I just want to highlight is the first bullet because it is really important for us to understand what form of violent extremism and what kind of radicalization is occurring in the community that your program, be it an existing or a demonstration program, is going to address. To talk a little bit more about the demonstration process, I am going to turn it over to my colleague, Phelan Wyrick.

Dr. Wyrick: Alright, thank you, John. So, the demonstration projects are expected to use an action research approach. Action research as it says in the solicitation is an approach and a process that NIJ has used previously to address such issues as firearms violence, and sexual assault. There’s more information in the solicitation of course. But essentially, when we talk about action research, we’re talking about something, an approach that engages researchers and practitioners in this active partnership to develop more effective solutions to specific problems and produce transportable lessons and strategies that may help other localities with similar problems. Researchers work with the practitioners to track program performance during implementation, analyze findings, and relay them back to  the group. This type of approach is often ideal, where you have formative programs, you have in this case demonstration projects where these are new programs. As we’ll talk about shortly our anticipation is that these awards are going to be 24-month projects. So, not a long project period to develop, stand up, and start to implement these programs. The close partnership between researchers and practitioners is critical so that the practitioners are getting high-quality feedback and input all the way through the process from development through implementation.

The successful proposals for these demonstration programs are going to include information about an intervention that clearly specifies the theory of change. It’s critical for us to understand how the proposed intervention is going to have an impact on Violent Extremism and how also you intend to generate evidence about the program's progress toward those goals. It’s important that we have a clear sense for the evidence behind the violence prevention interventions that you’re proposing and that you are demonstrating a clear rationale strategy for selecting participants. And that’s something that has to be developed at the outset of the intervention, clearly specified in the application as well. So again we are looking for a description of the intervention components, for whom, the population for whom the intervention is designed, where the intervention is going to be delivered, who will deliver the intervention components, length and exposure to the intervention, and how successful it will be determined. In general, whenever we have projects that involve evaluation, NIJ is always looking for applications to propose the most rigorous evaluations design that’s appropriate for the research questions that are being addressed. In some cases, for more developed programs that are being evaluated for example, it may be appropriate to go towards specific designs that include a high emphasis on cause and effect and the outcomes of the program. In some other types of demonstration initiatives or projects it may be that you are looking at more formative evaluation questions. So the critical issue is to identify the important evaluation and research questions that you’re going to address in your application and to propose the most rigorous and appropriate evaluation designs for those questions. We also encourage an analysis of the costs and if possible, the cost benefits for  the research that is being done.

As noted already we expect to see a team approach, we expect close coordination with the practitioners and  the researchers, community members as well. The solicitation outlines a number of the key stakeholders and parties who we expect would be involved. But of course this will vary to some degree with every project based on the design that you are putting together and the population you are serving, and the community that you are in the proposed initial demonstration program model and a research design that is most effective in addressing the challenge that you’re trying to address and provides a strong body of knowledge and evidence out of that work. You’ll see that there is a proposed funding split in this initiative and that allows applicants to apply funding to both research and implementation costs. Through this grant, you may propose expenses related to developing, standing up and implementing the intervention that you are proposing. That includes such costs as hiring, materials, training, and so on. Also, costs for the research itself,  the costs for the research partners and the evaluation costs. As noted before  these are two year, 24-month projects at a maximum with anticipated with at least six months for planning and implementation to follow this six months.

We’ll be making  these awards as cooperative agreements. What that means is NIJ has multiple funding mechanisms that we can use to support  these types of projects. And a cooperative agreement is essentially a grant in which there is a higher level of involvement by the National Institute of Justice personnel. We will work with all of our award recipients to help develop and finalize, for example, demonstration project designs, think about evaluation and research designs as the project goes on, identify major milestones and decision points and have involvement in that process. You can see in the solicitation more information about cooperative agreements. For the entire initiative, we have up to $3.5 million available. That means we’ll be making individual awards for amounts up to $750,000 for  the total budget period, project period that is. These are up to 24-month project periods. When you’re developing your application, you should be intending for a project start date of January 1, 2017. We do announce award decisions in the fall of 2016, but there’s a number of steps and activities that all award recipients are expected to take care of, even after they’ve been notified of their award. That’s one of  the reasons why we have to start date in January. There is a possibility for continuation funding. You can take a look at the solicitation to get more information about that. The solicitation itself is a detailed document. As you might expect, it is over 30 pages long. It has a number of requirements and expectations that are spelled out in detail and there are instructions and also resources available for you to seek help and assistance in preparing your application in terms of completing all the forms and complying with expectations. You will see on this slide that we have identified a number of those items.  They’re all described in greater detail in the solicitation itself. They are all also in  the applicant checklist that you can see in the solicitation to help you in developing your application. With that, I will turn it back over to John Picarelli.

Dr. Picarelli:  Thanks, Phelan. While we still have this slide up, one thing I did want to note is that applications are due back on Monday, May 9. And while Washington is historically known as a city that if it wasn’t for the last minute nothing would get done, I want to caution you from waiting until the last minute to apply. There is a discussion towards the end of the solicitation about when NIJ will consider late applications, but it is a very, very small window. I would strongly urge you to review all of  these requirements now. Some of these requirements take time in order to put into place if you need to. And so you should be working on that now, and ensuring that you have, for example, your DUNS number and other things in place now and that you look to probably try to insert your application into NIJ no later than Thursday or Friday of the week prior to Monday, May 9. That way, if unforeseen technical issues arise, you still have some time to ensure that you get your application in on time.

So speaking of your application, I just wanted to wrap up today's call briefly with some points on preparing the technical narratives that comes to NIJ, the 30-page narrative. We normally start out with a statement of the problem and as you can see from bullets on the slide, it is pretty straightforward. The one thing that I wanted to call your attention to is the current research and major areas. We already have mentioned the SIP. Many of you have probably seen recently that an interagency task force is been created for countering violent extremism. There is also the pilot cities program and some other initiatives that constitute the broader strategy for preventing violent extremism here in the United States. In terms of research, you should review not only what NIJ has done, but be aware that the Department of Homeland Security science and technology director has also funded some research in recent years, including some evaluations of the pilot cities program in Boston and in Minneapolis, as well as Los Angeles. We want to make sure that you’re aware of this when you are applying, especially if your proposal is going to dovetail with some of these efforts.

Moving on then to the design and implementation section. With the second bullet that you see about key methodological issues, one thing I wanted to mention is a website that we maintain where we list of many of our evaluations when they are completed. That website is Crimesolutions.gov. And on Crimesoluctions.gov If you go to the about tab at the top of  the page, you will see tips on ensuring high- quality, scientifically rigorous evaluation activities that would qualify you for Crimesolutions.gov. NIJ is always aiming to ensure that our evaluations are compliant with Crimesolutions.gov so that we can list our results on Crimesolutions.gov at the end of the evaluation. What this website provides is a one-stop shop for practitioners to review when they are looking for promising practices to implement in their communities. So certainly anything that NIJ funds out of this solicitation, we are hoping will someday be listed there on Crimesolutions.gov. On potential impact, I think these are fairly self-explanatory. Again, we are focused primarily on implications for criminal justice policy and practice here in the United States. In general, NIJ, while your results may not only be aimed at the criminal justice community, that’s fine but the thing we really want to emphasize is that it is aimed at communities here in the United States. We understand that like many other issues most of what we have seen so far about prevention and intervention to radicalization and violent extremism involves not just the criminal justice community, but nongovernmental organizations, other governmental agencies and so forth. We just wanted to flag that as well. It is here in the United States.

As we round  the turn here toward the close of the call, here you will find the selection criteria which is again also listed in the solicitation. You can see we put a significant weight on the project design and implementation. We really want to ensure that there is a scientific rigor behind what is going to be completed. We then put equal weight on the impact and the ability of  the project team to complete  the work. Between those three categories, that’s 90 percent of your score. When you're thinking about how to prepare your 30 pages, keep that in mind. And then how we review. For those of you who have not applied to NIJ before, when an application arrives to NIJ on May 9th, we will scan it for what are called basic minimum requirements and those are outlined in the solicitation. Essentially we need a certain number of documents in order to actually complete a review of the application and If those are not in the application packet, we have no choice but to deny it there at the gate. The overwhelming majority, though, pass basic minimum requirements and then move onto the next phase which is where NIJ will impanel peer review groups consisting of at least two technical reviewers and at least two practitioner reviewers. They will provide reviews for each and every application that is given to NIJ. We then meet with this review panel to arrive at a consolidated or a consensus review. Those reviews are then carried forward and are joined with internal reviews that are done by NIJ scientific staff and its leadership. All of this is brought at the end of this process to the NIJ director who has full discretion on selecting applications for approval and funding. All of this information will be brought to the NIJ director who will then make the decisions on funding.

Ms. Giovacchini: At this time, we are going to touch on the Q&A in the question and answer section. If you do have a question for John or Phelan please type it into the Q&A tab and we will answer those for you, or they will answer them.

Dr. Picarelli: So again, if there are any questions about the presentation or the application process, thank you.

Ms. Giovacchini: We actually have three questions at this time.

The first question. I was referred to attend this webinar without prior knowledge of the solicitation. Where am I able to locate the solicitation and instructions?

Dr. Picarelli:   All of our solicitations are posted to grants.gov. But you can also go to NIJ’s website, NIJ.gov. At the top there is a tab for funding. Underneath that is a sub tab current funding opportunities and you’ll find it there. You can also use Google and Google research and evaluation on Domestic Radicalization and Violent Extremism at NIJ and you’ll the PDF document describing the solicitation.

Q: Approximately how many applications do you receive? How many of those are denied immediately for not meeting basic requirements?

Dr. Picarelli:   So as I said, this is going to be the 5th year of this specific call for proposals, so I’m going to speak to those previous four. We have averaged between 25 and 35 applications per year. And any that have been denied in years past, and there have been very few, often times there have been duplicate applications. We have worked with the applicants to identify which application was the intended one.  And then before we have to deny the duplicate application. I don't believe that we have actually denied any applications for lack of proper documentation.

Q: Can you review the 25 to 75 percent split requirement once more, not sure I completely understand.

Dr. Wyrick: Sure, this is Phelan and also there is a similar question just below that, that also touches on this question. And it says, The 25/75 budget split suggested for demonstration programs, is there suggested budget split for evaluation of existing programs. So just to review, we are encouraging for the purposes of this demonstration program we’re fully expecting that applicants will be proposing to support costs related to implementation and development of the program itself. So that might include personnel, program expenses, equipment, materials, training, and other activities intended to counter violent extremism. We are looking, were willing to fund up to 75 percent of the award toward those expenses in the case of demonstration programs. The other 25 percent would support essentially the action research element which would be for the research partners to support activities related to data collection, analysis, and direct personnel costs to develop reports and so on and so forth. With regards to the evaluation of existing programs, that may split somewhat differently. For example, there may be an existing program that an applicant chooses to replicate in a new location. In which case, they can use the 75 split, 75/25 split much as the demonstration project did. There may also be applications where you’re looking to evaluate existing programs. In those cases, the existing program may already have funding. And so there may be, you have latitude in terms of how you would want to split or if you want to split funding in that way. You may have a greater proportion of the resources going toward the evaluation.

Q: I am interested in knowing what organizations are conducting work on violent extremism. Do you have a list?

Dr. Picarelli: We do not have a list. There are some organizations that you can find online. There are also some of the studies that have been published mention  other programs that are out there, but we do not have a list of that nature.

Q: The focus of this webinar has been on intervention and prevention. Will any consideration be given to research focused on understanding radicalization with an eye toward developing intervention and prevention efforts but no specific plan to do so?

Dr. Picarelli: So I am going to interpret this as more of doing more of the research side as opposed to diving headlong into an evaluation. The answer is we will take consideration of those applications, they will not be rejected. They will receive a full review but our emphasis this year is really on evaluation and demonstration efforts. So it’s hard to say here before we have a chance to review all of the applications that we would have enough funding to get to research if we have a good number of high-quality applications that will meet with impact in helping us to better look at promising practices as they pertain to prevention, intervention, engagement and disengagement efforts here in the United States.

Q: Are there any eligibility requirements for PI?

Dr. Wyrick: The eligibility requirements are spelled out on the first page of the solicitation. We encourage you to take a look at that. In general, NIJ is authorized to make grants to states, units of local government, federally recognized Indian tribes, nonprofit or for-profit organizations, institutions of higher education, and certain qualified individuals. With for-profit organizations, there’s a requirement that they forgo any profit or management fee. But foreign governments, foreign organizations and foreign institutions of higher education are not eligible to apply. Now with that all said, sometimes people, and this question was specific about requirements for PI's, the principal investigator. We don't generally have specific requirements for the principal investigator. It’s on the applicant to demonstrate the capabilities and competencies of the staffing and the personnel involved in the project. Sometimes people say well does the PI need to have a PhD for example or do they need to have a degree in a certain field? That sort of thing.  And there’s no specific requirements on that. Just remember that all the applications are reviewed by external peer reviewers as well as internal NIJ reviewers and final decisions are made by the director for the National Institute of Justice. So it’s incumbent upon every applicant to demonstrate the capabilities and competencies of the team that they assemble.

Q: Is there a consolidated list of official programs?

Dr. Picarelli:   I am not entirely sure. I think we’ve already addressed that we don't have a list of intervention/prevention programs. Of official, in terms of the awards that we have made, that’s available on our website. Otherwise, I am not entirely sure what official programs are. Maybe we can try to better address that question after the conclusion of the webinar. So there is a question of would research conducted by previous awards be available? Those projects that are completed are available if you go towww.ncjrs.gov there is a, you can find the completed projects. I believe also if you go to our website and you look at those projects that we have funded already, there is a pass-through link to the final report. I will tell you that of the 23 projects that we have funded to date, only a small handful have completed. We are in the process of reviewing a number of reports. So those will be published probably in the coming months. You can keep an eye on our website and our Facebook page for when these reports are released.

Dr. Wyrick: Just to add on that, this is Phelan, I would say that this particular solicitation is somewhat different than some of the past ones in the sense that we have a two-year project period on this solicitation. In years past, the projects were often three-year awards, sometimes even longer. Many of the awards that have been made in the years of 2012 to 2015 are still underway and have not yet completed reports.

Q: There is a question and I will actually answer it if you guys don’ mind. Will you post these Q&A?

Yes. The questions and answers will be posted in a FAQ document along with a recording of the webinar. That will be posted in approximately a week on the NIJ website. You will receive an email notifying you when it is available.

Q: Do all peer reviewers have backgrounds in CVE specifically?

Dr. Picarelli: Most of our peer reviewers on the technical side are going to have backgrounds that are familiar with violence prevention, are familiar with, it may be specific to violent extremism, but we are normally looking for technical reviewers that have a depth of scientific knowledge to be able to comment on the methodology, that are able to comment on the technical design and the viability of the approach. Then, we team them with practitioners who are more familiar with the violent extremism so that they can comment on the approach and therefore on the impact. and to ensure that what results from the study will have the intended results when it is completed for the field.

Q: At this time, we do not have any further questions. If there is something -- we have one more. Does NIJ offer certification programs in regards to the topic at hand?

Dr. Picarelli: I am not fully understanding the question, but we do not offer any certification programs.

We will give you a few seconds here to ask any last minute questions. Otherwise, that will conclude our webinar for today.

Q: What is the extent of evidence expected during proposal submissions pertaining to coordination with other stakeholders, communities and organizations?

Dr. Wyrick: When you are proposing an application and you are proposing to partner with stakeholder agencies or organizations, we do request that you provide letters of support with your application.

Dr. Picarelli: To that end, I get this question frequently. There are organizations that cannot provide  those letters. There are law enforcement agencies, government agencies, U.S. attorneys offices that may have policies in place where they don't provide those sorts of letters. It is okay to discuss in the application that you did reach out to those organizations and show us at least that you are trying to make contact with them. We really do, especially when talking about a program that will be evaluated, you can understand that we would like to see a letter of support that the program wants to participate in the evaluation.

Dr. Wyrick: Likewise, a demonstrated history of collaboration with those stakeholder organizations or partners is also encouraged.

At this time, we have no further questions. I would like to thank you all for joining today's webinar. I am sure NIJ looks forward to seeing proposals from everyone. Thank you and have a great day.