Operation250: An Evaluation of a Primary Prevention Campaign focused on Online Safety and Risk Assessment

The proposed study is led by a new investigator and uses an RCT design to conduct a formative and summative evaluation of Operation250: a countering terrorism program that focuses on the role of internet safety and online risk assessment among 350 students at a series of schools in the North Adams District of Massachusetts.

The formative evaluation will address questions such as: how the Op250 logic model aligns with unsafe internet behavior and extremism, what the potential barriers are to achieving strong congruency between Op250 and theory, and what data arises from Op250 interventions and how this data corresponds to the goals of the summative evaluation.

In order to carry out the formative evaluation, the applicants will rely on a preliminary evaluability assessment of Op250 that was conducted as part of a previously funded NIJ project (through Harvard). The results of this work will inform what elements of Op250 are deemed suitable for evaluation and what sample size and students’ characteristics should be targeted to maximize evaluation efforts based on youth online behaviors.

For the summative evaluation, the research team will conduct an RCT comparing the outcomes of Op250 with a control group of individuals who did not participate in the program. The applicant’s aim will be to measure the ability of Op250 to reduce unsafe online behavior and increase the ability of the teenagers to assess risks online. A suite of measures, including an adapted religiosity scale, resiliency and coping scale, and emotional stability scale, among others, will be used to assess results in this evaluation. Questions about internet safety and “risky online behaviors” will be assessed through survey questionnaires.

Focus groups and classroom interventions with a subset of the sample population will follow and will include the presence of teachers. The sample size will be determined by baseline data on youth online behaviors previously collected in the NIJ-funded Harvard project. The applicants state that this baseline data will allow them to calculate the percentage of youth in the population targeted by the Op250 intervention that have online risky behaviors as determined by the survey questions (i.e. talk to strangers online).

Based on the distribution of such behaviors and standard deviation of the risky behaviors score obtained, the applicants will calculate the number of youth to be enrolled to detect an effect size change of score of 0.5 with a power of 80% (a power analysis is included in the application).

The proposed study adds a more generalized online safety component to existing models of counter-messaging and counter-narratives in this particular area of study. The applicants propose to provide empirical support for the development and effectiveness of Operation250 as an innovative terrorism prevention/intervention program. The applicants propose to study the exact role that the internet plays in online radicalization, in addition to understanding why only a few people who engage in online extremist material actually engage in terrorist behavior offline.

"Note: This project contains a research and/or development component, as defined in applicable law," and complies with Part 200 Uniform Requirements - 2 CFR 200.210(a)(14)."

CA/NCF