Multi-Component Efforts to Improve School Safety - Breakout Session, NIJ Virtual Conference on School Safety

 -Hello, and welcome to the Comprehensive School Safety Initiative Conference.

So glad to be here with all of you virtually today.

My name is Barbara Tatem Kelley, and I want to welcome you to today's breakout session entitled "Multicomponent Efforts to Improve School Safety." If that's the breakout session you were planning to attend, we're so glad you're here, and if you made a mistake, you're welcome to stick around because I think it's going to be a pretty darn good session.

I serve as a social science research analyst at the National Institute of Justice, and I have interest in adolescent development, the relationship to victimization, delinquency, prevention, juvenile system improvements, mental health, youth gangs and education.

In this session, we have three presentations, and in the remaining time, at the end of the session, will be dedicated to answering questions from the attendees.

In order to have your question raised, you'll need to write it in the Q and A section at the bottom of the screen, and that way, we can view it and make sure that your question gets addressed.

This is a really interesting session because, going back to a saying from a few years ago, it takes a village, and it takes multicomponent efforts to actually advance school safety in our communities because what happens in schools is influenced by what happens in the communities, in the families, in the neighborhoods, in the gangs and also in the stairwells of the school.

So without further ado, I want to introduce our first set of speakers.

The presentation is going to be made by Troy Smith, Karen Crews and Sean Kelly, who are going to share their findings with the research done by RAND researchers and school system personnel at Montgomery County Public Schools, which is a school located outside of Washington D.C., in the Maryland suburbs, as they test the effectiveness of positive behavioral interventions and support, and how the addition of restorative justice to PBIS influences those schools' disciplinary approaches and outcomes.

So without further ado, I would like to introduce our speakers on this panel.

Thank you so much for joining us.

-Hello.

Thank you for joining us today for our presentation on exploring the relationship between school climate and safety, restorative justice and PBIS.

I am Dr. Karen Crews, the director of student well-being and achievement at Montgomery County Public Schools, Maryland.

I will be presenting about our work together today with Sean Kelly, one of our instructional specialists, and one of our partners from RAND, Troy Smith.

Dr. Becki Smith is the principal investigator on this project.

Next slide.

So I'm going to share with you a little information about the RAND grant and how that came to be.

First of all, we were very excited to get this grant from the National Institute of Justice in order to test the effectiveness of some positive discipline approaches that we have been considering as a district.

Today, we want to highlight the partnership between Montgomery County Public Schools and the RAND Corporation and the work we have been able to accomplish around the implementation of restorative justice in school-wide positive behavior interventions and supports and their effect on school climate and safety.

As educators, we are critical points of contact in the student experience.

Being the people kids interact with during much of their unstructured and structured social learning time.

Oftentimes, school staff are their guides through complex social and emotional and mental health situations they encounter while trying to reach their academic goals as well.

Our goal today is to give insight into the language, practices and research methods used in this study and their implications on school safety.

From the very beginning of this project, it has been a collaborative effort between MCPS and RAND, with MCPS really specializing in the implementation of the programs and RAND specializing in the measurement and the research.

Schools in the district were offered the choice of whether to participate or not in the study, and we ended up having a total of 43 schools that opted to participate.

In order to measure the effectiveness of the RJ and PBIS grant, we've staged implementation so that about half of the schools began their implementing in the school year 2019-2020, and the rest of the schools will begin implementing next fall.

Our partners at RAND are examining the implementation, the impact and the cost-effectiveness of PBIS alone and the two programs in combination with each other.

Sean will now tell us a little bit more about how these interventions fit together, and then Troy will discuss the research design and data collection.

-Thank you, Dr. Crews.

So, yes, I'm going to go ahead and take some time now to not so much explain what PBIS and restorative justice are.

I actually had the opportunity to attend some breakout sessions this morning.

You know, some of our colleagues in these fields are doing some amazing work, but really, what this study was more based around is how the two intervention systems can actually work together and sort of amplify the effectiveness of both.

So what we were hoping to look at was how to integrate the concepts of the two together.

So starting with positive behavior, interventions and support, PBIS is...

It's one of the most, you know, accepted and commonly used forms of behavior management within public schools.

It's been researched and implemented for over 2 decades now, and Montgomery County Public Schools, we are no exception.

The, really, basis of any PBIS system in a school is, first, the establishment of core school values.

What is this community of learning going to agree is what we are all about here? And once those values are established, we establish language by which to talk about those values and how to reward the demonstration of those values through behavior norms and other observational methods.

That use of values and common language lead to the positive part of PBIS, which is the rewarding and celebrating of students that, you know, follow the values, use those common language values and behaviors, and that way, school climate is enhanced.

It has strengthened through that common sense of working towards goals that everyone holds and that safety is then, therefore, hopefully, enhanced because all stakeholders involved in this school community have a firm sense of what the community stands for, and also gets to experience the positive outcomes behind following that behavior.

So knowing this about PBIS, that brings us to restorative justice.

Restorative justice, it's a little newer, and its application to public education is still, you know, being worked out and figured out, and for this study, what we wanted to highlight, again, was the connectedness between PBIS and restorative justice.

So, quickly, the interconnected concepts that really flow through restorative justice, restorative practice and the use of circles in schools are creating just and equitable learning environments, the sense that all staff and students feel like their needs and the things that they know to be right are acknowledged and accepted for who they are when they walk through the door.

Centering relationships, and this comes into what Dr. Crews was talking about in the beginning, that importance in that emphasis around social learning that takes place in all educational spaces.

Students are quite literally learning how to do relationships, so how we foster acceptance and respect are all parts of restorative justice and restorative practices, and then, ultimately, when harm occurs, and the conflict happens, how do we handle this normal part of life? So, much like in PBIS, where we have our core values, in restorative justice, we have relationships.

Relationships are at the center of everything we do.

And then, much like our positive behavioral recognition and common language that we use to acknowledge those positive behaviors in PBIS, what we have in restorative justice is the use of circles, and first, we have community circles.

Community circles are the circles that we use to establish and strengthen and nurture that sense of the community and the centering of relationships being at, you know, the crux of everything that we do within a school, and then, our restorative circles are what we use to resolve conflict and repair harm.

So that format of using, you know, the relationship-central approach that we've established when things go wrong is much like the use of the core values of the schools and then positively rewarding them.

It's really the common idea of establishing a common center of a school culture and then leveraging the understanding and the value of that culture into results for positive school climate and school safety.

So what we're hoping to see is that instead of using one of these two strategies in independence, when we use them together that that would ultimately increase, you know, the sense of strong school climate and school safety.

And I'm going to pass it over to my colleague Dr. Troy Smith, now, and he's going to take you through how, once these practices are established, we, you know, are measuring them through other research.

-Thank you, Sean, and thank you, Dr. Crews.

So, as Sean noted, this is a study where we're trying to...

So there has been quite a bit of work on PBIS and the effectiveness of PBIS.

There's been some newer work on restorative justice, although there's still a lot that we don't know about restorative justice and its effectiveness.

As far as we know, we're the first study to look at the combination of these two approaches together.

So what we wanted to do is do a rigorous evaluation of the efforts of Montgomery County Public Schools in looking at whether these things work and how they work together.

So to do that, we have a randomized controlled trial, and as Dr. Crews mentioned, we have 43 schools in the district that are participating.

The baseline was a little bit difference.

So some of the schools that wanted to participate and put restorative justice into their program, they were already doing PBIS, and others of them weren't.

So what we've done, in order to try to get a causal impact here and try to analyze this is we have two different components of our randomized control trial.

So in our first randomized control trial, we have about 23 schools.

That's about five middle schools and about 18 elementary schools, and the baseline there is they already have PBIS.

So what we're doing is we're introducing restorative justice and seeing how we can integrate that with a PBIS program that already exists, and then we're looking at the difference between PBIS plus restorative justice versus just having PBIS by itself.

In our second randomized control trial, there were schools that didn't have either of these programs to start out with, so they had neither PBIS nor restorative justice, and what we're doing there...

and there's about 20 schools in this condition.

There's about six elementary schools...

or six middle schools and 14 elementary schools, and, there, what we're doing is we're putting PBIS and restorative justice together, and we're introducing both of the programs into the schools that had neither program before, so that's the counterfactuals.

So we have a little bit different counterfactuals, but what we're hoping to be able to test is, what is PBIS plus restorative justice, what does that add in addition to just having PBIS, and then, what do the combination of the two programs look like if you...

compared to a business-as-usual, traditional, more traditional, disciplined approach.

And as I mentioned, this is kind of the first study where we have the rigorous design to be able to look at these things and be able to look at the combination of these programs together.

If we could go to the next slide, Sean? So, how are we looking at this? We have a number of different data collection activities, and what we tried to do is look at both a qualitative and quantitative perspective.

So the qualitative data is going to help us to look at the implementation and to answer the implementation questions.

How is this implemented in schools? What did the schools find valuable? What was their experience with these interventions? And then, that quantitative data is going to help us to look more at outcomes.

So, was this effective on these outcomes that we're looking at? What is the impact of these things? How do they differ between the treatment and the control conditions? So that, in terms of qualitative data, we have done, and I should say, we're in the process of data collection right now.

We actually just started our second student survey today, but we have some site visits where we're visiting schools on the ground.

We have about nine of the schools that we are visiting in any site visits, and those include interviews, focus groups and observations.

We are visiting both treatment schools and control schools, so we can really see what the difference is between the implementation and the treatment and the control.

Back in the summer of 2019, before the district launched these programs in the schools, we did some training observations to look at what the training that the individual school-based restorative justice specialists were getting in order to implement this program in the schools.

And then, also, in terms of qualitative data, we have staff logs, and so we randomly select four staff members from each of the treatment schools each month, and we send them a little survey.

It's about a 5-minute survey that asks about the actual activities that they're doing in terms of PBIS and restorative justice.

What have they done in the last week? And this gives us some kind of continuous measure of how these programs look and what is actually being implemented on the ground in the schools throughout this study.

In terms of our more quantitative data, we have a couple of different areas that we're looking at.

The first is that we have climate and practice surveys, and these are going to both staff and students.

So the staff survey is for all staff in treatment and control schools, so that includes teaching staff but then non-instructional staff and administrators as well, and then our student survey is going to students who are in grades five and above.

So remember, this is elementary and middle school, so that's grades five, six, seven and eight, and we're surveying both staff and students about the school climate.

We're asking about bullying.

We're asking about the environment, how they feel in this environment, to try to get at some of these different outcome measures that we're hoping that the restorative justice and the PBIS produce, or at least has been theorized that these are the outcomes that we should expect from these programs.

And then, finally, we're looking at...

we're getting administrative data, and there, we're looking at a couple of specific outcomes.

So we are getting suspension and expulsion data so we can look at, is there actually an impact of these programs in terms of disciplinary measures? Are they being used instead of some of the harsher, more damaging disciplinary measures that have been used in the past? We're looking at attendance data, so if the school climate is better, then does that encourage students to come to school and to participate in the school environment? And then, we also are getting test score data.

So as John Engberg mentioned earlier today, when they were looking at Pittsburgh schools, and they found that the restorative practices that they were looking at actually had some negative impacts for some of the populations on their academic outcome.

So we want to see whether that's the case here and see if that finding that they found would be replicated here.

So we're interested in the testing score and outcome data there as well.

And as I mentioned, we're still in the process of data gathering, so we're not able, unfortunately, to give you any results.

Stay tuned next conference, and hopefully, by the end of this year, we'll have some working papers out that we can share with you some of those results and definitely if we're invited back to present, we can present more findings next time, but, right now, we're still kind of in the very thick of things.

Can you go to the next slide, Sean? So, of course, like everybody else in the world, we've been impacted severely by COVID-19.

The schools in the district have been offering virtual education since March of 2020.

They're currently planning to open the schools and classrooms to some students, so that will happen in the next month.

I don't have to tell all of you that the pandemic has been very disruptive to students, teachers and administrators.

That's true in MCPS as well as across the nation, so it's been a difficult environment, number one, to conduct a study, and number two, to push out new programs and new ways of thinking and new paradigms in the schools themselves.

I have to say that our partners at MCPS, Sean and Karen and the teams that they work with, they've been phenomenal in helping not only to pioneer.

I think that they've pioneered a couple of different things.

So, first, how do you integrate to PBIS and restorative justice together and make these programs work together? So they've done some really amazing work there that they could, you know, share with other school districts and give trainings themselves on how you put these two programs together.

As Sean said, what are the complementarities? How do they work together on the ground, et cetera? They've also done a lot of great work on what does it look like to have restorative justice and PBIS in a virtual school environment? So the beginning of the pandemic, we did a lot of thinking and brainstorming, and what does this look like if you're not in a school building every day? How do we still train teachers and administrators with these methods? What does that look like to students? What is different? What is the same in a virtual versus non-virtual environment? So I think they could also write a book or a manual on how to do these things virtually as well.

They've really stepped up and continue to do training with the schools and continue to participate and give them the skills that they need, facing the different challenges that the students and teachers and administrators are facing in an online versus in-person environment.

So because of the pandemic, it was necessary for us, on the RAND side, to also modify some of our data-collection activities.

We were right in the middle of one of our staff surveys when the pandemic hit last year so that we were not able to get as many responses to that as we would have liked.

We've also modified our site visits.

We weren't able conduct site visits last spring.

We've modified our site visits this spring in order to do them virtually instead of in person, and then we've also adjusted some of our questions on our surveys to make sure that it's clear that we're asking about the school environment and that could be a virtual or an in-person environment, either one.

So the one nice thing that I'll say for us in the design of our study in the first place is that, because we have the control schools there, and then that gives us...

at least we hope that this will give us the ability to be able to still see what the causal impact of these programs are.

If it was just a before and after treatment, then we would really be scared of how COVID is affecting everything.

It's still going to be disruptive.

We're still only going to be able to say what the effect was in the environment that we were implementing in, which, of course, includes the pandemic.

But because we have both treatment and control schools, we can see they are both being affected by COVID, and then, hopefully, we can still see what the differences are between those that have implemented these programs and those that haven't, when both of them have been faced with the difficulties of the pandemic and virtual learning and virtual schooling.

If you could go to the next slide, Sean? So we're not going to have questions right now.

We're going to have questions at the end of the session after the other presenters have finished presenting, but let me just give another shout-out to our team, so Becki Herman, who's the principal investigator here.

We've worked with a lot of people in MCPS and on the RAND side.

We've listed their names here.

Let me call out...

Besides Becki, let me call out Stephanie Iszard as well, who has really spearheaded this work over the last 2 years and put a ton of work into making sure this happens.

And then, again, let me thank NIJ for funding this work.

Like I said, I think that this is an innovative study of these two programs.

We need more data here.

A lot of schools across the nation are implementing each of these programs, and some of them are starting to implement them jointly together, but we really don't have a rigorous evidence base to tell us what we should do or how they should be implemented.

What are the pitfalls with implementing? Do they work as we theorize and as we think that they work? So I think that this, through NIJ, this has given us an ability to be able to look at this and to answer some of these questions, which I think is very exciting.

So stay tuned.

Hopefully, we'll have some results to give you within the next year or so and get some evidence on these important things.

So thank you so much, and we look forward to having some questions and discussing, and I'll turn the time over to Tim.

-Well, I wanted to just interject here.

Thank you so much for this presentation.

I was contacted by Troy early on in this process with his plea that we could please include more people from the Montgomery County Public Schools, so I'm so glad we were joined by Karen and Sean because this is just a shining example of the comprehensive school initiative effort to promote research- practitioner partnerships in a way in which both benefit with a feedback loop from the evaluation informing the evolution of your program.

So thank you so much for coming today, and now we'll move on to Tim McCuddy.

His presentation is on an investigation of school districts in St. Louis, Missouri, and on patterns of school violence and also on those factors that contribute to safe learning environments.

And he's coming from the University of Memphis, but I saw that he has previous UMSL relationship, the University of Missouri at St. Louis, which I've worked with him many years over the years with the initiative, so it's great to have an UMSL alumni.

And so I'm looking forward, very much, to hearing your presentation, Tim.

Thank you.

-Thanks, Barbara.

I'm excited to be here and to share this project that's part of the University of Missouri-St.

Louis comprehensive school safety initiative, and I'll give a brief overview of this project, and it's a bit different from the previous project in that this was quite broad, and we tried to tackle different areas, exploring the causes and consequences of school violence.

And, today, I'm going to...

After giving a brief overview, I'm going to emphasize a set of findings related to the impact of social media on delinquency.

This is an area that I have been specializing in for the past several years.

And I was talking with Finn Esbensen, the PI of this project, and we could cover so many different topics in this presentation, but we thought this was rather timely, given the kind of transition that kids have experienced, spending a lot of time online, not a lot of time in school.

So we thought it would be useful to share some of our findings related specifically to this topic.

So the UMSL CSSI is a study that explores the causes and consequences of school violence, as well as factors contributing to safe learning environments.

It's very broad, but that was kind of the point.

We wanted to...

[Audio drop] ...in year 1, which was 2017.

In addition to the survey, we conducted 197 semi-structured interviews in 2018, and then, in 2019, we did 37 in-depth interviews with these students.

The interviews focused more on the context of victimization, kind of the who, what, when, where, how and why, but, today, most of what I'll be talking about will be findings related to the survey component of this study.

We also conducted surveys of school personnel in wave one, when all the students were in seventh and eighth grade, and then, in wave three, all the students had transitioned into ninth and 10th grade, and then we surveyed the high school personnel during that wave.

And I show on the slide here a variety of the topical areas that we explored in this project.

At the very bottom of this slide, I have a link to our website that provides a series of two to four-page fact sheets that discuss some of our findings related to victimization, delinquency, bullying, avoidance behaviors, a little bit of everything on this slide.

And again, this was quite broad, but our goal here was to explore multiple contexts, multiple factors that are all related to the causes and consequences of school violence.

And if anyone has any follow-up questions related to any of these other topics that I'm not going into detail today or any other, you know, questions about the project as a whole, I'd be happy to elaborate on that.

I was the project director for the last 2 years of this study, so even though I am at the University of Memphis now, I was at UMSL at the time and am intimately familiar with this project.

So, as I mentioned, we wanted to capitalize and utilize the diversity of our setting, and I have this map here just to demonstrate the diversify of St. Louis County.

North county is characterized by a higher concentration of non-Hispanic Black residents.

There are high levels of disadvantage within that region.

South county is more affluent.

It's more likely to be white, and there's a much lower concentration of disadvantage.

So, for our study, we chose three districts in the northern part of the county and then three districts in the southern part of the county to get a really diverse sample across race, across school disciplinary actions and across overall neighborhood disadvantage within those areas.

And just to give you a bird's-eye-view outline of the core project, given this is what I will be discussing, the research findings today, in the 12 middle schools that we started, there were about 4,700 kids enrolled.

We were able to get consent from 78 percent, so about 3,600 students, and then at wave one in 2017, we surveyed 3,640 students, about 99 percent of those with consent.

At wave two, we tracked down 86 percent of those students who were now in 33 middle schools and high schools, and then, by wave three, all the students had transitioned into 16 high schools within those districts, and we were able to survey 75 percent of those who had consent, and you can see there on the slide, I'll also point out the time periods in which we did the interviews.

The semi-structured interviews were at wave two, and then the in-depth interviews were at wave three.

And this should give you some background on the sample that we are dealing with in St. Louis County.

About half the sample was in seventh grade, half was in eighth grade, about 53 percent female, 38 percent White, 40 percent Black, 11 percent multiracial, and that was pretty consistent across waves, about 40 percent White and 40 percent Black.

At wave one, most respondents were just over the age of 13.

You can see there that about a fifth were 12 and younger, and then about a third were ages 13...

or excuse me, ages 14 and older.

A little half lived with both parents.

About a quarter lived with a single parent, and then 96 percent were born in the United States.

So, again, for today, I want to focus on a set of findings related to social media and delinquency.

And I will end...

What we kind of realized, as we started along this research, there's a lot that we just don't understand about social media and violence and the ways that kids use social media in schools and out of school, how things like bullying bleed over into the school.

And we were...

A lot of this research was really descriptive because we were trying to even understand what kids were doing.

What are they exposed to online? To...We had to figure that out before we could even begin the discussion of trying to figure out what could schools to address the negative influence, the negative online influence that children may be exposed to.

So I will tell you what we found, and then I'll end with kind of the future directions that we are headed, specifically within the context of social media and delinquency.

So this slide here demonstrates, I think, a lot of information in just one figure, and basically, I am trying to show the overlap between online peer delinquency and the traditional measure of peer delinquency that criminologists look at.

So we're catching the overlap of two things here.

First, in blue, are the offline peers.

This was captured by a measure asking how many friends do you have that have done the following items related to general delinquency, violence, theft and substance use.

The green bar refers to online support for delinquency, so this question captured exposure to online messages and support of the following different behaviors, whether that's general delinquency, violence, theft or substance use.

And within the online measure, it's really important to mention that all of the online peers that we looked at in our study are online peers who are not regularly seen in person.

Now, I'm happy to talk about some issues with this measure or the fact that most of these online friends are, in fact, the same friends in the school, the same friends in the neighborhood.

But our goal was to initially start out looking at the Internet as a separate context to understand, you know, how does that differ from the school? How is it similar from the school? Again, what, you know, what are kids exposed to online versus offline? So for the purposes of the discussion today, it might be best to think of online peers or cyberspace as a context.

Just like the school in the neighborhood, we have the Internet, and that's kind of what we were doing here.

We were trying to look at exposure across those different types of peer delinquency.

So in the first column, under any delinquency, is a 13-item variety score that captures a broad range of behaviors.

About 63 percent of the sample was exposed either online or offline, indicated there by the top of the bar, and then we see here that only 5 percent of the sample is exposed only online.

About 17 percent was exposed only offline, and then about 41 percent was exposed in both context, both offline and online.

So the main thing to note here within this column and kind of across behaviors is that the degree of online-only exposure is pretty small, those green bars.

A lot of kids, a lot of our respondents who are exposed online are also exposed offline, as indicated by the red bar.

A similar pattern we saw for violence and theft, although the degree of online exposure, unique online exposure, was a bit smaller for violence.

Substance use was a bit more unique compared to the other behaviors.

Basically, the Internet exposes youth to more substance use independent of offline exposure.

So if you were to look at the green bar relative to the blue bar, sometimes I'll show Venn diagrams and kind of look at the overlap, but I didn't want to throw more figures and make this more confusing than it has to be.

The idea being that our students were exposed to little unique online exposure, and looking across crime types, there was a little more unique exposure for substance use relative to crimes like violence.

But again, most students who were exposed to negative influence online also exposed offline.

Now, I will spare you on any regression tables right now, and I'd just summarize the findings across a few studies that I have done on this.

And simply...

To put it simply, there's just a lot of mixed evidence.

When I look at general delinquency and violence, I find that online peer support is not associated with the prevalence or the variety of delinquent or violent acts, but there was some partial support for the changes in those outcomes using a first-difference model, and I was limited with those first-different models of establishing temporal ordering, which is a big issue with peer research.

I want to make sure exposure to peer delinquency preceded involvement on self-reported delinquency.

But when I look at theft and substance use, pretty much any way I model it, if I look at the prevalence, the variety, changes over time, I do see an online-peer effect.

So I have a little more confidence in the effective online-peer support for delinquency relative...

I have more confidence in substance use relative to crimes like violence.

However, and what I really, really want to stress on this slide is that third bullet point, that offline peers remain the strongest source of influence across outcomes.

Again, across models, across crime types, the coefficients, the effect sizes were much stronger for offline peers.

Cross-sectional research finds some really strong associations between online peers and offline crime, but within our data, there is much less support when we look at these processes over time.

The very first wave of data I looked at and found much more evidence of this online-peer effect, but now that we have three waves of data, and I've been, again, looking at these longitudinal models, there's much less effect of online peers, which is really important to highlight because anyone who is familiar with this literature is aware that most studies out there on this topic are cross-sectional, oftentimes using college samples.

So I think this study was unique in that we could look at a younger sample's use of social media and look at these changes over time, but there were year-long lags in between these measures, so I am looking at how exposure to, you know, social media content at time one effects behavior a year later, so, of course, there is a limitation of that.

This next finding is another just descriptive finding that we all thought was just pretty interesting, and we wanted to tackle this idea about self-disclosure.

Do students talk about things online differently than how they talk in school or in neighborhood, or at home? That is, is the content of their messages different online? So we asked a question during the...

in our survey that said, "If you did something illegal and you wanted to talk to somebody about it, would you do it online or in person?" and then we provided a seven-point scale in which our respondents indicated their preference.

And we can see here that about 66 percent of our sample prefer talking in person about their delinquency, and only about 9 percent, here at the bottom, prefer talking online, and then about a quarter of respondents, 25 percent, said that both were equally likely.

Now, we could take, you know, the quarter, the 25 percent here and the 9 percent, add them together and say that maybe about a third of respondents are willing to talk online.

Either they prefer talking online, or there is no preference of talking offline or online about their own involvement in delinquency, but, again, 66 percent, two-thirds say that they would prefer talking in person.

About 44 percent, if you just look at the top part of this figure right here, a full 44 percent said they would only talk in person, versus about 4 percent said that they would only talk online.

So the key takeaway here is that most students still prefer offline communication, and that is where most of the discussion about involvement in illegal activities is taking place.

There is a lot of literature to suggest that youth may be disinhibited when online, maybe more willing to talk about sensitive material, and to be perfectly honest, I kind of thought that I would find more of a stronger preference for online disclosure, given what I was seeing in the literature.

However, again, only about 9 percent of our respondents preferred talking online about involvement with their illegal activities.

Moving onto another area that we looked at within the discussion of social media in violence.

At wave one, we had about 314 respondents who were coded as gang-involved based on self-reported items related to self-identifying as a gang member and then whether or not they consider their group of friends to be a gang.

Based off of literature, we were seeing that a lot of gang members were active online.

In fact, some research was suggesting that gang members were more likely to prefer talking online and then made friends easier online.

And given the data that we had, we thought it would be really interesting to see if that was the case and then try to examine if online peers had a differential effect on gang members relative to non-gang-involved youth.

So first of all, we found that, yeah, online postings about offline illegal activity are more common among gang members.

We had another item in our survey tapping into when our respondents first learned about their friends' illegal activity.

Was it online rather than in person? And we found over half of gang members reported first learning about their friends' illegal activities online, compared to about 22 percent of non-gang members.

So again, that one, I thought, was quite interesting.

And if gang members did something illegal and wanted to talk to their friends about it, they were more likely to prefer talking online relative to offline.

The findings I showed in that last slide, small difference, 16 percent versus 9 percent.

It was significant, but subsequently, not too different.

But across all of our measures, gang members were more likely to make friends online, prefer talking online.

They have more offline friends who are online.

So across the board, there was more social media utilization among gang members, but in our regression models, trying to look at the effects or look at the association between exposure to online peers and then self-reported delinquency, we found that online peers do not have an additional influence on gang-involved youth above and beyond the influence of offline peers.

We saw the pretty much same associations for non-gang-involved youth as we did for gang-involved youth.

And then finally, I want to touch on an area that the principal investigator, Finn Esbensen, and I have been exploring now for a couple of years, and we really wanted to look at the link between bullying, victimization and delinquency, and for my purposes, I was especially curious about the link between cyberbullying and delinquency.

Finn and I previously used data from the G.R.E.A.T. evaluation, and that study demonstrated an observed association between cyberbullying and delinquency using longitudinal data.

Some have argued that cyber-victimization is a more potent form of strain, that it could, perhaps, have a stronger link between bullying victimization and delinquency compared to school victimization, and we wanted to test that.

We wanted to see if that actually held true.

However, there were some recent redesigns to the NCVS, the National Crime Victimization Survey, that made us ask several questions about operationalization.

Could all these different findings we're seeing in the literature, some supporting with the association between cyber-victimization and delinquency, some not...

Are the mixed findings related to measurement, or is something, you know, theoretically different, you know, about cyberbullying? And that's kind of what we wanted to look at.

So we were inspired by some other work to examine mismatch between bullying items.

So we had two sets of items in our survey, a generic item asking, "Have you been bullied at school? Have you been cyberbullied?" and then we have a behavior-specific approach, and that took the form of asking respondents, "Have any of the following things happened to you?" and have those items listed on the slide.

This was the approach used from the NCVS from 2005 to 2015.

Prior to that, they used the generic approach that we also included.

Starting in 1999, they were using that approach, but in 2015, the NCVS decided to do a split design, where they were testing the effects of providing a definition to respondents and then having respondents indicate whether or not they were being bullied.

So again, we just had the generic measure and the behavior-specific measure in our study, and we found, focusing attention on school bullying, about 17 percent of the sample said they were bullied using the generic measure.

About 51 percent said that they were bullied using a behavior-specific, and then we created the category of victimization.

About 49 percent said that they were not a victim of pure victimization or bullying.

About 35 percent mismatch.

That means they said, "No, I wasn't bullied, but I had one or more behavior-specific victimizations happen." And then about 16 percent of the sample matched where they said, "Yes, I was bullied, and yes, I had one or more of these victimizations happen." So about twice as many students mismatch compared to match on the school measure.

And then for cyberbullying, which has a lower overall prevalence, the degree of mismatch relative to match was three times as large.

About 14 percent of respondents indicated that they had mismatched this victimization, no on cyberbullying but yes on one of the behavior-specific items, compared to about 5 percent who matched, who said yes on the generic, yes on the behavior-specific.

So this got really messy really quickly, and we felt quite strongly, or at least I felt strongly, that we needed to resolve this before we could even start analyzing the causes and consequences of school violence or of bullying.

You know, we need to understand what is bullying? Does operationalization matter? So yeah, that was our goal.

We wanted to figure out this measurement issue.

Does that explain the mixed findings in literature? And we were a bit surprised that we found no evidence of the association between cyberbullying and delinquency using longitudinal.

Despite any way we modeled it, there was no cyber effect.

There was if we looked at cross-sectional data.

Similar to what I said before with peer delinquency, an argument can be made that the effects of victimization are contemporaneous, that a year lag between measures is simply too long.

So we used one wave of data to analyze the associations found stronger evidence of an association between cyber-victimization and delinquency, but again, those models can't control for temporal ordering, which, of course, was a problem.

So in our attempt to understand the implications of measurement, we found virtually no effect of cyberbullying victimization on delinquency.

So again, that was not...

That's not what we predicted.

And, I think, a finding from one of our fact sheets can shed some light on this issue.

This here is one of our fact sheets on bullying measurement, and I'm capturing the overlap between cyberbullying and school bullying using the generic measure.

So, have you been bullied at school? Have you been cyberbullied? And here's what I found, and I'll point out that this is not necessarily a groundbreaking finding here.

Other scholars have found very similar results.

And basically, most of our students who are cyberbullied were also bullied at school.

Seventeen percent said that they were bullied at school.

Six percent said that they were cyberbullied, so this Venn diagram is catching the overlap among victims.

So among victims of bullying, 28 percent bullied in both contexts.

Among victims, two-thirds only bullied at school.

Among victims, 7 percent were bullied only online.

That 7 percent right here corresponds to 1 percent of the sample.

So, in other words, only 1 percent of our sample reported they were only bullied online, but again, that is relying on the respondent's understanding of what it means to be bullied.

You know, I could show Venn diagrams of behavior-specific responses.

It gets messier, obviously, pretty quickly, but the same general phenomenon we observe across items, that most of our students who were bullied or experienced peer victimization online also experienced it at school.

And that is not to suggest that cyberbullying does not have harmful effects, but in our sample, there was that large degree overlap and the school victimization appeared to be driving that association between victimization and delinquency.

So conclusions as it relates to the topic that I have been emphasizing today, social media and delinquency, we still have a lot to learn about online influence.

Now, more than ever, I'm convinced that it requires interdisciplinary mixed-methods research.

This study addressed some unanswered questions, but it raised so many more.

And I think a key takeaway, and I'm going to wrap this back into the school context, is that fear of online influence may be greater than the risk.

The impact of online delinquent peers and cyberbullying appears to be driven by offline processes.

Again, I initially predicted much stronger associations, and I soon realized we're probably overestimating the online effect due to selection, effects attributed to offline peers.

For example, those who have offline delinquent peers are likely to have online delinquent peers.

But I want to stress that this study does not look at social media influence among in-person groups.

So your best friends that, or child's best friends that are at school, they may not...

this is missing, that level of online influence, again, because we were focusing on the online context being a unique context.

So that's the obvious, I think, next step that we need to do is look at how, among students in school, you know, what are those peers saying online? How is that different from what they're being exposed to in school? But the next steps with what I can do in this data is look at if this online context, if what exposed to, this peer delinquency bullying, does that vary by school, and if so, what school-level factors explain online influence? Prior research has shown that school-level characteristics can condition peer influence.

Well-connected schools can reduce negative peer influence, and descriptive analysis has already shown online peer influence varies in similar ways as traditional peer influence, but we have yet to uncover why.

So again, what school-level characteristics explain this variation? And I hope that will help lead to more concrete policy recommendations, that is, what can schools do to reduce the harm related to online influence? Still have a lot of work to do, but I'm really curious to see what we uncover moving forward.

Thanks.

-Thank you so much, Tim.

That was really fascinating.

I'd be interested to have a conversation to see how it is that this research plays out during the period of COVID in terms of increased time with kids being online exclusively versus in person.

So moving on, though, before we get to questions and answers, to Dr. Stephanie Hawkins from RTI International.

Stephanie has been a contributor to the juvenile justice field for many, many years, particularly focusing in on work that we used to work on together on girls in the juvenile justice system, and she's now going to discuss her evaluation of Shelby County Comprehensive School Safety Initiative, which was designed to reduce school violence and student misbehavior while minimizing exclusionary disciplinary approaches such as suspension and expulsion.

So with that, I turn it over to Stephanie.

Very glad you could join us.

-Thanks so much Barbara, I appreciate it, and I really am delighted to be here to share sort of what we are working on in Shelby County, and, Tim, I noted that you're now at University of Memphis, so this is very close to your home.

But, you know, I think I'll start off by just acknowledging that, even in the midst of COVID and in the midst of schools starting to return, we need to really focus on issues related to suspension as well as gang issues, which we've heard a little bit about in a previous presentation.

So just to give you a little background, you know, in the United States, we have about 60 percent of middle schools that report disciplinary actions, specifically suspension, expulsion or transfer, in response to some type of violent offense at the school.

Now, when we take a look at Shelby County middle schools, and just to orient everyone.

Shelby County, at least the one I'm referring to, is in Tennessee, and it encompasses mostly Memphis, the city of Memphis.

But we see that during the 2018-2019 school year, the suspensions and expulsions were higher than the national average.

So specifically for middle school students, we see about 8 percent of that student population received one or more out-of-school suspensions.

About 13.8 percent received one or more in-school suspensions, and then we have about 1 percent receiving expulsions from school.

So this is an issue for Shelby County schools in general, and, you know, I'm really grateful for the funding mechanism that was offered, where we get to team and partner with practitioners, and in this case, it was the school district because they got to implement the things that they are expert in, and we got to team with them and wrap an evaluation around what they're doing, so it really was and it still is a very fruitful partnership.

And so, in our early discussions that led up to the proposal, we decided it was important to build on what they had was a very successful diversion program, as well as a pretty robust gang prevention program, all implemented at the high school level.

So in our conversations, we looked at the data, and we realized, you know, if you really want to impact a reduction of problems, specifically that you're experiencing at the high school level, well, let's focus in at the middle school level.

And so they agreed, and they decided, "Let's try and implement these same programs at the middle school level and sort of reframe the language around it because, at the middle school level, we're not implementing a traditional diversion program, but what we can do is implement a suspension-diversion program because we already knew that issues of suspension were prevalent at the middle school level, so that's what we did.

We took their SHAPE program, which was designed for traditional diversion at high school level, and made it a suspension-diversion program, and then the same for their gang intervention program.

We brought that down to middle school as well.

So again, this is a part of the Comprehensive School Safety Initiative, and we are working with Shelby County Schools, which is the 25th largest school district in the United States.

So this is an overview of the design that we had, so our focus, really, was on a randomized control trial, and so we wanted to see if we selected 24 middle schools that don't have any of the current programming that I just mentioned, could we randomize them into one of three conditions? So business as usual, continue prospering with whatever you're doing at the school district.

But then, we randomized into a set of schools that we implemented the SHAPE program and the GRASSY program, and I'll talk a little bit about those in just a moment.

And then we had...

And we call these student-focused programs, and then we had what we termed a comprehensive focus, where we had both SHAPE and GRASSY, but we also had a program called Safe Corridor, which was positioning law enforcement in strategic places on the route that kids walk to and from school.

We also had discussions about ways in which we can share data across systems, and then we also had an idea that we would really influx youth mental first-aid training into this comprehensive school condition.

Today, however, I'm really going to focus on the 16 schools that were receiving the suspension-diversion program through SHAPE and the gang prevention programming through the GRASSY program.

As I mentioned, both of these have been expanded from high schools and are now being implemented in middle schools.

And we really wanted to understand the strategies that are being used to successfully implement these programs at the middle school level and also understand the challenges for program implementation.

So hopefully, NIJ will host one more school safety conference.

Hopefully, it won't be virtual, and at that point, we would welcome the opportunity to really describe all of the quantitative findings, and those are things that we're actively working on that really look, like, are their differences across these different conditions? But today, one of the things I decided to really focus on is, if you're developing a comprehensive approach, let's talk about implementation because that is the place in which you're success will either thrive or not.

So the outcomes that we are focusing on in general at the school level are ways in which we can improve the general context of safety at the school, ways in which we can increase attendance, increase academic performance and decrease disciplinary violations that do result in suspension, but also at the student level, increasing attendance, academic performance, decreasing suspension and decreasing gang activity.

So generally, those are the targets that we were focusing on with these programs, and just to sort of give you the background for the SHAPE program.

And I mentioned earlier that this was a program designed at the high school level, and it really was a traditional diversification program designed to reduce the number of students that are transported to juvenile court, and because this program is operating very well at the high school level, we decided to move it down to middle schools and focus on suspension.

And in this case, you know, instead of students being referred to the program through law enforcement, now, we're removing law enforcement from the disciplinary issue, and we're actually recruiting students from schools and administrative staff, teaching staff, are the ones who are making the referrals to the program.

So as you can imagine, we also had to change the signage and the recruitment signs for this program because, initially, they were very sort of criminal-justice oriented.

You know, "Hey, if you engage in behaviors that could potentially allow you to find yourself in front of juvenile court, here's a program that you can participate in." And so we realized, in terms of implementation, using those same signs in a middle school, A, parents are not wanting their children to be involved because it doesn't seem like something they want their kids to be associated with.

So there was a lot of learning about how to move these programs from the high school level to the middle school level.

But just a quick overview on the GRASSY program, which is the Gang Reduction Assistance for Saving Society...

It's a long name.

We call it GRASSY, but again, long-standing program in Shelby County, but this program is designed to address the needs of students who are at-risk or involved in gangs, and it combines a number of approaches that range from prevention, intervention, outreach, suppression.

It focuses on...It's more of an individualized program, so it really meets the student where they are.

It also encompasses discussions with families, really thinking about what the quality of life concerns are.

But the way this program works is that the Shelby County School District hires vendors to deliver the program within the schools during the day, whereas SHAPE is an after-school program.

So just to give you a context of gangs in Shelby County, I was unaware, but we might know Memphis, Tennessee to be home of the blues, but it's also home to various Chicago and California-based gangs, as well as gangs that started within Memphis.

So these gangs and gang-related issues infiltrate the schools.

There are currently over 182 known gangs, sets, cliques, and they're growing.

There are over 11,000 gang members.

So this is a serious issue within the Shelby County School District and one that they've been dealing with for quite some time.

Just as an example, you know, there was a quote from one of the gang intervention specialists from one of the members that said, "When you come to Memphis, we're going rally right across from you.

It's going to be Young Mob, Crips, Bloods, GDs, Vice Lords, Goon Squad, every gang you can think of.

You come to Memphis. We're going to be waiting on you.

It's versatile down here.

We've got every gang you can think of." So this is the context for which the Shelby County Schools, in addition to their primary focus, which is educating young people, they also have to deal with gang involvement within the schools, but also protecting students who are siblings to gang members and figuring out ways to protect and encourage them to not participate actively in gang life.

So what are the characteristics for all of these different programs when we think about implementation? Well, for the GRASSY program, several of the GRASSY implementers are prior gang members or have experience with gang life, and this actually resonates with the students, and it helps them relate to that person on a really specific level, so it's not like a teacher just telling you, "Don't become a gang member," but it is someone who can meet with you, understand the pull to become part of a gang because they can talk from their personal experience.

Now, for the SHAPE program, the type of facilitator or implementer, the best type to deliver this program is actually a school staff member, however, one that does not have commitments to other after-school activities.

So, you know, there's a lot of pull on coaches and music teachers who have really good relationships with students, but what we found in Shelby County is that it's good to have that familiarity but not be taxed with other activities.

So the idea is that these teachers will have the familiarity with the disciplinary and behavioral conduct within a given school, someone like a behavioral specialist, but not also be the coach of the school.

They're really trying to both leverage the relationship that the staff member may have with students, but not at the sort of in these extracurricular activity type of ways.

They found that that is not actually helpful for this type of program.

However, for both programs, they found that an effective implementer is someone who is willing and able to communicate with the parents about what is happening with the student.

So as you all probably know well, working with young people is one thing, and working with their family members and guardians takes a different level of skill, and that is required if you are going to really address issues of gang membership and disciplinary problems that might result in suspension.

So being able to engage with family members is critical.

So let's talk a little bit about how do you...

How do implementers really meet the needs of these students? So we talked about the characteristics, but what we found in talking with, and this is through focus groups, the ability for both the GRASSY and the SHAPE workers to be flexible to the needs of each student is really critical.

You know, students enter these programs with a whole bunch of needs, varying needs, and so the ability for the implementers to be flexible and figure out ways to address a core set of needs for these students really does contribute to the success of the project.

And you can see on the quotes, one of the implementers said, "Every student is different.

I don't have a specific method because it depends on where they are.

Is it that they're missing love from home? Are they getting love from this gang? We work through these individual steps," and that implementer has to tailor it to the student.

He says, "I can't fuss at all the kids the same way, so I don't have everybody at the same starting place.

This is my starting place and it's based on restorative justice." So that flexibility really is critical, and the same is true for the SHAPE implementers who do the after-school programs.

So I won't read because I'm sure you all can do that, but really honing in the point of flexibility.

So if we take a different look and look at the characteristics to successful implementation overall, you know, it's buy-in, and this is known, right? If you don't have that buy-in, it's a complete barrier to successful implementation.

So just noting a few that we found, it was challenging to get buy-in from school administrators because of the turnover with schools.

The program could be successfully implemented one year because it's under the direction of an administration that was invested in it and familiar with it, but then, let's say that principal leaves the school, and so it's that relationship-building the next year that can either help with implementation or completely challenge it.

So you see one of the quotes from a principal who was saying, "This is my first year as a principal, so I'm really trying to map it out to be a lot more effective for me this upcoming school year, and then I really was unclear about the GRASSY program.

I just know that, one day, a person showed up, and so as I did my investigation in terms of 'Why are you here?' he kind of outlined to me what he was supposed to work on." So, you know, the communication with staff, the buy-in from staff, again, will make or break the program.

So when we think about what the challenges to successful implementation are, continuing the timing of the program.

It really did impact some of the implementation.

So I mentioned that the SHAPE program is implemented after school, and this posed a few logistical challenges for some youth.

Now, remember, this is a diversion program, so these young people are on the brink of suspension, out-of-school suspension, so participating in this program is a way that they would not be issued that out-of-school suspension, but transportation is a real logistical challenge.

So, a lot of times, parents could not take time off work to pick their children up from an after-school activity.

So these young people typically rode the bus to get to and from school, but because they can't take time off, some young people can't participate in the program.

If you can't participate in the program, that suspension goes on to be the actual suspension.

Time of the day: So, summertime, no problem.

Young people can walk home from school.

Most of these schools were centrally located, but winter is a different story, so the sun sets earlier.

Some of the students having to walk home from school in the evenings when it's dark presents a safety challenge, so again, that became a logistical challenge for some of those students to participate.

The GRASSY program was different because, again, it took place during school.

So you see, one of the SHAPE workers just sort of gave a quote about where their school is located.

It's not in the heart of a neighborhood within hours...

And then, with the hours being after school, these young people really need transportation.

So it's a disadvantage for parents that don't have transportation, and this person acknowledged that some of the families are homeless, so it was really difficult for them to arrange rides or get the students from here to there.

So now, we're battling with the challenge of there's an opportunity for a young person to participate in the program and not have the out-of-school suspension, but then there's the logistical challenges of them participating, so they're sort of, oftentimes, you know, between a rock and a hard place.

So let's talk about, well, what have we learned? You know, there are a lot of lessons learned from implementation during this Shelby County School District study.

To effectively engage youth, you really have to work with implementers who have experience, particularly with gang intervention, with gang activity.

A lived experience, there is no substitute for it, and it really makes the rapport development with students so much easier, but the same is true for the SHAPE coordinators.

Their rapport development with students is critical to the success of the program.

There's also the finding of more communication, so communicating with the administration about the programs, how to leverage the program in ways that help the school and students align with their academic priorities.

There's a quote here that talks about, "Well, Mr, and we won't say his name, he knows all that, so he can relate.

He knows stuff that we don't necessarily know," and so those kids have all his energy.

"He's able to talk to them in a soothing voice that helps them understand that he's been where they are, and he knows that there is a way you can turn all of that around," so this is the lived experience quote that we were talking about with regard to engaging youth and the critical role that that plays, but let's talk about some of the strategies, so engaging students with multiple mediums.

I think that has never become more important than now during this time of COVID.

It's really critical in reaching students and maintaining interest in programming.

So, of course, there may be limitations that schools have with the ways in which you can communicate with young people, but to the extent it's not just a person sitting in front of a student or groups of students and just lecturing and talking at them, but engaging them with videos or music or changing the ways in which they're or the places in which they meet has been really important, other strategies, as you can see, using real-life scenarios and role-playing, navigating decision-making when you're in a compromised position.

Doing the generic, "These are ways you enhance decision making," may be good for some students, but really, what's most important as we found was, how do you navigate those decisions when you're in a compromise when it's really tough? Like, when your back is against the wall, learning how to make decisions in that context is critical because that reflects their real-life scenarios.

Also, varying strategies based on gender, the school district found that boys may respond to more active strategies.

So, of course, I would caution because Barbara mentioned that we did a lot of work on girls.

I would question making generalizations about that, but being specific to what's relevant, and what are you learning from your specific school district or your specific group of students? So, in fact, boys may respond to more active strategies, but, in some cases, that may be true for girls as well.

And then, last, the SHAPE and GRASSY implementers recognize that offering rewards to students for good behavior is key.

Now, we all know, you can't use grant funding to purchase food, so, as you see, some of the implementers said, "Because food is a strong motivator, even though I only do it," meaning the programming, "About an hour a day, and sometimes it's longer than that because a lot of these parents forget that they have children, and they don't come to pick them up, so I do rewards that I pay out of my own pocket." So, you know, if we think about culturally responsive programming and for students who don't have a lot of resources, students who are after school, food is critical, and so, you know, these implementers are finding ways to meet the needs of their participants by offering rewards, and that's often, and it shows up as food.

In thinking about considerations for implementing similar school safety programs in other places, we've talked a lot in Shelby County about, what would it mean to replicate this work, and how would we be able to do this in other places? So just bringing everything home, finding the right person who can build that rapport and relate to students is important.

Implementers from all interventions indicated that they would like more opportunities to share information across programs.

So the GRASSY implementers were doing their work, the SHAPE implementers were doing theirs, but they often didn't have opportunities for cross-sharing and learning, and that's critical.

That's a missed opportunity.

So that's an opportunity for, as the district is thinking about continuing these programs, that they create the space for these implementers to come together because, often, the same young people are on both of their caseloads.

Violence prevention and intervention programs at middle schools should consider identifying the behaviors that are indicative for future behavior problems, so that's really what the whole point of this is.

Let's bring the interventions down to the middle schools so we can stop the trajectory of them continuing these behaviors in high school.

We talked about multiple modalities and then just figuring out the best way to implement these programs in the middle school context.

So we were chugging all along.

This last school year would have been our last school year, and then everything changed with COVID.

So I have just a few slides just to add to the conversation.

Well, what does school safety mean in a virtual environment? So, now, all students, at some point, became virtual students, working from home, and I was still really interested, and our team was still really interested in understanding, well, all of these students who had challenges or who were involved in gangs are still engaging in that, so what does school safety mean for Shelby County Schools when you are now in a virtual environment, and there are some things that I thought would be helpful to really think through, perhaps for future studies.

So schools address the access to hot spots and Chromebooks, you know, but about the suspension and gang supports that were in place? So we have to really think about what else is next to really support these students.

And so, you know, the reality is that school and home are the same place.

So when we think about the home life that the gang intervention specialist were working to get these young people to think about, how do you get supports at home? Well, now, school is home, and so really being mindful about the supports that students need at home is important, so when students aren't showing up and turning the camera on or acting out, if a teacher takes them out of the classroom, you know, on some levels that's akin to suspension, and is that really the solution? So just ideas to think about, but I know we have Q and A coming up, so I think I'll end there and allow for the Q and A to start.

-Thank you so much, Stephanie.

That was really fascinating.

In particular, I loved the discussion of the implementation science and how simple things like transportation and dark streets and lack of parents picking kids up or lack of safe bus routes can just derail a program concept.

I was also really fascinated with your discussion of the people having the experience of getting knowledge to service the rapport-building individual with these individual kids.

So we have one question posted so far in Q and A, and it'd be nice if all the speakers would go ahead and put themselves on video so that we can see all your handsome and beautiful faces, and then I'm going to go to the Q and A.

So there was a question for Tim.

"Was there any data regarding females versus males? I would think that girls are more likely to be bullied online.

I'm not a teacher, but that is what I hear from all my colleagues and friends in schools." We can't hear you, Tim.

-Can you hear me now? Okay, good. Sorry about that.

-Yes, thank you! -I don't know what was going on.

I wasn't muted there.

So, yes, I think there is good reason to believe we would see gender and racial differences across peer influence and across bullying victimization.

I was trying to pull some descriptive tables that I put together a while ago.

We haven't formally tested the gender or racial differences, but based off traditional research, we would expect these differences.

Females relative to males were more likely to prefer online communication.

They're more likely to be exposed to drug use online, which actually mirrored the offline findings as well, but I think one interesting gender and racial issue to keep in mind is related to some of the mismatch findings I was talking about.

There was a study by Nadine Connell and her colleagues that looked at this idea of mismatch, and essentially, she found that males and Black respondents were more likely to fall into that mismatch group.

Whether it's this idea of appearing tough or there's some stigma associated with acknowledging one's own bullying victimization, that there could be racial and gender differences in terms of why certain youth may or may not report their victimization as bullying.

So I think that is a really important methodological issue that we're going to need to deal with before we can start looking at causality and start to understand whether or not that there are these differences that are actually affecting the association between...

in my example, like victimization and delinquency.

We need to understand are the individual differences related to, you know, these differences and identifying the correlates of victimization or the correlates of exposure to peer delinquency.

Again, in terms of the peer delinquency, it seems to mostly match.

If kids are more likely to be exposed offline, they're more likely exposed online, but again, I haven't formally kind of dived into that.

But looking at that in addition to how these things vary across school context are kind of my next areas of how to extend this work.

-Thank you, Tim.

We have a new question, and there's also a thank you from the person asking the question.

So the next question is more directed at Montgomery County Public Schools and RAND.

"I know you stated that you're waiting on data, but do you see any positive impact on disproportionality in your district?" -Yeah, so again, like, we'll all be able to look at that when we have the full administrative data, but let me turn that over to Sean and Dr. Crews to give their impressions.

They're on the ground looking at this, so they may have a better idea of what this looks like on the ground right now, and then we'll come back to you later when we have the full data that we've analyzed.

But, Sean, Dr. Crews, do you want to jump in? -Dr. Crews, I hope you don't mind if I go first.

So, while we are still waiting on the data from this study, what I can say, as someone who is in schools, interacting with schools on a daily basis, you know, I think the biggest impact that we have been able to have since not just introducing restorative justice, but then demonstrating how restorative principles can enrich a PBIS program, which is already in most buildings.

What we're doing is we are creating conversations and creating very real alternatives from the suspension.

A lot of like what Dr. Hawkins was talking about in her presentation, we all know suspension is a disproportionate discipline practice, so the more options, the more conversations that principals and administrators have around alternatives to suspension, I think, inevitably, we're always going to see a positive impact on disproportionality, and that's also what we're hoping to see with the data from our study as well.

-Yeah, and I'll just add, I completely agree with Mr. Kelly.

Hopefully, everyone can hear me.

You know, we are really making sure that we have spaces that we're creating for not just the principals but for school leadership teams to have those discussions around disproportionality and, as Mr. Kelly stated, alternatives to discipline.

And so I believe that, you know, that is going to definitely change the way that we view students, which hopefully will also reflect in our data that there will be a change around disproportionality.

-Thank you very much for those answers and sort of building on a bit to the whole area of disproportionality and disparities.

The next question is, "Since the national spotlight on social injustice, do you see a large buy-in in school-based staff and in the families that focus in on social injustice?" So I open that up to the group.

I don't know which of you would like to respond first.

-I'll take a stab.

I will say...

and so I'm not sure if the question...

Well, I'll answer from the perspective of this study in Shelby County Schools.

I have to say, I've been so pleased with Shelby County Schools and their focus on families, students and families sort of before the national spotlight because that school district is predominately Black, I think maybe 97 percent Black and Latino or Black and Latinx, and so there's always been a focus in our conversations on sort of access and ways in which to sort of work with families, and just one, for example, during COVID.

They actually created a COVID hotline in addition to their regular school tip lines to really address the stresses and the needs that families have when, all of a sudden, they're all at home with their kids trying to support education.

So I don't know for the other panelists if they've noticed a difference with the national spotlight, but at least for Shelby County Schools, I think it just helps to facilitate what they were already doing.

-Did anyone else care to comment? -Oh.

I'll just say that it absolutely has in Montgomery County Public Schools.

We've, you know, since last spring, in the social justice movement, really got into the full swing of, you know, being the restorative justice unit and, you know, district-wide in Montgomery County have really started to take a look at how we can work to be an anti-racist system, you know, for our students and families, and, you know, I think that the community support and, as the question stated, that buy-in, you know, is certainly there now.

You know, I don't know if it was there before, but, you know, there's certainly a lot of community energy and interest around moving in that direction.

-And we have one more question, but I'm afraid that we are running out of time, so I think that one will have to pend.

I will just read it, but I don't think we have time to answer it.

"How could a local district partner with a corporation like RAND to secure this program for their communities?" I think that's a question a lot of people would like to know, and listening to these collaborations between researchers and practitioners, it's gratifying to see how well you're working with your jurisdictions and how well practitioners are engaged with you to get the research questions they need answered asked in the first place.

So on behalf of the National Institute of Justice, I want to thank each of you for very clearly presenting what you're working on and how you can bring together different components across your communities to help better serve these youths.

And I hope that all of you will be able to tune in tomorrow.

Our opening session is a plenary session focused on Sandy Hook, and it's going to be starting at 11:00 Eastern Time which I know, for those of you on the West Coast, is more like, what, 8:00.

We tried to be gracious with the start time.

So please join us tomorrow and thank you so much for attending today, as well as presenting, much appreciated.

And thank you to Jennifer, our tech support, for helping us get through this without showing any bad effects.

Thank you, Jennifer.