Student Mental Health and Trauma - Breakout Session, NIJ Virtual Conference on School Safety

 >> Good afternoon, everyone, and welcome to this afternoon's breakout session.

The theme of our session this afternoon is student mental health and trauma, and we will have the opportunity this afternoon to hear from three different projects on this theme of student mental health and trauma.

So we'll hear from Kristina Childs and Sara Bryson looking at a implementation evaluation of -- Pardon me, Evaluating the Implementation of a Multicomponent, School-Based Mental Health Program in a Southeastern state.

We'll hear from Marilyn King and Johanna Bertken talking about school and family engagement, trauma-informed project, their research findings and their path forward that they have taken.

And then finally we'll hear from Stacy Overstreet, who's going to talk to us about a project where they're using trauma-informed approaches in schools to foster resilience among urban Black youth who've been exposed to neighborhood violence.

So I will turn it over to our first set of presenters, Kristina and Sara.

Kristina, you are on mute.

>> Okay.

Sorry about that.

Am I okay now? Yes, okay.

Hi, everybody.

Sorry about that.

My name is Kristina Childs, and I will be presenting today about some lessons learned that Sara, my copresenter, and I experienced as part of our work on evaluating a multicomponent, school-based intervention.

So just to give you a little bit of background, Sara and I led an evaluation of a comprehensive mental health initiative in five schools in Brevard County from 2015 to 2017.

The program included several components with different targets of intervention, teachers, students and law enforcement officers and also a variety of stakeholders from the law enforcement agencies, from the school board, from the administrators at each school and our behavioral health partners.

And so when we started this project, we had high hopes of carrying out a rigorous longitudinal evaluation, and then as the program implementation took off, we kind of quickly realized that a rigorous evaluation was not a priority among many of our stakeholders and partners.

As implementation progressed, the evaluation team sort of maintained this clear outsider role in the effort, and we sort of strongly feel that that led to some of the shortcomings of our evaluation, and so today we wanted to highlight some of the key lessons that we as evaluators learned about the importance of collaborative and consistent communication, which we call the three Cs, and we'll get back to that at the end of the presentation, that we have integrated into our work since this project and have found most of them to be successful.

So the program -- The overall goal of the program was to improve school climate and safety, and implementing different mental health initiatives to target the mental health needs of students in the schools was the overall objective of the programs.

The project included five intervention school located in one school district, and it was four elementary schools and one middle -- high school that created a feeder pattern.

We placed one licensed social worker in each school, two in the middle and high school, one of the middle school students and one for the high school students, and these social workers are really responsible for carrying out all of the program components, so the program components involved the social workers being trained in Youth Mental Health First Aid as instructors and then delivering the 8-hour training to all of the teachers in the school, developing a crisis intervention training model for the law enforcement officers in the school.

At the time, the schools did not have full-time school resource officers, and so we were targeting the law enforcement officers that were sort of assigned to those precincts and that were involved in issues in the school but also outside of the school in the community.

And then the social workers received referrals from the guidance counselors and the principals for high-discipline students, and the social workers worked with the students and their parents to complete the CANS, which is the Needs and Strengths assessment, and then the young youth through sixth grade completed the 18-week Zones of Regulation program, which targets emotion regulation, and then for the younger and the older students, the social workers matched the CANS results with community-based referrals and then coordinated care throughout.

And so just some, you know, broad study outcomes, we targeted three different levels.

We wanted to measure whether or not targeting the mental health needs of students and training professionals in the school would lead to school-level changes and perceptions of school safety across both teachers and students, and so we -- Every semester, we surveyed all of the teachers and all of the students in each of the school and then also surveyed students at three comparison schools, and unfortunately our results suggested that...

I'm sorry.

I'm right here.

That our results suggested that there were no significant differences within or across the schools.

The second piece that we targeted was the youth that were referred to the social workers and participated in the Zones of Regulation and community outreach program.

We interviewed students at baseline prior to involvement with the social worker and then after completion, at 6-month follow-up and at 1-year follow-up, and then we also assess change in the CANS scores from baseline to transition, and we also found here that there were significant improvements in academic achievement, suspensions, delinquency and all of the CANS domains, and this is mostly based on self-reported data, and then our third finding, we did pretest, posttest of the trainings, both the Mental Health First Aid training and the CIT training, and we did see statistically significant improvements in mental health literacy, stigma, competence to identify youth mental health needs and then intentions to intervene with youth in crisis.

But one thing that I think is really kind of important to point out is that, with each of these levels and these data collection efforts, we had significant shortcomings to what we proposed to do and what actually ended up happening, and so at the aggregate level, for instance, we were not able to measure within individual change, so we were able to only kind of look at what are the averages of our key measures of school safety and then compare those averages over time instead of looking at change within teachers or within students over time.

For the behavioral health improvement component of the evaluation, we had really low response rates at follow-ups.

We had a difficult time reaching students that had moved schools or that were kicked out of schools, and that happened pretty frequently because this was a high discipline population, and then we had proposed to do a lot of different longitudinal analysis with administrative data, which we found out in the end was unreliable, and we weren't able to use most of that.

And then the last piece with training, we didn't have any fidelity measures that were able to account for differences in how the training was administered, and so oftentimes, you know, when we talk about evaluation, we focus on the shortcomings of the methodology or the study measures without considering how project partnerships can also contribute to these limitations.

The importance of true and effective collaboration becomes even more important when your program that you're evaluating has multiple components.

There's no doubt that the execution of a complex evaluation requires the support delivery of the program and so -- And an example in our case, you know, we relied a lot on the social workers to produce the CANS scoring sheets, complete data forms, schedule follow-ups, and so we relied a lot on the commitment of others to carry out our evaluation, and so today we want to talk about some of the lessons that we learned from this experience and mainly stressing the importance of collaboration and communication and how the lack of these two pieces can impact the methodological shortcomings that we -- are often found and we often discuss in large scale evaluations.

So our first lesson that we sort of wanted to bring attention to is, we strongly suggest clearly articulating the expectations of the primary agencies and the partner agencies during the proposal stage.

Many larger agencies, such as educational institutions and behavioral health care providers, seek out external funding to either implement new programs or to sort of support some of the institution's priority areas, and a lot of times, this is really focusing on, you know, the couple first pages of a proposal without really going through the entire proposal.

It's important to kind of recognize up front the request for proposals or objectives within proposals are not always the same, so some are very programmatic, and this is a lot of times what educational institutions and behavioral health care agencies are used to, programmatic funding at the state level or some federal agencies where there might be some sort of performance measure component but not necessarily a research component, and then other RFPs, such as NIJ School Safety Initiative, are heavily research-focused, and so just sort of putting in the evaluation to get the money to implement the money, it's really not effective once you actually get the money.

From our experience, you know, we had a grant writing team that was separate from the project team, and so the project was sort of developed, and then there was -- You know, the valuation plan kind of was put in at the last minute, and so when the money came and the team changed, the feasibility of what was proposed in the evaluation really wasn't feasible and didn't come to fruition, so some of the recommendations that we came out of that is really kind of as the evaluator to really talk up front about feasibility with the appropriate people, and so it might not be the grant writer or the intervention specialist.

It might really be more about your principals at the school, your guidance counselors at the school and really mapping out your data elements with the people that are familiar with the database and the people that are going to be extracting the data for you because, you know, proposing certain things and using certain terms that you measure might sound good, but if you can't extract that from the database, it's really going to affect your evaluation, so kind of to summarize our lesson one, we feel like really kind of being on board with the program goals, the RFP goals and also the feasibility of the proposed evaluation.

For us, lesson two is to develop a strong and active leadership team.

There's a great deal of literature out there, particularly in public health and social work disciplines about the sort of tensions between the insider and outsider role as the evaluator, and we can't stress enough in our experience how important a strong and active leadership team that includes the evaluator or the PI is really important.

Decisions made about programmatic pieces most of the time are going to impact the evaluation in the outcomes that come, and so two examples that we experienced particularly with our training, the -- They learned very quickly that an 8-hour training was not feasible for teachers during the school year, and so they ended up splitting the training model into 4 hours 2 days a week, and sometimes those 4 hours were a week to 2 weeks apart.

We as evaluators, that's very important piece of information in evaluation.

It speaks to fidelity of the evidence-based Youth Mental Health First Aid model, and so it'd be a really important part of a rigorous evaluation.

Unfortunately, because we weren't involved in those programmatic conversations, we weren't aware that that was happening until it was too late to sort of measure that.

Another example that we experienced was pretty significant.

We kind of sort of found at the beginning of the project in our data that the social workers really weren't making the community-based referrals off the CANS that we had expected and proposed would happen, and what we learn -- But we had to start questioning, you know, why are these referrals being made, was that the social workers felt like they could do the counseling and provide the service themselves in the schools, but that wasn't being sort of realized in the data sheets that we were collecting, so it was sort of appearing as if they weren't being referred, but in fact, they were referring, or, in fact, they were -- The kids were receiving counseling directly from the social workers, so that kind of communication is stuff that maybe might happen in sort of a monthly leadership team where the evaluator could kind of understand, what are the ebbs and flows of the program? And we can then be able to incorporate that into our evaluation.

The strong -- This active leadership team is then, as a unit, would really be able to then have regular communication about the evaluation with the on-the-ground administrators, the principals, the guidance counselors, the frontline personnel who are, you know, being asked to participate in evaluation through submitting a lot of the data collection forms, and so, you know, through this sort of leadership team, a really healthy continuous quality improvement process can come out of that.

Lesson three for us is to find or to be an evaluation champion.

We feel this is really important in any research, collaborative research, practitioner, partnership.

You have to find the people that, you know, want to hear about the evaluation, that understand the importance of evaluation and then are kind of communicating and spreading that throughout the various stakeholders.

You know, ways to kind of get this done -- and some of these efforts we did implement.

The one is to train your project personnel, your frontline personnel that are helping, you know, collect the data and submitting the forms to you on the importance of that work and, to really drive home, there was a perception for us, we feel, that we have to do this.

It's part of the grant.

We have to do this because it's part of the grant, but that there was really no realization of the importance of the work in sort of the information and then would be fed back about sustainability of the project and how things are going.

We didn't have -- We weren't able to have an evaluation team member embedded in our primary agency.

Looking back, that would've solved a lot of the issues that we had, and we would've had an evaluation champion, you know, on the ground regularly interacting with folks.

So we strongly suggest if there is enough money or if there's -- Of course there's a lot of confidentiality and different issues that might go into that, but from an evaluative standpoint, sort of having a team member, maybe a student or an intern or somebody, though, that's, you know, embedded within the agency would really create an easy way to have an evaluation champion present.

Lesson four, our last lesson, is to really target the key program components that you want to evaluate.

It's the tendency of most evaluators, you know, to want to kind of measure all that we can and account for any potential confounding factors, but there really is a balance, and we learn this pretty quickly between obtaining every possible data element and obtaining valid and reliable data.

Particularly, again, when these data are being collected by frontline personnel who primary responsibility and their skill set isn't related to data collection or DI identification or transferral of data, a lot of those really important human subjects protocols.

A very common complaint from our work was from the social workers in how much time it kind of took to send us all of the paperwork, and so what kind of ended up happening is that we didn't get complete data, or we didn't, you know, we didn't -- We missed a lot of CANS scoring sheets, for example, and so then that sort of creates tension and more work to kind of continue asking for more data and all of that, so really kind of think about, you know, what are the targets of your evaluation? What's more important? Is it the process? What are the processes that will lead to your main sort of objectives? Logic models are a great tool for this, and we kind of started using logic models in all of our projects moving forward.

I also think this sort of relates back to lesson three.

Because we didn't have an evaluation champion on the ground, we really weren't able to get a lot of the data that we expected having, and so an example is kind of going back to our administrative data.

Because we didn't -- we proposed something that wasn't feasible, it turned it into a ton of work for our project coordinator, and so it was sort of a rushed job, and it ended up being that all that work went to waste because the data wasn't able to be used in a valid manner.

Given the comprehensiveness of program components in our study, we focused mostly on the outcomes without considering most of the processes that led to the outcomes, but unfortunately, you know, fidelity to program components, merging different organizational cultures, the ebb and flow of different stakeholder buy-ins, these are all process measures that can really contribute to improving the way that we engage in large scale interventions in the future.

And so, in summary, you know, we believe that conversations around collaboration and communication should start at the beginning of the proposal stage and carry through to the end of the project.

It's really important for all stakers to be on board with the evaluator being a primary member of the leadership team and with the work that's going to take to carry out the evaluation.

For this kind of collaborative partnership, I think to work on a broader scale, the traditional role of the evaluator really does need to change from outsider to team member, and no doubt the expectation of interdisciplinary teams is going to get stronger and more of a priority for funding agencies.

Particularly with projects that involve children and adolescents, the, you know, the overlap with criminal and juvenile justice and school safety and behavioral health is only going to grow over time, and so this is sort of a burden, I think, on all of us, on academic researchers and agencies to really kind of, you know, build a trust around each other and respect for each other's skill set and knowledge base and really kind of start to realize how we can really learn from each other and also on funding agencies to kind of make these collaborative partnerships more important throughout the process, so sort of in conclusion, what we really learned from this experience is that the three Cs, collaboration, communication and consistency, is really key to carrying out multicomponent evaluations.

Thank you.

>> Thank you, Kristina and Sara.

I think it's -- can be difficult to share a story where you have so many sort of challenges with the study, but I think it's really helpful for us to reflect on the importance of those partnerships and working with one another to learn so great, so now we will move on to our next presentation, which will be given by Marilyn King and Johanna Bertken, and, Kristina, I think you maybe will have to stop sharing, and then we can transition to the next.

Thank you.

>> Good afternoon, everyone.

I think we'll start with some introductions and then jump right into our presentation.

My name is Marilyn King.

I'm currently serving as the Interim Superintendent for the Bozeman Public Schools in snowy Bozeman, Montana, and I served as Co-PI for the NIJ grant entitled "School and Family-Trauma Informed: A Project to Examine What Keeps Schools Safe." >> And I am Johanna Bertken.

I'm the Student Assistance Coordinator for the Bozeman School District.

I'm a school psychologist by training, and I have recently in the last 2 years since this grant taken over this position with Bozeman School District and am in charge of our Student Assistance Specialists that we have now as part of our -- the continuation of our SAFE-TI program.

>> Thank you, Johanna.

Next slide, please.

So our grant was awarded several years ago and actually ended a couple of years ago, and as we mentioned, our grant had to do with school and family engagement, trauma-informed, so you might hear us.

We came up with a snappy acronym called SAFE-TI, SAFE-TI, and that's what we're referring to.

The Co-PI, who was Dr. David Schuldberg, and also our Project Director, Laura St. John, have since retired, but I want to give a shout-out to them because they did a massive amount of work, and we had some very, very interesting findings.

So the purpose of this presentation is going to be to give you a pretty quick rundown about our findings from the grant, and then Johanna is going to tell you what happened next, which is pretty exciting.

So our grant focused on the link between childhood trauma and behavioral health problems, and our premise was that if we were able to provide a multitiered package of trauma-informed interventions that that would offer a promising alternative to traditional, more punitive disciplinary approaches.

We had two main research questions, and you see those listed here.

One was a randomized clinical trial, and the question for that was, what are the effects of availability and referral to appropriate trauma-informed services and care on students' school safety indices? And our second question was, how does this impact school practices, culture and climate? And we used mixed methods for that.

We utilized -- The grant provided us with fiscal resources to utilize Student Assistance Specialists.

So those were some additional staff members that we had for the purposes of this grant that were used to provide additional tiered social, emotional and mental health supports.

Next slide, please.

So as promised, I'll just go over some of the impacts according to grade-level band.

At the elementary level, we had -- We definitely saw successes and impacts.

We utilized the STAR assessment for reading and math, and we saw that, for students that were receiving SAFE-TI services, there was a 26 percent increase in their reading growth and a 20 percent increase in their math growth.

There was also a huge attendance increase for students that received SAFE-TI services, and students identified increased feelings of identification, ability to express their feelings in a safe place.

They were better able to self-regulate their skills and increased trust in relationships.

At the middle level, next slide, we also had positive results, not as high as at the elementary level, but we saw a very significant increase in growth expectations in reading especially and some in math for students who received tiered SAFE-TI support services.

There was an increase in attendance, and we also had a decrease in behavior incidents for students that received the SAFE-TI services.

And then, at the high school, next slide, we had a slight increase in attendance for SAFE-TI students, but we had also implemented a check-in program, and we had with the Student Assistance Specialists, and we had high school students that would check in with those Student Assistance Specialists, and for those students who utilized the check-in, 34 percent of those students had a positive change in their attendance from one year in contrast to the previous year.

We had students who participated in SAFE-TI had a nine percent decrease in behavior incidents, and we noticed a three percent reduction in the likelihood of dropping out for students who participated in SAFE-TI services.

And I'm going to turn it over to you, Johanna, now.

>> Sorry, I had to close it out in order to turn off my mute.

So I want to talk about impacts and next steps, so what has happened since the grant and where we're headed.

So where we are now, we were able to have -- continue with the Student Assistance Specialists at the high school and the middle school level.

So we have one Student Assistance Specialist at each of our high schools, and one at each of our middle schools, and they are currently providing selective and indicated trauma-informed supports.

The method by which they're doing that is through group and individual counseling.

So in those counseling groups and with individual students, they've been doing Cognitive Behavioral Intervention for Trauma in Schools, or CBITS, and then they've also -- For more of the tier three students, they've been doing Attachment, Self-Regulation and Competency and using that framework to provide counseling-type services.

They are also doing regular check-ins with many of their students in order to develop relationships, and they provide a great deal of mediation with teachers and families.

So this means that if there's misunderstandings between students and their teachers or conflicts within families, our Student Assistance Specialists can step in to provide support in order for each party to understand each other better.

The focus of our Student Assistance Specialists and the services they provide...

...can make a big difference and having space that they -- and having space that they know they can escape to, where they're welcomed can increase their willingness to come to school and can encourage them to say even when they're in periods of distress.

So the services through the Student Assistance Specialists are increasingly focused on generalizing coping skills and relying on more natural supports that are built into the students' environment so teachers and family members and friends, coaches and religious leaders, people that are naturally in the students' lives.

So part of that is moving from co-regulating strategies to assisted regulation strategies.

Excuse me, moving from co-regulation or assisted regulation strategies to more self-regulating strategies.

So because we know trauma is a developmental phenomenon that affects the brain differently depending on when a child -- when in the child's life the trauma occurred, we know that students who have faced challenging circumstances often rely on regulation strategies that are not necessarily developmentally typical.

So, you know, for example, teenagers who have experienced complex trauma beginning in early childhood may not yet be able to self-soothe or apply any of the metacognitive strategies that are typically promoted by typical therapeutic practices.

So instead, they may need to co-regulate, almost as if they were much younger than their developmental age.

So sometimes they might need regulation strategies that include touch, rocking, singing or the application of pressure.

Our current model uses a lot of co-regulation.

The Student Assistance Specialists, if someone was -- a student was to seek out that person for support, the Student Assistance Specialist would help regulate the student, calm them down in order to send them back into the classroom.

Our goal right now is to start here and then build relationships that ultimately empower students to develop the skills and trust in themselves to engage in increasingly sophisticated self-soothing strategies, and part of that is doing that through a system of distributed support.

So by working with teachers and other adults in the student's life to provide reminders and, in some cases, assistance to use the coping tools that they've learned in counseling sessions, we know those can be more effective and more generalized to the classroom and other places in these students' lives.

Distributed support is also essential to being trauma-informed.

The top-down processing such as, you know, controlling our impulses through self-talk, doesn't usually work for students who have experienced complex trauma.

What works or doesn't work, as well, so to speak.

What works for students who have experienced trauma is robust relationships, developing a feeling of safety that allows their brains to release dopamine which results in the production of myelination that allows brain signals to travel more quickly.

So we know that the cortisol production that happens when students are stressed, overwhelmed or distressed in the school environment can inhibit learning.

So by encouraging and facilitating positive relationships with lots of people and making school a safe place, we're enabling our students to learn coping skills and, as an added bonus, enabling learning to happen more in general.

So the biggest impact of the safety diagram was not necessarily just our tier 2 and tier 3 services through our Student Assistance Specialists.

It's the normalization of the use of trauma-informed language, and what we've noticed is that there has been a significant shift in our school culture.

Teachers and staff are far more willing to interpret behavior and social emotional challenges through the lens of trauma.

We've noticed that each year, even since the grant has ended, there has been increasing referrals to our Safety Eye Program and our Student Assistance Specialists services, and this is a change from previous years because instead of responding with disciplinary practices, we're now looking to get these students additional support, sort of moving from the "What's wrong with you?" to the "What happened to you?" idea.

One of the things we know, however, is that there is a big distinction between being trauma-informed and being trauma-responsive.

So, you know, just referring to a trauma, someone who is trauma-informed isn't itself a very trauma-informed practice.

So that is one of the things that we are working on going forward.

We are also working on an increasing focus on tier 1.

So we've been engaging in some professional development on trauma, including Trust-Based Relational Intervention so TBRI and site-based trainings through the Student Assistance Specialists.

We've also been rethinking disciplinary practices, including at the elementary schools, rewriting our discipline code to get rid of the term "willful disobedience." We are also increasing our emphasis on predictable and safe environments and encouraging those practices through our professional development, and we've been using -- we've adopted a universal screening tool, the DESSA, for -- to identify social emotional skill deficits.

So instead of looking at behavioral problems or social skill limitations as being indicative of necessarily a mental health disorder, we look at them through, "What can we do to increase the teaching of these skills?" We've also increased our adult-focused support and interventions.

We know that to be trauma-responsive, we have...

...additionally trusting relationships with families.

Oh.

So why tier 1? Why are we choosing to have this as part of our focus? One of the things is that referrals to a trauma-informed specialist will never keep pace with the problem.

We're finding that with the increasing referrals, we are continuing to be behind, and even if we were to add additional supports, those additional supports would also become maximized very quickly.

Also, we have found that this distributed support model is more -- It's easier to do in a school environment than the behind -- and more effective than a behind-the-doors model of treatment.

So often a more traditional approach is involved, sending a student to a therapist where, you know, the therapist then fixes the problem and sends the student back out into the environment, and in the schools, we have an opportunity to have those students learn in the environment that they will be practicing those skills.

Also, this is more of an educational model as opposed to a mental health model.

The mental health model has a lot of benefits.

It's still very important.

It's a valuable thing, and having mental health treatment options available to our students is still a part of our tiered systems of support, but this is more along the lines of teaching skills and generalizing the skills into the everyday environment which requires every staff member to be -- to not only know about trauma and what it means, its prevalence and its impact, but also what tools they need to use to create a safe and relationally robust environment and also a -- what tools they need to use should a student be in distress.

And we know that even if the students we're working with have not experienced trauma in their lives, these are practices that are still beneficial.

They don't do harm to students who are not specifically students who suffer from or have been affected by trauma, and one of the things that is a direction we're headed is to help our staff and help our staff understand that you don't have to be a therapist to be therapeutic.

One of the things that we've in some ways been able to leverage -- It's hard to say that there's any silver linings about the -- about 2020 or the pandemic, specifically, but having these conversations about trauma in this moment in time have -- It's really been able to illustrate the point about how important these strategies and tools are, and many of our teachers and staff are able to relate to the content, the ideas in trauma-informed and trauma-responsive schools because of what they are currently going through after prolonged periods of uncertainty and stress.

So being able to communicate with teachers in order to encourage self-care, but also to allow for us to reflect on how trauma might be impacting the way students behave and how they might be thinking.

Oh.

So some of the challenges that we've run into with these efforts, first off, this is an issue that is about hearts and minds.

Anything having to do with challenging behavior can be emotional for adults in a school building, and specifically, you know, things having to do with aggressive behavior, offensive behavior, traditionally, schools have responded to that with disciplinary practices.

So for some people, changing disciplinary practices or encouraging trauma-responsive tools and strategies can be challenging to belief systems.

People can think that this is -- that it's not fair, and that makes it a hard subject to broach with our staff, in some cases, depending on the culture and the climate of the school.

So it really takes a lot of persistence and different strategies to get this adopted in a broad way.

And many of our teachers and staff feel like this is just another thing that's being asked of them.

It feels like something on top of everything else without anything being taken off the plate.

So, really, it's going to take some time to get the idea across that this is not something that is going to be taking more time.

In fact, it's something that's just more effective than what we've been doing in the past and could end up saving time, energy and effort in the long run.

There is a lot of conflict still with a more traditional medical model of therapy.

For so long, psychology has promoted this idea that there are professionals who have, you know, professional skill sets, and they are the only ones who can solve -- who can "solve" medical conditions, psychological conditions.

And so when people run into challenging behaviors, they're facing their own feelings of, you know, "I'm not qualified.

I don't have the tools.

I can't approach this problem or do anything to help it," when we know that, yes, in many cases, mental health treatment is necessary, but it does not mean that that student should not have other healing and therapeutic relationships with the people in their lives.

So really what we've learned, in summary, is that this is -- becoming trauma-responsive requires ongoing commitment and revisiting and revisiting the issue and explaining it not only in different ways, but continuing to provide additional tools and evidence of this being a benefit not only to the students, but to school culture and climate.

Thank you.

>> Thank you so much, Marilyn and Johanna.

We, you know, we typically have an opportunity to hear about the projects themselves when they're ongoing and when they have concluded.

But it's so rare for us to have an opportunity to hear about what happens next, how have you taken that work and kind of -- and gone on and used it to enhance the work that you're doing.

So I wouldn't be surprised if we have some questions on that theme which is my reminder to folks.

If you have a question, and you're saving it up, please feel free to put it in the Q and A, and we will get to it after our last presenter has an opportunity to speak.

And our last presenter today will be Stacy Overstreet.

Stacy? >> Thanks.

Just working on pulling up my slides.

Johanna, have you stopped screensharing? Ah, thank you.

Let's see if I can do this.

Well, let me -- Sorry about that.

Maybe my thing timed out here.

Ah, here we go.

As I'm pulling up my slides, I just want to say, Johanna, thank you for setting up my talk perfectly because our work enters in at tier 1.

So it was really interesting to hear that you guys started at tier 2 and tier 3, and you sort of moved down to tier 1, and we start in at tier 1 and kind of move up to tiers 2 and 3.

So that was really neat.

And let's see if I'm in, okay.

In, we're good.

Okay, great.

Hi, everyone.

I'm Stacy Overstreet.

I'm at Tulane University in New Orleans, and I'm going to talk to you all today about just sort of a piece of our work that has come out of what -- our Safe Schools NOLA Project, and our Safe Schools NOLA Project was funded through the Comprehensive School Safety Initiative in 2016, and so we just wrapped up last academic year as part of the project.

So Safe Schools NOLA is a multiple baseline study of implementation and strategies for trauma-informed schools at tier 1, really to set up those tier 1 systems.

So following that baseline year of 2016-2017, for the next 3 years, we randomly assigned two schools into kind of an implementation year where our work with them really moved them through the exploration and installation stages of implementation in an effort to sort of get them ready to make an initial adoption and implementation of trauma-informed approaches.

Our work during that year really focused on three implementation strategies.

Again, Johanna has already, like, hit the highlights of these.

So the first thing is that we start our work with schools doing a foundational professional development on trauma, the prevalence, its impacts, also the key principles of trauma-informed care, and we use SAMHSA's key principles in our work.

But it -- I really call this PD, like, winning hearts and minds.

So it's our first chance to sort of get with everyone, do some shared learning, get people really excited about the work and get everyone onboard.

And so we spend 6 hours with folks doing that work before the school year starts.

Then we embed a full-time social worker into the school to facilitate the second implementation strategy which is really skill-building professional developments and coaching.

So that first PD is really all about sort of the conceptual work and the need for it, and then we focus in on skill-building which I'll talk to you at the end of this talk about, to really have teachers try out some skills so that if you are going to become a trauma-informed school, here are some of the practices that you would likely be implementing.

Let's teach you these, coach you up in them and get you to practice them and see how they feel.

And then that sort of also feeds into the third implementation strategy which is really building a trauma-informed action plan so that the end of the year of work with us really ends in each school developing their own trauma-informed action plan based on a needs assessment of their system and also based on their teachers' experiences of these practices that they've been trying.

So it's both really a needs assessment at really a policy and procedure level at the school level, but also a needs assessment in terms of what would need to happen if teachers are really going to change these behaviors.

And so in terms of our outcomes, we just, as I said, finished wrapping up data collection last year.

So we haven't -- We're just beginning to get into the data, but what we're measuring is, you might imagine is kind of adult-level impact.

So the most proximal outcomes that we're hoping to push are around teacher capacity, increasing their knowledge, their attitudes and their behavior around trauma-informed practices and then also school capacity so making sure that the school has the procedures and policies it needs in place to really support the work.

And then, we hope, then that the distal outcome of that work will be that kids are feeling safer in schools.

So you see shifts in school climate but also shifts in student behavior and also teacher reactions to student behavior.

That becomes more about support and less about discipline.

So what I want to do today is just give you a little snapshot of data from our baseline year and then really talk about the central role of student perceptions of safety as an important component, really the foundational component, I think, of all trauma-informed schools work is that -- is the importance of students feeling safe when they're in the school environment.

So we'll talk about that a little bit.

You know, we've already, again, heard that we know that when students are exposed to trauma, and particularly chronic levels of trauma, that that can really result in neuro, biological and psychological adaptations that leave students in really a profound state of alarm.

And so they are hypervigilant to danger.

They are aroused and have difficulty regulating both their emotions and their behaviors because of this chronic experience of trauma, and so that dysregulation can make students more reactive to all sorts of things, teacher interactions, student interactions.

So the dysregulation can lead to increased rates of student aggression in schools and then, particularly for our sample of mostly Black students, but for Black and brown students, we know that disruptions in school is more likely to lead to out-of-school suspensions.

And so ultimately this kind of pathway is at the heart of what trauma-informed schools is trying to disrupt, and with this notion that if we can create a safe environment where kids can sort of turn down that arousal and gain some better control over their emotional and behavioral regulation skills, then we can disrupt negative pathway and foster resilience in kids who are attending trauma-informed schools.

So I'm just -- so this snippet of data is from my -- one of my students did her thesis on looking at this pathway, a little bit more complicated pathway, but I made it a little simpler today.

So Whitney Davis examined in her thesis whether perceptions of school safety could actually serve as a buffer to disrupt that relationship between trauma that kids have experienced and aggression in schools.

And so for this subsample of the data, this is -- We worked in six schools, and this is in the baseline year, and so this subsample is with 611 third through eighth grade Black or African-American students, 54 percent female, and although we didn't measure trauma exposure directly in our study, we were able to -- Whitney was able to, not me, geocode the 911 calls for base -- using kids' addresses, right? So we were able to narrow down to their census block and map out 911 calls for FBI crime categories, things like homicide and robbery and assault, to really get a measure of neighborhood violence.

And so what we found is that, not surprisingly, neighborhood violence does -- is associated with student aggression.

So violence experienced in the year prior to the academic year predicted student-reported aggression midyear, and that student-reported aggression predicted out-of-school suspensions at the end of the year.

We did not find that perceptions of safety moderated the relationship between neighborhood violence and student aggression.

So that relationship held steady, but we did find that perceptions of safety had an independent impact on student aggression so that the more kids who reported feeling safe at school, the less aggression they reported in terms of their own behavior.

And so I want to just kind of zero in on that a little bit and think about what that means from a student perspective.

Because, as you all know, when you're collecting data, and you've got surveys, you know, that are student perceptions of school safety, it was just a [Indistinct] measure, and some of the items were just, "I feel safe at school." So the question really is, what does that mean for kids, and how can we capitalize on sort of that understanding to really build out our trauma-informed approaches? And so I'm going to talk a little bit about that.

As part of our work in each school each year, we did do focus groups with students in fourth grade and in eighth grade to really begin to try to understand what it would mean to them to be in a safe school.

So what does safety in school mean to them, and what might it look like sort of on a day-to-day basis? And so here are just some of the responses from our students about what school safety means to them, and I pulled -- So one of the major themes that came out of this data is that teachers are a big element of what makes kids feel safe at school.

Now, they did also mention some other dimensions, but we're focusing in on teachers today.

So there was some responses around student bullying and sort of peer-to-peer things that could make students feel safe or unsafe.

But a lot of what came up was the role of teachers in safety.

And so some quotes, so, "Teachers help make students feel safe because they are there to support.

Teachers are there to talk to you if you are feeling sad in class.

They can pull you into the hallway and talk about it," right, or "Teachers who are proactive and address issues quickly can make students feel safe." And then they also told us some teacher characteristics that don't necessarily make them feel safe.

So, "Teachers who have nasty attitudes can make students feel unsafe," or, "A teacher that doesn't make us feel stupid" can then make us feel safe.

And so those are just some perspectives of what makes them feel safe, and so then the question, another question is then, how does that work? So how did these teacher behaviors actually foster a sense of safety? And there are kind of three lines that I want to draw out here because they relate to some of the intervention work that we do in schools.

So one is that those teacher behaviors can actually make the classroom feel safe.

So the student in the middle says, "They let me go to the safe place if I need to.

Some teachers give you a paper, and we can say more about how we're feeling or what we are happy, sad or mad about," so this idea that teachers can help create safe spaces in a classroom.

Or the last one, "One time I was really upset in class and wanted to go home.

My teacher had me write down things my mom would say to me when I want to go home on little pieces of paper and keep them in my back pocket.

Then she told me to practice taking out those notes and reading them when I'm upset." I just love this because you can imagine, you know, a teacher becoming frustrated as a kid becomes upset in the classroom and might engage in disruptive behavior, and here, a teacher is really facilitating a strategy that enhances a student's own ability to self-regulate and sort of manage those feelings so making the classroom feel safe is one thing that teacher behaviors can do.

Students also commented on this idea that regulated teachers can also help students regulate.

So Johanna talked about this idea of co-regulation.

It is really kind of one of the themes that we use in our work on teacher practices that are trauma-informed.

So the idea that "Students know when you come in it's going to be a calm environment, and it's quiet, and she'll put on music that we can work to." Okay, or, "He is calm when he talks to students.

When people yell, I always think to myself, 'I'm not about to do nothing they say.' You got to make me want to listen." So this idea of teachers as calm, kind of co-regulators is another way in which students can make students feel safe.

And finally, teachers can communicate care which is really critical for us to feel safe, right? Trauma and violence can really disrupt our connections with others, and so when teachers can recreate those connections, then you can really enhance safety and begin to help kids really make the most out of their opportunities for learning.

So, "I like that my teachers don't assume that I'm doing something bad on purpose.

They just know I'm having a hard time, but it's not on purpose," or, "At least he tries to listen to where I'm coming from, and he respects my space." So this, I think this notion of really being seen and being acknowledged as sort of a full human being in the classroom that needs that interpersonal connection and needs someone to sort of really understand, understand where they're coming from and what they've gone through.

So those three kind of themes that came up really emphasize our approach to the work at tier 1.

So a lot of our work with the skill-building and coaching that we do with teachers is all about enhancing relationships and enhancing student regulation, but also really teacher regulation because that's really a key component to teachers being effective in their classrooms.

And so we -- Our three skill-building PDs, I'll just briefly tell you about each one.

The first, which usually happens at the very start of the year, is around safe and supportive classrooms.

So again, it's really having teachers develop some skills and strategies to create an environment that feels safe to students.

We have teachers -- We do kind of a joint overall session talking about the importance of this in relation to trauma, and then teachers are able to choose a learning station that they can go and sort of pick which strategy they want to use to create safe and supportive environments.

Our three key strategies that we focus in on are the importance of rituals and routines, and so for a teacher who may be feeling a little less sophisticated and wanting to try out trauma-informed approaches, you know, we start with the basics, like, "How can you create some rituals and routines that are going to help kids enter a classroom and feel safe?" And so you can see this kind of greeting routine that the teacher is doing at the door.

We had another teacher that had students lead openings to their gardening class, so the students sort of led that routine as folks came into class, and then another teacher worked in a student closing to their PDIS system so that there was a routine and a ritual around student engagement and student involvement.

The other thing that we do is, or another learning station teachers can choose to go to, are calm-down corners, so again, places in the classroom for students to be able to self-regulate so that they don't have to leave the classroom and just, again, sort of the really intentional and conscious way that teachers use these.

Right? So it's not just like, "Go to the calm-down corner," but how can we work calm-down corners into the classroom environment that really intentionally build skills for students to learn self-regulation skills as they use that strategy? And then our other routine for safe and supportive classrooms is community building, so teachers can learn how to work in community-building circles into their classrooms as a way to create a safe and supportive environment between them and their students, so that's just some photos of that.

And so what happens is, for each of these skill-building PDs, this one happens at the start of the year.

We try to find teachers in the school who might already be doing pieces of these, and then we really colead the PD sessions with the teachers so that it's not just sort of us advocating these new skills, but they're really also learning from their peers, and then after that session, for a period of about 2 1/2 to 3 months, our coach at that school would help teachers set goals around this behavior and then do some live coaching, consultation as they practice these skills and really kind of get their feet on the ground as they're using them.

So then the second skill-building PD that we do is around teacher regulation and really focused on having teachers become more aware of as incidents are happening in the classroom, what are those automatic thoughts that are going on in their head for better or worse? How do those thoughts drive how they're thinking and feeling and then what they choose to do? So basic CBT for the psychologists in the room, right? But for some teachers, they haven't necessarily intentionally sort of gone through this process, so again, we do some shared learning with everyone around the basics of this and the importance of this, and then teachers can choose one of four learning stations to go to around this.

So for teachers who might be having more difficulty sort of identifying those maladaptive thoughts or judging themselves really harshly for their behaviors, they can choose learning stations that will really help them kind of explore their own dysregulation, help them practice self-compassion so that they're able to sort of tolerate that dysregulation a little better.

But for teachers who feel like, "Okay, I've got this.

Like, I know what my buttons are.

I know how to shift my behavior around them.

I really want to focus more on, 'What do I do?'" then we've got learning stations that build on the basics of what we've already taught.

Right? So it might be that a teacher started using a calm-down corner in the first PD, but now they really want to advance their skills in using that to help students really learn how to regulate their behavior, and so they might get coaching to sort of move that forward.

Same with community-building circles, so maybe that started at a basic level to generally build community and relationships, but maybe a teacher wants to evolve their use to handle conflicts in the classroom, and so how do you then expand that skill to do more solution-focused conversations? So that's our teacher regulation, so same thing.

Our embedded social worker would set goals with teachers, do observations, coaching consultation, and then the last skill-building PD is around strengthening teacher-student relationships.

And for this skill-building PD, we use materials developed by the Search Institute which really sort of outlines in a nice way the different elements of relationships that teachers may have with their students, and we have developed a self-assessment tool based on these elements of relationships and kind of what it looks like, so we have teachers after we do some common learning.

Teachers complete a self-assessment for themselves and then spend some time identifying which of these domains they might want to, you know, grow in.

Right? And so they identify a goal based on their self-assessment, and then that next phase of coaching is really going to help them kind of really zoom in and develop growth around that goal.

So the whole idea of this, right, is that if we can build teacher capacity around sort of understanding what trauma means for kids, a willingness to really examine sort of their role in both building a classroom and managing the dynamics in that classroom and then try some new behaviors, we hope that that is going to lead to changes in school climate where kids are feeling safer, more supported, more secure and thus having fewer aggression problems or for the need for disciplinary action.

And so that's kind of a deep dive into that one implementation strategy that happens as the other systems of work is going on.

So can this work? We don't know yet.

So as I said, we have our -- moving into data analysis this semester, so we're hoping to have some big-picture answers at the end of this semester, but as Christina was talking about, you know, when you get data, which, Christina, I could relate so much to the difficulties in getting data because it's not easy, but when you have it, then you can really begin to go and get a closer look at, like, "What are the mechanisms happening, and how does this actually work?" So we're looking forward.

Mary, maybe we'll come back in a couple years and really get into the big-picture results, but that is it for me.

>> Great.

Thank you so much, Stacy.

Three projects on the same theme, but sort of at different stages in the process.

I'm waiting for questions to come in from the audience, but I thought I would share a comment and see if anybody wanted to respond to that.

So I was -- what I -- Something that I wasn't expecting but was struck by with all the presentations is the role that perceptions played in the work that you're doing.

So I'm thinking in both Christina's presentation, Christina and Sarah's, and then Marilyn and Johanna's, it was sort of the, "Who should do the work?" idea and how you sort of worked or had some challenges related to that idea.

And then, Stacy, in your presentation, these perceptions of the students and what made them feel safe or what made them want to behave a certain way, that seemed to be, like, sort of really, you know, playing an important role here, so if you all wanted to add anything or make another comment on this notion of perspectives, I'd very much like to hear it.

>> Yeah.

I'd love to comment on that.

I think that for us, again, one of the biggest challenges is, you know, perceptions about what is justice for a child that is seeming to be misbehaving, that -- I mean, there's been so many years where we have not been very creative about our disciplinary practices, and we just don't have a lot of tools in our tool kit, and a lot of our teachers really feel like if it's not suspension or one of those, you know, punitive strategies, then it's not going to result in behavior change or it's not delivering the message that this is not acceptable behavior.

And I really think that many people really believe that shame is an effective tool for behavior change, and it might be effective for some people, and it might be effective in the immediate moment, but shame in the long term creates many more problems in a school environment, so it's just about changing people's beliefs about behavior and discipline to become more trauma-informed, and that's the biggest challenge.

I think it's the hearts-and-minds question again, and that takes a lot of time and energy and effort, and it has to take -- It requires evidence, and I think they have to see it work in order to believe it can.

>> I can piggyback on that.

We had the most interesting conversation with our principals about the term willful disobedience, and that used to be a, you know, behavior infraction with that language, "Willful disobedience." And it goes back to your question, Mary, about, you know, who takes ownership? And willful disobedience with a trauma lens that our principals had gained over the course of these last many years, they were uncomfortable with that term, and they realized that that was putting the behavior on the student and as adults that we weren't owning that, and we needed to own our response to it.

And so that was just a perfect example of how our thinking has evolved to these two words, willful disobedience, that have probably been in our behavior code for the last 50 years.

>> Great, thank you.

>> Yeah.

We have it in our code, too, so -- But you're right.

So, like, we address this issue around teacher perceptions of discipline in our foundational PD, and we intentionally sort of bring it up because it is true.

When you are first doing this work with teachers, you know, their reaction can be, "You're just letting the kids off the hook," or, "You're coddling them," or, "This isn't going to do any good." And so we have an exercise where we really have teachers explore, "Can you do trauma-informed work and still hold students accountable?" Right? And you can.

It's just it doesn't become only about punishment.

It becomes about both discipline, accountability and supports hand in hand, right? But I think the other thing that helps is this idea of just giving teachers a moment to understand the perception of students.

And so we actually use, in the relationship PD, we use snippets from students at that school to really have teachers reflect on, "Okay, this is what students are sort of reporting," and it's a positive exercise.

It's students talking about what a supportive teacher means to them and then just allowing the teachers to kind of sit with that and hear that and then reflect on their relationships with students and their goals.

It has been really powerful.

I mean, it's so simple, but it has really been one of the more powerful things that we do with teachers.

>> Thank you.

Christina or Sarah, did you want to add anything to that? >> Yeah.

I think for us and kind of what we talked about today, you know, we're thinking more along the lines of perceptions of evaluators, really, and of researchers.

And, you know, it's common for programmatic partners to not look at the evaluator as a project team and so to kind of perceive the evaluator as an outsider.

And that really limits, you know, the value that the schools receive, the value that we could have provided in terms of the effectiveness of the program and the areas that were affected and weren't, and so it's really, for me, kind of getting at this idea of, "How can we change the perception of the academic researcher, and how can we change the perception of the agencies and how they -- you know, the unwillingness to work?" So I think it's on both the perceptions of kind of both partners working together to kind of value what everybody brings to the table.

>> Thank you.

>> Can I say one more thing there, Mary? >> Absolutely.

>> Because, Christina, really your topic just hurt my heart because of all the common struggles.

>> Yes.

>> One thing that we tried is, you know, during the baseline, you know, our school said, "Well, you're collecting this data.

Are you going to share it back with us?" And, you know, I think sometimes when you're doing intervention work, there's this real fear, like, "Can you share that early data? Is it going to impact your intervention?" But, you know, we talked to our methodological consultant who's been doing this kind of work for a lot of years, and he said, "Look. If sharing data was going to be the only thing that led to change in these school-based interventions, then we wouldn't, you know, we wouldn't be doing all of this." Right? And so it really just gave me the perspective that it's not necessarily going to contaminate or ruin everything if you share some of the data and then show them how it can be meaningful and sort of just how it drives what you're thinking and what you're trying to do.

So we incorporated that into our work so that each year the school would get, like, a little summary, not of all the data, but of some of the things that they might be interested in.

And I think it just created, like, a different level of investment in terms of their perception of what research means and what it could do for them.

>> Yeah.

Yeah, and I think that's a great point.

You know, I worked on other projects where we do that continuously, kind of like an every day, every-90-day discussion, and it works really well.

We did, Sarah, worked really hard every semester, and, you know, did an individual report for each school, and, Sarah, can you help me? Did we ever have a discussion? You know, we tried to have meetings.

We never really heard back about kind of what they -- You know, it was just, again, very kind of, "Thank you for the report." You know? And so again -- And it kind of just depends.

I really truly believe that from the beginning how you develop those partnerships and where the collaboration kind of lies can really set the stage for a lot of that.

>> No.

I don't think we ever heard back from anyone saying anything other than, "Thanks for sending it." And then their final takeaway at the end was, "Okay. This worked, awesome," but there was nothing along the way, and what worked for them and what was a positive, like, outcome for them wasn't necessarily what we saw as positives, so there's just a lot of different components along the way that weren't necessarily agreed upon, I guess, for a lack of better word, but yeah.

We never heard back from the reports throughout the whole, usually, for the whole project, so just a little strange, but a learning process.

>> It's interesting.

I'm looking at our -- I think of you as our Montana folks, which probably isn't a fair way to label you, but knowing that you all came from within, you know, the school district that really, from my perspective, speaking of perspectives or perceptions, that was driving the work.

The research was helpful.

You sort of used that, but you were really driving the work in the school, and so you were in a bit of a different position, it would seem, that some of the other folks on our panel.

And so I think just sort of a value of that -- I don't know.

Just it's interesting to me to sort of reflect on how that may have played a role in your ability to implement what you wanted to implement.

Maybe it came with challenges as well.

I mean, I often wonder about sometimes when you're internal to something, does that come with a different set of challenges, so it seemed like you had so much success from an implementation perspective.

And, Marilyn, I remember throughout the project, I hope you don't mind me saying this, sort of hearing from you and hearing from others on the project about how the work that you were doing got picked up in the news, and they were engaged and sort of wanted to know what you were doing and why you were doing it, and that can be unusual, so just curious if you have anything to add on that.

>> Yeah.

That's a great question.

We had a very supportive superintendent and just a lot of champions already in the district, and we made sure that we got a lot of press about it, and we made several presentations over the course of the grant to our board of trustees, and that was picked up by the media.

So they kind of grew up with the project, and when the project was over, they couldn't wait to hear about our final results, and then it was like, "Well, what's going to happen next? Now there's no financial support." And there was just kind of this outcry that because we had been very transparent with our results and our results were significant that this wasn't something that we could let go, and, you know, our business manager said, has always said, "We can do anything we want.

We just can't do everything." And with that in mind, this was prioritized, so, you know, we couldn't sustain everything, but the grant and our findings allowed us to make a case for the Tier 1 interventions and then keep some of our higher-tiered interventions in the form of the student-assistant specialist.

We couldn't retain them all, but we put them where we needed do, and that's what has allowed us to create a sustainable project, program.

>> Right.

Well, this is certainly very challenging work, and I certainly have learned a lot.

Even though I followed your projects for so long, I'm continuing to learn more about what you learned and what you're, in some cases, continuing to do.

I don't see that we have any notes from our audience.

They're quiet today.

I know they're there, but I will, if anyone else wants to make any -- If you want to make a final statement, any of our presenters, before we close, I'll invite you to do that, and then we can end our session.

>> I'd just say, "Thank you for attending the session." >> Okay, great.

Okay.

All right.

Well, thank you to all of you.

This is a great way to end the second day of the conference for me, and I look forward to having you all join the conference again tomorrow on our final day of the session, so have a good evening.

>> Thank you.

>> Thank you.