Tip Lines - Roundtable Discussion, NIJ Virtual Conference on School Safety

>> Okay, well, for the sake of time, I want to go ahead and try and get started here.

So again, welcome to our first round of roundtable discussions.

Today, we have Dr. Mike Planty, who's going to be talking with us about some of his work on tip lines.

So there's a couple options that are available to you in the roundtable discussion.

First, you have the chat option, and there you can go in and submit specific questions that you might during the presentation or any potential technical issues that you're dealing with you can put in there, but because this is a roundtable, there's also the option for you to be able to raise your hand, and then we can actually call on you and completely unmute you and share your video and allow you to comment and ask questions and actually hopefully participate in a little bit more of a discussion on this important, important topic.

So we'll go ahead and get started, and I would like to introduce Dr. Planty.

>> Hey, thanks, Caleb.

Hi, I'm Mike Planty.

I'm with RTI International.

I'm a part of this session here to help facilitate a discussion because I do work on a tip line study out of Oregon.

We are evaluating Oregon's SafeOregon Tip Line Program that was rolled out a couple of years ago through 1,200 schools, K through 12.

We have conducted also a national survey on school tip lines primarily focused on public middle and high schools and examined how they implemented and what their perceived impact of those tip lines have been on their school safety programs, so we interviewed school administrators across the country, a representative sample.

We have also conducted a focus group with our state coordinators across the country.

There's a group of state leaders that meet regularly or have been meeting regularly to discuss the implementation of tip lines.

This include Michigan, Colorado, Oregon, Maryland and other states, and together, we created a tool kit because what we've learned right away, right off the bat, is that tip lines are not something where you just throw a number up and you walk away.

It's one of these things where it's really about a process that's integrated with a comprehensive approach to school safety, and so that tool kit just goes through seven or probably closer to nine or 10 different areas in terms of the mechanics of a tip line, stakeholder involvement, marketing and awareness, legal and ethical considerations and dissemination, and I could put both of those actually into the chat box, or you can just...

Is that going to...

Yeah, maybe they're not.

The links are not working there.

But if you can e-mail me or some other way I can get those two reports, they're available online, and we have a third report that we've just done where we look at state laws around mandatory or the encouragement of the adoption of tip lines across the country, so a lot of places will make it mandatory but don't really talk about the specifics of what it should be.

Should it be anonymous, confidential? Who should own it? Should it be with law enforcement or Department of Education or some other entity? How often do they need to report out on it? But, more importantly, I think the question that comes up is that if you set up a tip line, and you're receiving this information, what you have in line to process and triage the information and then to act on the information.

Anecdotally, we have seen many schools that will go through the adoption of the tip line but are not prepared with what they receive, and I think Al touched on this.

I see he's online, and maybe he can say a little bit more about this, but the one thing about the tip line, it is focused on prevention, but it is a one-stop shop for every problem in the school, and just like I think what Al showed in his previous presentation just an hour ago, we also in Oregon are following tip lines over the last couple of years, and we see a lot of bullying, a lot of self-harm.

That was probably one of the biggest surprises or concerns that we see is that tip lines really identify students in mental health crisis, whether they're talking about suicidal ideation or depression or things related to any other types of self-harm.

Well over 50 percent of the tips we're getting in focus on peers or self-reporting.

We do have bullying, but we also have other things: bullying, threats to schools, threats of violence, but other things have popped up like sexting where people learn about other students passing on images.

So maybe we can start off this about focusing on what these tips...

the scopes of the problem, the scope of the problem because one thing we're seeing in Oregon, too, is this changing in the way you characterize tip lines.

So it's no longer a threat assessment but a safety assessment because so many of the tips that are coming through are not your conventional threat, and they require, as Professor Cornell was saying, a different type of response and different type of approach especially around mental illness, but when you're talking about drug use or bullying or threats of violence, mental health, these all seem to suggest that we might have to have not a single approach but a multifaceted approach, customized approach to these problems.

So I'm really interested in learning how maybe others who are dealing with this, their perspective on bifurcating at least between the mental health and the threats of violence and the response patterns there with other types of problems.

Folks, because this is...

We have probably like 20 folks, so feel free to raise your hand or speak up.

This is the nature of this roundtable discussion.

Any thoughts about how school tip lines are being implemented and the response or the lack of or appropriate response? There's been anecdotal stories.

Well, with Pennsylvania, for example, a big study covered the mental health of their students they weren't prepared necessarily to deal with because it suggests that maybe instead of a law enforcement response or a threat assessment response that you need counselors in schools, and do you have the right ratio of counselors to students? Do you have the right services in place? Thoughts about that from people? >> Yeah, Mike.

>> Yeah, yeah.

How you doing? >> This is Al Stein-Seroussi from Pacific Institute.

We are currently conducting a study of a tip line in the state of Nevada.

I was trying to figure out how to raise my hand, and I couldn't do that, so I'm jumping in.

Yeah, I agree, Mike, that these are thorny issues that I think anyone who is thinking about starting a tip line, whether it's at the state level or at the school level, you really need to at least think through what your approach is going to be, and I don't know that there's a right answer yet.

I don't know that there will ever be a right answer, but I think that there are a lot of studies now that are beginning to explore that, so like you said, do tips go to law enforcement or do they not? Is it a law enforcement-centric system, which is what we have in Nevada but with strong partnerships with the Department of Education and working on partnerships with mental health at the state and local level? Or is it a system that is more mental health associated from the outset? So tips go to social workers or counselors to then disseminate the tips, and I don't know that we have enough data to say which is the better approach, but you've got to be prepared to be thinking about what your approach is going to be, what the implications of that approach will be and how you're going to follow through.

Right? You've got to be prepared if you're putting up a tip line, and ideally, it's a tip line that's operating 24-7, right? You need to have the person power to get those tips initially.

You need to have the person power to respond locally.

You need to have training, both wherever the tips come in and locally.

You need to have to, as you said, have an understanding that some of these are going to be more social, emotional and mental health-oriented.

Some of them are going to be more widespread harm or violence-oriented.

Some of them are going to invoke law enforcement in ways that maybe you didn't think about, like so if a kid in school is reporting that his buddy may be doing drugs, well, now you've got essentially a criminal offense happening possibly on campus.

What is going to be the response to that? Is law enforcement going to be invoked or not? There are just a lot of things you need to be thinking about and aware about and at least have an approach in place.

Again, we don't know the right approach yet, but there are just a lot of implications of setting something like this up and running it.

>> Well, thanks.

Yeah.

Well, I'd like to touch on the role of law enforcement, and one of the models that we initially looked into at the beginning was who owned it, and in Oregon, it's run through the offices...

It's run through Oregon State Police versus the Education Department, but the role of SROs and law enforcement at the school level, let's park that for a second, and I think we have a hand raised with Kati Garner? I think you have to unmute yourself.

Yeah, okay, yeah.

>> Sure.

I'm from Colorado, so I'm actually...

I'm a former school counselor, district administrator where I oversaw Safe2Tell for my school district in Colorado, which is the reporting system that Colorado uses, and now I'm with the Colorado School Safety Resource Center, so I've kind of worn a lot of different hats.

But, Al and Mike, I think you both hit on some important points, but I think the key is you have to have a systematic plan when you're coming to the schools.

If you do not have a systematic plan, there's a lot of confusion within the schools.

You do have to give some level of autonomy to the schools in order...

and the community.

There has to be autonomy because no two districts are alike.

Al, I'm actually very familiar with your program, oddly enough, because my mom worked in Battle Mountain, Nevada, but you can't have a program run in Battle Mountain like it would run in Las Vegas, and so there has to be some autonomy to the local district in terms of, how are we going to run this and respond, and really, it has to be in a lot of ways a community approach with community stakeholders because you talked about how, who is going to take the tips, and how is it going to be open 24-7 because as you guys know our students are the front line of information, and usually, information does not come in, in a convenient manner because as a former school counselor, it was always at, you know, 2:45 right before the bell rang on a Friday that you're getting a suicidal student, and that's kind of how tips are received.

It's 11:45 on a Friday night.

It's not during a convenient hour, so really, what is that community approach? How is there a community buy-in, and how do we get community partners to work together? Because I can tell you that one of the biggest frustrations that I ran into and continue to run into and try to help districts with is, how do you get local law enforcement and schools to play nice when there are two very different structures and when you have multiple jurisdictions involved? So if you have three or four jurisdictions, who's responding? Who's responsible for responding? Who responds at what time? And so really having a clear community format of understanding of not only who owns those tips but then when the tips do come in who is responsible for responding and really having that clear outline for schools and for law enforcement and for all involved because if not, a lot of times those tips end up just getting dropped, and then you lose the faith of the community and the school and the students because they say, "Well, we used that system, and no one responded," and the reality is that the state set up this wonderful system, put it in the hands of the district and the community to give them that autonomy, but there wasn't really that guidance to bring them together as a community.

They don't necessarily know what to do, so that infrastructure falls apart, and then those tips go unanswered, so it has to be very systematic and continued training and really a cheerleader representative to be able to come in and say, "I see your pain.

I see your pain points.

Here's how we can help address that for your specific district, for your specific pain points." >> That's a good point, and we have a related question about, how should a tip line be managed? What is the best practice behind that or evidence-based practice? And there really isn't any.

The only answer is what types of resources you have at the local level, and you see like Al just mentioned some...

I believe Utah runs it through clinicians and 24-7, and it's really a hotline, a referral system versus a reactive tip line, so our national tip line study, we found exactly that: widespread variability across the country even within states with supposed the same model.

A lot of it's driven by having a champion at the school level.

I think we heard that on the previous presentations.

In order to have it to be really active, you need someone at the ground level to really carry it through and sustain it, and given the turnover we see at schools, whether it's with the administrator, he or she leaving...

Average principal is 3 1/2 years or something.

You really have to have the backing and find someone, and I think this...

We can get in a little bit about that, but the administrators find themselves also with the unintended consequence I think here.

It's a 24-7.

They're on-call all of the time, even when we see that probably 95 percent of these are rising to what we could call tier level one, level two in terms of exigent circumstances, but they're getting these calls all of the time throughout the year.

I think one question was raised about, any thoughts about whether it should be anonymous or confidential? We see wide difference across the country on that.

I think the hypothesis is if it's anonymous, students would feel more comfortable with supplying the information.

Thoughts? Any hand raising? >> I can tell you in working directly with students as a counselor, most of them, if they knew it was anonymous, they were more willing to report than if it was confidential, and that's directly from working with students and talking with students one-on-one in a district of about 32,000 students, asking them directly, "If this was confidential versus anonymous," explaining what that meant, of course, and then hearing feedback from them.

A lot of them, the perk of it was that it was anonymous, but with that, there is a can of worms, so that goes back to educating the students on what it is and what it is not so that you don't have misuse of the system.

>> Yeah, and the other thought about students implicating themselves, right? They find out about something because they skipped school or they were participating, using drugs or other situations, and what does it mean for their own self? Somebody was going to say something, I think? >> Yeah, I was going to say...

And it gets even more complicated than that, I think.

Number one, I think a lot of us on this call and who are researchers, we sort of understand the differences between an anonymous source and a confidential source, and I'm not sure that that always trickles down to an understanding at a local level.

Then there's of course how you set up the system.

There are practical sort of technical details about whether something is anonymous or confidential, and a part of that depends on what you actually ask of the tipster, right? So, in Nevada, there's never any information that's asked of the tipster about his or her identity.

And it's I believe at this point is portrayed as an anonymous system, but there are complications because there may be times when someone is self-reporting and is in imminent danger to themselves or possibly to someone else.

And so there may be circumstances under which you want to be able to get information about who that person is if it is a matter of saving that person's life, and I know that this has happened in Nevada.

Even the issue of anonymity and confidentiality is not so clear-cut, and there are a lot of issues that have to be addressed, and you can make something that probably for all intents and purposes is anonymous but at the end of the day could be set up so that in life-saving circumstances a person could be identified by an IP address, for instance, and you may have to go through a series of steps in order to obtain that address, but that's a...

It raises a lot of areas of gray that, again, I think when you're going into this you really need to think about the implications of all of that.

>> Yeah, one of the...

>> And I would think that...

>> Go ahead.

>> I think that goes back to how the system works and how it communicates because if you have a student that is in a life-or-death situation, being able to have a one-on-one, real-time conversation with them even with anonymity is possible depending on the system, so being able to have a, "Are you safe? I'm worried about you? Would you be willing to talk to a counselor? Where are you located?" so being able to have I think that all, and that goes back to how you're setting the system up.

Are you able to have conversations with them in a one-on-one, live-time conversation? That makes it a piece of the pu.

le, as well.

>> Yeah, that's a good point, and we have some comment...

If you check the chat, make sure folks are checking the chat there.

I think one thing that gets to the quality information is another concern, and there was a comment about inaccurate tips versus false tips or pranks, and I think those are very different.

If there's a nefarious prank or you're trying to get a person in trouble by false claims, that's one issue versus what we've seen...

Again, in Al's presentation earlier he highlighted the number one thing we've heard from principals was insufficient information to act.

So even though the form for SafeOregon is primarily closed but with a narrative, sometimes it's not populated the right way.

It's not filled out, so they don't even know what...

There's a drop-down box for schools, but you go any further, you can't sometimes figure out what to do, who owns this, and that insufficient information is a big concern, and so that goes to the education and the awareness and marketing of it.

How do you fill out the form? What do you fill the form out for? What is this system used for? And then awareness was another big concern.

Just kids and teachers didn't know about the tip line.

You have your initial rollout, and then without a champion, either the day-to-day routine of running a school, it might get lost, and you don't have that marketing or awareness program, and I think the Sandy Hook Promise program, we could talk about a little bit.

That's one program that really talks about having a student-led champion at the school.

It's about marketing.

It's about awareness and getting people to use it the right way.

In terms of false claims, they're always a concern, but from what we can gather anecdotally through our systems, it's relatively low occurrence.

Is that what others are thinking? I think...pretty sure that's what I've been hearing across many states is that reporting false information is not a prevalent occurrence.

So maybe jump right into it: One thing we touched on was around the law enforcement role, and one model is to have law enforcement involved and leading this because they have access to different databases and data sets.

When you think of active school shooter or something of that nature, that might be the model.

On the flip-side, concerns around a criminal justice approach to solving some of these issues has been raised, and in Oregon right now, again, like I mentioned at the beginning they are moving away from threat assessments towards identifying safety assessments because of the variation of problem so even the semantics, and across the countries, like elsewhere, there's the push of moving SROs, getting them out of the schools and law enforcement approach as a solution so relying on school security officers funded by Department of Education versus SROs.

For example, in Montgomery County in Maryland where I live, 25 high schools have 25 SROs but over 225 or 50 school's personnel, safety personnel, funded through the Education Department.

And so what's folks thoughts about SROs and law enforcement and their role on the tip line? And multidisciplinary teams I believe is the recommendation to have some sort of law enforcement entity there.

Any hand raising? Maybe I'll pick on you again, Al.

You're looking at MDTs, right? >> Yeah.

It's interesting.

I have these data, and I'd have to pull them.

Nevada is an interesting place because as large as it is, there are only 17 school districts, and I'm thinking in contrast, for example, we visited Colorado early in the process to see this Safe2Tell system in Colorado, and we were in a...

I think we visited a county that had 15 school districts in that county.

Okay, so Nevada has 17 school districts, which means it's just a much smaller number, so you don't have as wide a variation.

My point where I was going with this, Mike, is that we don't seem to have a whole lot of SROs on the MDTs, which I was surprised at.

But, on the other hand, Clark County School District has its own police department, which is completely independent of the Las Vegas Metro Police, so Clark County School District has a 24-7 police force, and that is accounting for 65 percent of all the students in the state because that's what Clark County is, right? So it's kind of a...

It's such a weird state in that sense.

It's hard to generalize what that means.

But your point is also that there are many different arrangements that may have.

You might have school districts with their own police departments.

You may have school districts that have SROs.

You may have school districts that have a 24-7 police department, or they may have their police departments, but they're only operate during school hours, so you have to have a very strong relationship and MOU or whatever the case might be between the local school district and their police department with the other jurisdictional police department that is in play whether it's a city or a county.

So this doesn't directly answer your question except to say that there are a lot of different kinds of arrangements out there, and I think part of it goes back to what Kati was saying earlier about local jurisdiction, is like you have to I think let a lot of this fall down to local jurisdiction and have them figure out...

within broad-based parameters for how the program is going to operate for people at the local level to determine how it is that they are going to receive tips and who is going to be responsible for them in a way that's most effective for those communities.

>> Excellent.

Yeah, question, Sarah? >> Follow-up to that.

I'm wondering.

I'm also in Colorado, and we are working on a project where we're helping rural school districts connect and integrate their systems for threat assessment with anonymous reporting, and one of the things I'm finding in this work is that there's not that standardized or even guidelines for how a team that investigates these tips should be functioning.

Have you seen any examples where states have built a good process for how that should work, what the steps are to take, how you follow up on cases or hand off this child to the right resources? >> Well, I think I can address in Oregon and speaking...

There's experts there, but say on Kaiser Threat Assessment model has been in functioning I think since 2000, and they have a whole tiered system.

You get a referral, and then tip lines are relatively new for most schools what we found out in the last 5 years, in part because of the funding of federal government stopped granting others, the implementation of state laws, and what you find there is that there's a systematic process, I think exactly what Al was talking about, is who gets the tip or the referral is usually a principal, a counselor, possibly an SRO involved in that team, and then there's a decision made to elevate it to the next level where you have a multidisciplinary team, which might involve other folks, other parts of the community.

And I think that's where there's a lot of unknown.

The functioning of the multidisciplinary teams...

Dewey Cornell's work also touches on this where they talk about recommendations or best practices about who should be there, how often they should meet, what kind of training they should get, et cetera, et cetera, but we know very little from my assessment about that whole process, and even if there is a process in place, what is the fidelity of that process at the school level, right, where everything happens? And you see in his work in Virginia at the beginning, everyone had a mandatory threat assessment process but only half actually had a threat assessment underway, and it has grown.

So, in Oregon, they have a new program, a relatively new program where they're developing regional multidisciplinary teams that work with the school level, and I don't know if there's folks from Oregon on, but just to give a...

The opportunity here is that many places don't have the resources to handle these types of incidents and cases, so what those regional teams do are try to support where there is need, and in Oregon, it's unique, too, like every other place in America, right, is that you have these urban areas, but then you have not only suburban and rural, but you have frontier places.

And even access to local law enforcement, the sheriff might be many miles away, and so doing these types of check-ins is just not possible.

So having regional support is one thing they're trying to roll out, and I'll say even further, they have regional support on behavioral assessments and then also support on suicide prevention, very different models, so they're bifurcating that because they feel it's just a different response from what I understand.

Any other thoughts about that? Let's see.

Looking through some of the chat here.

So the one question around 911 call centers and integration of anonymous reporting systems, and some places do have a line from what I could gather from what we did with our survey that they have a heavy influence from law enforcement to respond, and of course, that's the number one.

If a tip comes in, and it is a very serious case, you just call 911, right? You're not handling it through these other teams.

That threat level is immediate, and in Oregon, we have so few of those, just like I think most people.

These are rare events, and a lot of it's on suicidal ideation or prevention, and so then you get down to the level two and threes where they need immediate action but not necessarily a law enforcement response.

So I don't know if there's other models where the 911 call centers are actually integrated, right? Nine-one-one is the quintessential tip line, right? It always has been, and maybe schools are just catching up on capturing that part of prevention.

>> The other thing, too, I was going to say, Mike, is that it's interesting to think about this bifurcation between mental health and law enforcement, but I think the reality, too, is both sides have to be prepared and coordinated to work with the other.

However, whoever is operating the system, wherever the calls are going, it's never going to be quite so clean-cut, right, and so both sides have to really be well prepared, well trained, communicating well to determine who they're going to give a hand off to if that's going to be the case.

In many ways, the tip line...

This is going to...

Pun intended here, right? The tip line is just the tip of the iceberg.

It's the entry point for a whole slew of things that have to follow, and you have to have everything, again, well thought out, use best practices as they're available to then figure out how you are going to be responsive and have the right parties respond and have the right parties communicating with one another.

All of those things have to take place, and the tip coming in is just the beginning of it.

>> Yeah.

I think one point in chatter is good, the mental health therapist working with the tip line.

Well, that's the variability, right? A tip line is usually where someone provides information whereas that might be more along the hotline, if you will, where you actually call, and you get a live voice, and actually, you're looking for a referral system, and that's not always the case, and I would imagine that...

I will have to check what we found, but a tip line is really more of a form you fill out, and it goes to a third party, the school administrator directly or a vendor.

In the case of Oregon, it goes to a vendor, and it's triaged to the right school, and then from that...

And that's where we're talking about insufficient information.

You're not really talking with somebody, I think Kati was mentioning.

That model is not what we have versus having a staff that's online receiving calls and handling them through more sophisticated training, so variability is out there.

Again, I think it's largely driven by resources and what they could afford, and that's it.

So it gets into the sustainability, also.

If you're hiring a staff of a dozen people to sit 24-7 to handle calls coming in across the state and to triage them, whether it's law enforcement, mental health clinicians or et cetera, that's a very different model than a reactive one through a vendor.

>> And I was going to say to that point, Mike, it really is a matter of what you have available and what resources have been allocated to that, and Colorado has been fortunate and unfortunate because we have had so many different experiences that after Columbine a tip line was created in response to that, and so we've had it for a long time.

I believe it's been around since 2001 if I am correct, Safe2Tell has, and so we've had a lot of trials and tribulations with that, and so we do have what's called the CIAC, Colorado Information Analytics Center, through the state, and they are the staff that answers those phone calls, so we have three different lines.

They can either submit a form, do it via mobile app or actually call into a hotline, and there are people monitoring those phone calls and those tips through the app so that they can have live conversations with those kids to get additional information because like you said when you have an incomplete report, there's nothing more frustrating than trying to figure out, what am I supposed to do with this? What steps am I supposed to take? And then they're sending that to local law enforcement and the school at the same time, and then based on how law enforcement and the school have kind of parceled and delineated out how they're going to respond, then that's how the response happened, and I know that Colorado is working right now and has been working that if it is a mental health concern, they can be referred over to the Colorado Crisis Services so that they can get direct help immediately through Colorado Crisis Services for mental health concerns, so that is a new addition to the Safe2Tell process.

>> We're getting great conversation, and I think we have about 5, 6 minutes left.

One thing we're really interested in understanding is, really, what's the return on investment with a tip line? And you can do the numbers pretty easily when you save one life, right? It kind of makes sense, but really, the return on it is, what's the independent contribution of the tip line over and above traditional reporting methods, right? And I think what's really interesting is, one of the things we're trying to look at in terms of the tip line is just not how many averted attacks did it...

How many attacks did it avert? But, necessarily, did it change the climate, right? So people are now more comfortable with reporting even minor events, so they don't rise to the level of a incident, right, and so trying to measure what you would think in terms of crime and discipline within a school.

Once you implement a tip line, that logic model is a little bit funky, right, because what you're trying to do is prevent, and it's just not trying to measure something that didn't happen, but it's really about the seriousness of a case, so instead of something gravitating to someone bringing a weapon, you actually hit it when it's a low-level offense, and you address it there, so does it become a discipline record? Does it become an incident record? And so trying to link up the adoption of a tip line in a school and changes to discipline, discipline disparities and other school safety measures is a little bit odd, and I wonder what others think about that.

I think you end up measuring more about these gray area investigations, and the concern I have is of course, are they really measured? So if someone thinks there's a problem coming up with a tip line, now you have something papered up, right? So you have something.

What's the disposition? What happened to that case? And whether there's follow-through or a written document versus in the old days, I guess, or even now you would have someone tell a teacher or administrator, and it may not ever be reported, or it may be handled, so now does it change, this technology, this adoption, change how things are recorded and how they appear in the record systems is something we're trying to get a handle on.

>> I can tell you as a former district administrator, yes and no.

Things are put into the P3 system.

I worked with Mesa County in Colorado, which is the Grand Junction area if you're not familiar with Colorado, the western side of the state, but things would go into the P3, but I still had my administrators transfer that over into the Student Information System based on the events that happened, so basically, like Al said earlier, that's your foot in the door.

That's cracking the door open to get your foot in the door for an investigation, but if it led to a threat assessment, there is a tracking area.

There is a separate place where that threat assessment needs to be stored within a system or a suicide risk assessment or a disciplinary action or that kind of stuff.

So you'd still have to close out the tip in the reporting system to say a threat assessment was processed or a threat assessment was done, but then I better see a threat assessment in the actual system, and the district was fortunate enough to where I oversaw both, so I was the one making sure, "We have this tip.

Did the administrator open it? Did they say what they were going to do, and did they actually do it?" so was there a cross-reference of what happened and where are we documenting those processes and disciplinary actions if they happen.

>> Excellent.

>> Yeah, we're dealing with the same kind of thing in Nevada, as well, and we use P3 as the communication and tracking system, and the school districts use...

The districts use Infinite Campus as their student tracking data, and getting those things...

They don't communicate with one another, but getting data to cross-reference is a real challenge.

Number one, right, you need to have the people reporting into P3 be as accurate as possible.

What are those actions taking place? If a threat assessment, right...

If a threat assessment did take place, it should be in P3, and it should be in the student's records, but that's been a challenge is getting people to use, even the multidisciplinary teams locally, to use the reporting system as effectively as they could be using it, so that's one real challenge.

And then, even then, if you really I think want to follow up on what really happens with those tips, then you do have to dig into those student records.

You have to follow the tip forward to see, were the assessments done? Were services provided like you would hopefully expect services to be provided? It can be kind of a long trail that you have to follow to kind of figure those kinds of outcomes out.

>> That's a good point.

Al, because we were talking about tip lines, and then we're talking about safety or threat assessments and immediate dispositions, but it's really about that longer-term return when you refer a student to services.

Okay, so what then happened to that student, right, and is there a repeat event? That's the longer-term question, here.

Didn't mean to cut you off, but.

>> Right.

No, no, you're exactly right.

And it's never quite as cut-and-dried, and then you write the other...

There are all sorts of...

From a research perspective, there are all sorts of challenges here because, like, we're challenged to know, was SafeVoice the only mechanism by which the information was coming into the school? Right? SafeVoice is meant to be a mechanism.

It's not meant to be the only mechanism.

Maybe it's even a mechanism of last resort because you'd like people to be reporting things potentially to a trusted adult in person if you could, right? So SafeVoice is there as one mechanism to report.

It doesn't mean it's the only one being used, and so that raises questions about, well, what is the return on investment because can you really easily tease apart whether they're using this mechanism versus other mechanisms or they're using a bunch of mechanisms.

There are just a lot of challenges to tease those things out.

>> And I know we're at time.

I would just say, though, to that: They are using the tip lines, right? >> Yeah.

>> We know it, so.

And are they satisfied with those others, or they want redundancy, but they are using them, so I'm happy that I can justify some return on investment there, that they're actually being used versus no tips, right? >> Absolutely.

>> So, yeah.

>> Well, thank you so much, Dr. Planty, for taking the time to talk with all of us and to engage us in this fantastic conversation.

The next set of sessions begin in 15 minutes at 3:30, so you have a little bit of a break now.

I mean, I can leave this room open if you guys want to keep having conversations.

>> I'm free.

I can stay until everyone drops off.

>> Well, I'll go ahead and stop the recording for now, and I'll let any kind of impromptu conversations go ahead and continue.

>> Well, I think one thing...