Dual System Youth: At the Intersection of Child Maltreatment and Delinquency

MARY JO GIOVACCHINI: Good afternoon, everybody. And welcome to today's webinar, "Dual System Youth: At the Intersection of Child Maltreatment and Delinquency," hosted by the National Institute for Justice. At this time, I am going to turn the webinar over to Barbara Tatem Kelley.

BARBARA TATEM KELLEY: Hello. And on behalf of the National Institute of Justice, I would like to welcome you to our Research for the Real World presentation today on "Dual System Youth: At the Intersection of Child Maltreatment and Delinquency." My name is Barbara Tatem Kelley, and I privileged to serve as the National Institute of Justice Social Science Research Analyst in the area of Dual System Youth. I would like to note that in recent months, I've been working with Paul Haskins, a Technical Writer for NIJ, on an upcoming article on this topic for release in an NIJ journal edition, which is being dedicated exclusively to youth-related research topics. I'm very grateful to Paul for embracing the importance of dual system youth and encouraging today's webinar to take place. This work has been supported over the years by the Office of Juvenile Justice and Delinquency Prevention, also known as OJJDP, which is our sister agency. In fact, the primary study we will--we will be discussing today, the Dual System Youth Design Study, was OJJDP's response to a congressional mandate to examine the topic of youth served by both the Child Welfare and the Juvenile Justice Systems.

Today we'll be hearing from Dr. Denise Herz who served as the Principal Investigator on this study along with Carly Dierkhising who served as a Co-Principal Investigator. Both Dr. Herz and Dr. Dierkhising bring their extensive knowledge as criminologists at the California State University in Los Angeles to bear on this topic. They are a dynamic duo and a force to be reckoned with. In addition, we will be joined today by Magistrate Richard White of Mahoning County Juvenile Court in Ohio, who has dedicated most of the last decade to advancing the identification of his court involved youth with dual system experiences and the delivery of cross-system collaborative case management and services. He also served as an advisor to this particular study.

But before getting to this contemporary discussion, I would just like to note that this topic is not new to our field. As early as 1984, the Office of Juvenile Justice and Delinquency Prevention convened 31 experts spanning the fields of Sociology, Criminology, Psychology, Law, Medicine, Social Work, Juvenile Justice, Philanthropy, Child Abuse, and Child Advocacy to address the state of the art regarding the relationship of child abuse to delinquency. And as noted in the symposium's report, and I quote, "Child abuse and delinquency are not separate problems. They are intertwined in known and unknown ways. Isolated statistics and separate studies have existed for some time, and common sense leads one to postulate a strong link.” The symposium experts went on to determine that retrospective studies of juvenile delinquents consistently found that these youths experience maltreatment at rates much higher than the general population. The authors noted shortcomings of existing research and terms of inconsistencies and definitions, lack of comparison groups, and reliance in self-report or official records rather than both. The authors recommended further research in development focused on child abuse prevention and coordinated intervention for delinquent youth who experience child abuse. The research, prevention, and intervention issues raised in this 1984 symposium still permeate our current research practice and policy.

Over the decades of the past, progress has been made in terms of research examining the life course events, punctuated by children and youth involvement across both the Child Welfare and the Juvenile Justice Systems. In the 1990's, under the Rochester Youth Development Study, research has examined the history of child maltreatment and child protective services intervention among a general population sample. This longitudinal study provided strong evidence that youth who experienced maltreatment during childhood displayed significantly increased risk of at least 25% for adolescent problems, including serious and violent delinquency, drug use, low academic achievement, symptoms of mental illness, and teen pregnancy. Then more recently in 2015, OJJDP worked with the National Center for Juvenile Justice to convene researchers and practitioners to explore the feasibility of utilizing child dependency court records to address this topic. These experts determine that many dual system youth were not being handled within the Child Dependency Court. What is needed is linked automated administrative records across Child Welfare and Juvenile Justice to track, dual status cases and individual youth outcomes. In Denise Herz and Carly Dierkhising's contemporary literature review, which they just recently completed with their study, one of the major observations across the study is that a far lower percentage of child welfare recipients prospectively cross over into the juvenile justice system than the percentage of youth in the juvenile justice system identified as having retrospective involvement in child welfare.

Being involved in child welfare does not mean one is destined to become a delinquent. However, when contrasted with youth only involved in the Juvenile Justice System, dual system youth exhibit higher levels of mental illness, substance abuse, educational challenges, such as truancy, suspensions, and lower academic performance, and recidivism. Dual system youths were also found to be more likely as young adults who experience homelessness, incarceration, and unemployment. Research findings such as these have informed the groundswell of action nationwide to better serve these youths. Recognizing the negative consequences associated with dual system involvement, researchers and practitioners have emphasized the need to reframe policy and practice to increase the: one, efficacy and delinquency prevention among the child welfare population; two, systematic identification of dual system youth; three, collaborative case management across child welfare and juvenile justice; and four, provision of trauma-informed services. Collaborative efforts in over 100 jurisdictions in the United States have been identified by Dr. Herz and her colleagues. And these have been guided and supported to the training and technical assistance delivered by the Center for Juvenile Justice Reform at Georgetown University, and by the Robert F. Kennedy Children's Action Corps National Resource Center for Juvenile Justice.

What happens now? There remains a key need to improve the capacity of jurisdictions to establish and utilize integrated data systems. Integrated data systems can help to increase understanding of the incidents, circumstances, trajectories, and system responses to dual system youth, and to identify promising and effective practices in cross-system collaboration to improve life course outcomes. I really appreciate you joining us again today, and I hope that as you listen to the conversations which follow, you will find ways to utilize this information to improve your practices, your policies, and your future research. And now I'd like to turn the mic over to Denise Herz. Thank you.

DENISE HERZ: Thank you, Barbara, for that phenomenal review of the work around dual system youth, which is not new at all, as you said. And I believe I speak for Carly when I say that we are both privileged--feel privileged to be part of this work and I feel as if we both stand on the shoulders of many giants that have been part of recognizing this work and documenting this work along the way. I believe what this study has been able to do is really fill in some of the gaps that we--throughout the course of looking at this issue, new or existing, but hadn't really gotten to the point where we had enough data to document it in a convincing way to really drive practice and policy forward in a intentional and deliberate way. So, I hope that everyone enjoys today's presentation. What we'll be doing is reviewing the results of the study, but also really filling in more details in the story that Barbara just laid out, which I believe, of course this is a personal opinion, but I believe we are getting to the point of data documenting findings that are truly pushing us to recognize the importance and impact of the intersection of maltreatment and delinquency. So with that, let me begin.

So as Barbara introduced us, I'll just take a quick moment to, again, introduce myself and Carly. We are both from the School of Criminal Justice and Criminalistics at Cal State University, Los Angeles. And we were co PI's on this project. We worked very closely with Richard White, who is just a phenomenal advisor to the study, and we have relied on him to bring a lot of illustration from practice to reinforce what we're finding in the data but also inform us as we look through the data. So, Magistrate White will join us after Carly and I present the results of the study. So my next few slides, I'm not going to dwell on, but this work could not have been possible without the progression of research that Barbara gave a review of, but also our collaborative partners. We headed the study, however we were very much arm in arm with so many researchers, phenomenal, brilliant researchers that really made the findings and the work that you're going to see today possible. So, we had collaborators from Chapin Hall at the University of Chicago, from the Center for Innovation through Data Intelligence in New York City, through the Center of Urban Poverty and Community Development at Case Western Reserve University. We also worked closely with Westat on some of the methodology work that we proposed. And we also had a wonderful partner in Actual Intelligence for Social Policy at the University of Pennsylvania who were very insightful about using administrative link data, and the impact of that. And then of course our partners at the Center for Juvenile Justice Reform at Georgetown University were critical in facilitating our look at the best practices in the field.

Now what we have today is the opportunity to also include not only results from the original adult system design study, but since that time, we have been able to expand the research to Los Angeles County. But I forgot I had this acknowledgement so, like, sorry about that. So we also want to acknowledge, again, Georgetown University Center for Juvenile Justice Reform for their use of data, but also Cook County, Cuyahoga County, and New York City, all of the agencies that provided the administrative data that allowed us to link that information. And then similarly, moving on to the Los Angeles study, we do have this extra bonus because of the time that had passed since we've completed the study, as a result of a collaboration with the Children's Data Network at the University of Southern California who has been doing a lot of work with administrative link data and has been part of the dual system youth conversations for quite a while. As a result of funding provided by the Reissa Foundation, as well as other foundations for CDN generally to support their data infrastructure, we were able to replicate the exact methodology that we used in the dual system design study using Los Angeles County data. So this slide just recognizes CDN, the Reissa Foundation, and the other foundations critical to allowing for the analysis. And also those who allowed us to use their data, California Department of Social Services, and then Los Angeles County Department of Children and Family Services, Probation and Staff at the CDN.

So, let me lay the basis of the OJJJ--OJJDP Dual System Youth Design Study. In its origination, there were two primary goals. One was focused on how do we get a dual system youth incidence rate for the nation in a representative way? The study was not about producing that estimate per se, but rather to explore how that estimate could be produced and what would the cost of it be and what would be the most efficient, and effective, and most accurate way to produce it. And so we partnered with all those that you saw on the slide to really access the data that they had been using and their skill and expertise around that data to test out: how could we best get at an incidence rate for dual system youth? The second major goal of the project was to begin to identify best practices in cross systems collaboration and figure out a way, and propose a way, to begin to measure and collect information across the nation again on different jurisdictions and their approaches; what their success was with those approaches; and what were their challenges, ultimately beginning to identify those promising and best practices to support cross systems research for better serving dual system youth in their family. So that was the groundwork of the design study and what we started with. Then, of course, we had the fortune of being able to replicate the incidence rate work in Los Angeles County. And the idea was the same basic goal, which was to really understand what was the incidence rate of dual system youth and what were their characteristics and experiences.

So all of that today is what we'll be covering. And our presentation will really focus on three elements, summarizing the results from the Dual System Incidence Studies, summarizing Best Practices from the field and what we produced as a best practices rubric, and then, illustrating best practices from experiences in Mahoning County, Ohio. So Carly will be handling and illustrating how we went about best practices piece of this particular study and an overview of the rubric. And then, of course, Magistrate White will give some insight from his application of Best Practices in Mahoning County, Ohio. I want to just mention, and you'll see this link several times, there are quite a few elements to the study, and the report has multiple chapters of findings. We can't produce--or I'm sorry, present all of those today, so we encourage you if you were intrigued and want to read more about the methodology or the additional results that we've produce, this is the link. You can also search for it in a search engine and it usually comes up pretty readily using the keywords Dual System Youth Design Study.

So moving into the actual result. One of the biggest obstacles, and Barbara mentioned this in her opening comments when she summarized the findings of the 1984 workgroup, it has been clear for many, many years that there are some barriers to truly understanding the overlap between maltreatment and delinquency. You know, there's many--we could probably have a whole presentation today on those alone, but they really fall into three basic categories. One is that we often think of youth that touch both systems as crossover youths, that has been a traditionally common term used to identify those youths. The only problem with crossover youth is that it is used pretty loosely. And it's always not the only term that is used. So the various terms and definitions that have been used across studies and jurisdictions can create some confusion around really clearly understanding who this population is and what their characteristics are. There are no national studies or estimates of incidence in the nation, so we are limited to smaller studies. Now, there is some consistency in those studies and we're going to see some of those consistencies bear out today when I go through the results. And finally, going back to the use of crossover youth as a term, that term has been extremely helpful in identifying a category of youth to touch both systems, but it is all-inclusive. And one of the things that we started to see when CJJR was implementing the Crossover Youth Practice Model, when jurisdictions were talking about the crossover youth in their jurisdictions, there would be different ways in which these youths were touching both systems. And so we started to think about that as pathways, right? And the notion of pathways also reverberates in some of the literature and some of the development that Barbara was referring to as well. But we haven't really gotten to a point where those pathways had been defined.

So, this study really started at that point, and we all, as a collective, we had a lot of people around the table, we said, okay, these are the critical issues from which we need to start. And so this study for the first goal around incidence rate started with finding dual system youth in their pathways. Then, as I said, we capitalize on the expertise and access of our colleague, partners to look at linked administrative data from Cook County, Cuyahoga County, and New York City. Using that data, we produced incidence rates for dual system youths in pathways based on our defined pathways for youth in a cohort of first delinquency court petition across all states. And here's an example of what we're not going to cover, but I refer you to the report. We also did the same analysis for our first arrest cohort. We compared characteristics and experiences across pathways for both cohorts. We also compared young adult outcomes for that first court delinquency court petition cohort. We then said, look, you know, we've got all this. We've got a lot of work. And trust me, the partners were like, "Denise, Carly, we have done a lot of work." I mean--but we all said, yes. But there's one more thing that I think we need to look at. Up until this particular part in the development of the study, everything has been based on conceptual or theoretical definitions of pathways. What we wanted to also test was whether sophisticated statistical model such as sequence analysis could look at the data and identify pathways. So, we'll be talking about everything you see with an arrow will be covered today in some part, and the items with the sun, I'll call it a sun, are in the report, and I refer you to the report to further explore those issues.

So, moving forward and really highlighting the key findings that we found from linking the administrative data in all three of those sites, and then, as I said earlier, the replication in Los Angeles County. The first step though, for everyone, is to just clarify how we distinguished key terms. So, we still kept crossover youth in the nomenclature, but we really felt that that was this big umbrella term. And we wanted to dig in a little bit more to actual system contact and begin to define that in very specific ways. Researchers love very specific definition. So, what we decided to do was use the term dual system youth to indicate crossover youths who touched both the Child Welfare and Juvenile Justice System. So, dual system youths are specifically saying these youths had in fact touched both systems.

What was our methodology? This I'm going to cover very quickly. You can ask me questions, and then, again, the report is a great go-to for any further detail. But essentially what we were able to do, because of our partners' access and agreements with their government agency partners and their expertise, we were able to--at each of the three sites under OJJDP Dual System Study to be able to identify that we had all of the juvenile justice data. And I say juvenile justice because it came from multiple agencies, right? So, sometimes it was from probation, sometimes it was from multiple courts probation, so all of that data had to be collectively matched to be juvenile justice. But we had juvenile justice data and we had child welfare data for each of these areas, including Los Angeles. So, I'll include them in this discussion because we applied the same methodology. And we looked in that data, we said, okay, let's take all the youth who, between 2010 and 2014, had their first juvenile justice petition to court. So these would be youths that are being adjudicated in the Juvenile Delinquency Court. If they were just arrested but not petitioned or sent to court, they would not have been in our cohort. So, we wanted to have a clean kind of apples to apples comparison of the same part of the process and the decision-making around juvenile justice. And you can see the numbers for each of the jurisdictions. The two caveats to mention here: New York City because of data issue had to limit their cohort for 2013 and 2014. And then in Los Angeles County, we were able to update the data given the time had elapsed, so we have 2014 through 2016. But we did have to limit that cohort to those who were born on or after 1998 because of the access to child welfare data. So, as you all are listening to what we were able to produce, if you're thinking about your own jurisdiction, the catch is that you would have to be able to understand how far back your data go because we were, at this point, looking at the youth who were getting their first delinquency petition during these time periods, and then looking backwards to child welfare data to see if they had touched the child welfare system. So that is the basis of all of the results you're going to see today.

And that lets us jump into our first data point. So the big question was incidence rates across sites. And when we looked at OJJDP Study Site Cohorts, what we were--really I think we all thought and hypothesized that the incidence rates would be, you know, somewhat similar, but in some ways, they were shockingly similar. So in the three sites, Cooke County was a little bit of an anomaly. And, again, what we're going to see here is probably some different policies and practices that vary from jurisdiction to jurisdiction. But even with that, they were at about 45% where Cuyahoga County and New York City were just over two-thirds of all the first juvenile justice petition youths having touched the child welfare system. And in Los Angeles County, we were, again, a bit shocked to see how similar the incidence rate was for those youth. Sixty-four percent of youth in Los Angeles County with their first juvenile justice petition had touched the juvenile--or child welfare system at some point in their lives.

Now I should stop here and pause and give you one more piece of a definition because many of you are probably saying what does child welfare system contact mean? And in this study, in all of these studies, child welfare contact meant that the youth had had at least one investigation by the child welfare system. So that investigation may or may not have been substantiated but they at least had one investigation that was pursued by the child welfare system at some point in their lives. So for some of these youths, it could've been when they were born. For some of them, it could've been when they were eight years old. And for some of them, it could've been when they were 16, 17. So it did not matter what age they were. They just had to have a record of a child welfare investigation. And when we looked at that, again, this is a really fundamental finding that I think changes our way of looking at dual system youth. Half to two-thirds of all those with the first juvenile justice petition had touched and had at least one investigation by the child welfare system.

So who were these youths and what did they look like demographically? One thing that is consistent in research going back to the 1980's that looked at youths who touched both systems is females are more likely to be in the dual system population than they are in the juvenile justice only population. So what we were able to do, and what you're going to see here in the next couple of slides, is you're going to see the youth with first juvenile justice petition that had no child welfare background or touching, and compare that to those that are dual systems who had the child welfare contact. And you can see there's about a 10% increase for females in the dual system population. Correspondingly, we also see that Black youth were more likely to be in the dual system population. These results are identical to what CYPM has produced for all their sites where we see an increased percentage of females in dual system populations, and we see an enhanced amount of disproportionality in the dual system population.

So one of the things that, in the LA cohort that we were able to do, we did not do this in the OJJDP study, but my colleague, Andy Eastman, said, you know, let's look at the rate at which the youth in our first juvenile justice cohort were likely to become dual system. In other words, what percentage of our juvenile justice cohort, by race and gender together in intersection, were likely to become dual system youth. And what was truly pretty shocking was when we looked at these numbers, you can see the green represents male. White male, Black male, Hispanic male. In all three categories, males are less likely to become dual system or have a less likelihood of being a dual system youth than their female counterparts. Females overall are more likely to fall into the dual system population, which that's how we see that increased likely--or that increased percentage of females occur. When we look at and compare the females across race categories, they are very close. However, Black females are, by far, the highest likelihood of youth to become, or to be in, the dual system population. So we're starting to really see how that race and gender combination is impacting and related to dual system population.

What we don't know is why that is happening, right? So we wanted to dig in a little bit more and that meant that we had to understand better about dual system pathways. So there are multiple ways in which youth can touch both systems. So essentially in this study with our framework, we said, look, it really boils down to you can have dual system contact concurrently or you can have it non-concurrently, right? So non-concurrent is when you're touching but not at the same time. Dual--and concurrent is when you're touching at the same time, simultaneously. So we distinguish these groups by the term dual contact use and dually-involved youth. We then had to really drill down a little bit because what we found was that it wasn't so simple. When we really looked at it, we saw that we could distinguish dual contact from dual system or dually-involved. That was easy enough. But we then saw the complications. Some of our dual contacts, who were not touching at the same time, touched the child welfare system first and some touched the juvenile justice system first. And then same thing for dually involved, some were touching the child welfare system first and some were touching the juvenile justice system first. But what was interesting, so in many ways, this can be simplified in that you--touch system at the same time or not at the same time, and you either touch the child welfare system first or the juvenile justice system first.

But the slight complication was that for many youths, what we were finding was that they had a previous but closed child welfare case, right? So even here, and I'll highlight this group and then I'm going to show you numbers, this group dually involved juvenile justice first with a historical case. It appears, at first glance, that they're touching the juvenile justice first before they have their child welfare contact. But actually, they're touching the child welfare system first. But that case ends, and then they get involved in the juvenile justice system, and then there is another investigation by the child welfare system. So we wanted to see, how do these pathways look? Are they equal? Is there a particular pathway that matters more than others? So what I tried to do here, because this can be a little confusing, but I tried to show you with different demarcations the comparison between OJJDP sites and Los Angeles Cohort. The big news is that 58% and 52% of all of the cohorts were dual contact, meaning that they weren't touching both systems simultaneously. That's a huge finding because we always think of these youths as simultaneously being involved. And we always measure them as being simultaneously involved. But the truth is that most of our dual system involvement is not happening simultaneously. And so if we only look at simultaneous involvement, we lose the information and the knowledge to really drive appropriate programming, because we don't know or we don't consider the maltreatment history. Okay.

And then I'm going to just talk about OJJDP here for a moment and the big group here is the dually-involved child welfare pathway and the dually-involved child welfare pathway with a historical case. So, essentially, these are the same in the sense that they're both touching child welfare first and they're having simultaneous involvement. About a quarter of them, of the youth of dual system involvement, fall into that category. And then for the OJJDP sites, there's about 10% that touched the juvenile justice system first, but had a historical child welfare case. One thing to point out because after this, I probably won't talk much about these two groups: the dual contact and the dually-involved youths that touched the juvenile justice first and then child welfare are very small. Only 5% total had touched the juvenile justice system first without previous child welfare contact. When you look at Los Angeles Cohort, it is essentially the same finding. The only difference is that for some reason, in Los Angeles, the larger group is the dually-involved juvenile justice first with a historical child welfare case. And that is followed by the dually-involved child welfare cases. So these three groups in both--in all of the sites for the predominant site--groups that drive dual system involvement when there's simultaneous involvement. The big takeaway here, which we'll return to, is that almost all of these youths, you can slice and dice it whichever way you want, but almost every one of these youths are touching the child welfare system first, right? And we're going to return to this in a moment, but I think there's a lot of policy implication there. Okay.

So the next question that this leads us to is, okay, well, you will complicate this, Denise and Carly, and make all these groups, what does that mean? Is there any meaning to that? Does that complexity--is it necessary? And when we looked at their characteristics by pathway, we concluded that, yes, actually, it is important to know about those different pathways because when we looked at those who were not touching the system simultaneously, the dual contact child welfare first youth, we saw that they were males overall. They were largely Hispanic females and males in Los Angeles. And they were very similar in that they had fewer investigations, cases, placements, they had shorter stays in child welfare, they had lower detention and juvenile justice, and, ultimately, they had lower juvenile justice recidivism. These three groups, however, in all three sites, there were slight variations in which one was higher or lower, but all of them in a graduated way with child welfare first, and dually-involved indicating that they were the most system involved, they had more investigations, more cases, more placement, and we're not talking one or two. The difference was quite noticeable especially with placement, they were staying longer in child welfare both in care and in placement, they had higher detention rates in juvenile justice and ultimately their recidivism was higher as well. So all this leads us to understand that it does matter, the pathway by which these are touching those systems, and those that have deeper more entrenched involvement are going to, ultimately, have poor juvenile justice outcomes.

So as I said, in the OJJDP report, we were curious because up until that slide, everything had been based on the theoretical conceptual category. So we wanted to see if we were able to link that data as the teams did and use sequence analysis which is a sophisticated statistical process by which the analysis or the computations actually go through the data to find the patterns, right? And our partner groups led this analysis. I learned a lot. And they produced this slide and this slide is I think wonderful because it visually really captures what we're trying to say in all of these different data points which is there are particular pathways, there are particular trajectory by which youth are touching, getting involved in, touching both systems. And you can see there the sequence analysis, the statistical modeling produced. Think of it as they're going through the data and figuring out which comes first, right? And by doing that, they were able to identify four different groups, and they were called clusters but you can see that each one is described here. Similar to what I've just presented back here, we saw less system involvement graduating to more system involvement.

The same thing was produced in the sequence analysis by the clusters. There was a large group of youth that had limited and late child welfare involvement and then we start to see the darker colors come in. The darker colors represent more involvement in the child welfare system through placements, through court involvement and so forth. So cluster two, we start to see more blue but still a lot of pink and, kind of, background fill, then as we go into cluster three, long duration, we start to see the blue and the purple come in and we also see the long duration with out of home placement. So the analysis, the empirical analysis, replicated in some ways, not necessarily exact same definition, but replicated the basic pattern of findings that we have used, that are touching both systems, but they're not touching both systems in the same way. And the way that they touch those systems appear to have consequences with regard to their juvenile justice outcome and how deeply they penetrate the juvenile justice system and how that impacts their long-term adulthood outcome because these youth, those that are more involved not only have more severe or more involved levels of contact with child welfare before their involvement, but when we look at young adult outcomes we see the same things. Those that are more involved are exhibiting poor outcomes in young adulthood. That tells us a tremendous amount. They help us really drive practice and policy forward.

But before I can summarize that piece, let me just summarize and highlight some of the key findings that I think really begin 40 years later practically to define in pretty specific ways what dual system involvement means, or dual system contact means. So when we look at incidence, even though we have only four sites, all four a very large metropolitan diverse communities and jurisdictions, and they are all showing similar results between half--I'm going to take the liberty of rounding up on that 44.8%, to about two-thirds of all first juvenile justice system youth, were dual system youth. That is a game changer in many ways. We don't typically think of juvenile justice population being equivalent to having maltreatment history. We see it as a special population. I think that begins to really question our thinking there. We also know that females are more likely to be in this population than males, and Black females are the highest. Ninety percent, over ninety percent, touched both the child welfare and juvenile justice system, but they touched it through the child welfare system first. And when we look at the levels of involvement females and Black youth were more likely to have deeper system involvement.

And finally, another key finding that has significant implications for practice and policy is that approximately 50% of our dual system youth did not touch the systems at the same time. So a juvenile justice assessment for example, is not going to necessarily know that that youth has been a victim of maltreatment or a potential victim of maltreatment. They won't necessarily know that they have touched the child welfare system, unless they are specifically asking those questions and potentially working with the child welfare system to understand what is most appropriate for that youth. And again, just a minor but very significant detail is that a third of the dually involved, those that are simultaneously touching both systems had historical child welfare cases. So even though they're 15, 16 now, they had touched the child welfare system when they were very young.

So all of this taken together. I think all of this taken together could be a two-hour discussion on practice and policy, but let me drill it down I think to the most important points. Dual system contact occurs for the majority of youth involved in the juvenile justice system. We said that for almost 40 years we've had little pots of data to support that finding, but these four sites are really beginning to convincingly show that maltreatment and experience with the child welfare system matters for youth in the juvenile justice system. The pathways matter, knowing how they touched matters quite a bit with regards to their level of involvement, and all of these really brings us to prevention is essential. And we can't think of prevention just as early primary prevention or secondary prevention at the very beginning levels. We have to include that, but we have to think about prevention as a continuum, beginning with preventing maltreatment, by preventing maltreatment, your essence are also contributing to the prevention of delinquency, then if that doesn't work and there's a youth and family enter the child welfare system, then is about preventing or interrupting persistent and adolescent limited maltreatment. And, if they do touch the juvenile justice system, it’s presenting that delinquency from moving further into the juvenile justice system, diverting them away from the system and/or giving them intervention services to reduce the likelihood of continued delinquency. All of these required system collaboration and coordination, and Carly is going to speak to this, I'm sure much more, all of our findings really do indirectly point, and I think one could argue directly point to the need for trauma and forms and culturally gender-appropriate programming.

So when we think about these results in the context of the goals of the overall study, the last point I'm going to connect these results to is that question of a nationally representative incidence rate. So in order to look at that, we looked at the capacity for using linked administrative data nationwide, all of the work that I just presented to you was, kind of, proof of concept. Yes, administrative linked data works, administrative linked data gives us a really good sense of what's going on with dual system involvement, so how do we get that at the national level? So we had to assess the capacity, the availability of child welfare data, the availability of juvenile justice data; what would that look like in terms of representative model? I am going to refer you to the report for a lot more detail on this. Suffice to say, what we found was--that there was a good news, bad news scenario when it came to child welfare data and juvenile justice data supporting linked administrative data. Child welfare data is in better shape. All states have them, sort of, child welfare data for the state, but juvenile justice data is not structured that way for a lot of different reasons, largely, we think, because of the funding mandates from the federal government. But nonetheless, there is no easy way to say, okay, we're all going to do linked administrative data now. In order to get a national representative study sample and ultimately rate, we would need to either ultimately, in the study what we propose is doing a hybrid study of using those states and jurisdictions that do have child welfare and juvenile justice data that can be linked and then randomly sampling other jurisdictions to see if we would be able to get that data. Of course, that results, as you can imagine, into a very large price tag.

But one of the things, and that's why I wanted to include Los Angeles County results in this study, is through--Actionable Intelligence for Social Policy at University of Pennsylvania, they currently have 35 states and jurisdictions that have linked administrative data. So they have been working actively to produce that for other social issues. And that represents 53% of the nation's population. Our four sites represent four major metropolitan area. So it may be that we don't need to invest an exorbitant amount of money into a nationally representative study but rather capitalize on doing administrative linked data analysis in these networks of states and jurisdictions that are already doing that were combined with other areas that are very representative of the population because what we're starting to see through these four sites is that there's a lot of commonality. We're not seeing divergent results.

All of these, though, as you can imagine, really says that we're going to have to invest in linked administrative data if we truly want to understand the relationship, and traffic, and track the impact of policy and practice over time. The value is substantial, it not only encourages and supports information sharing and collaborative practice, but it also streamlines case planning. We are at a point, though, where confidentiality is very, very important, and there's been a lot of work to resolve some of those issues. There's still more work to be done, but ultimately researchers are becoming much more their capacity to secure linked administrative data storage and analysis have increased, as well as the protocols. So I think we are in a time and a place where we can support this work, and really learn more about these youths and begin to support and build upon the best practices that are currently already going on in the field. And that leads us to my colleague's presentation on the Integrated System Best Practices and the Best Practices Rubric that we produced as part of the study. So Carly, I am going to hand things over to you.

CARLY B. DIERKHISING: Thank you so much, Denise, for that amazing summary of just a massive amount of work that went into this project, and truly for the folks that are going to look at the report that is going to be associated with this webinar, you'll see just how much work really went into this stuff. It really was a really succinct overview of really complex data. So thank you to that.

And so now, I'm going to transition to talk about the more practice-focused piece. So the Integrated System Best Practices and the Best Practices Rubric. And really, this work, we were doing this work in parallel to the linked administrative data subcommittee. We have the jurisdictional case study subcommittee, which included a variety of stakeholders, and practitioners, and experts that were working more in the field. And our goal was really to address that second overarching goal of the project, about what are the actual best practices for integrated systems and class system collaboration work, you know, essentially, what are people actually doing on the ground, and, importantly, what is working. And once we dove into a lot of that, then that informed the best practices rubric, which I'll be showing you today.

So just a summary of what I'm going to talk about. So really, first of all, one of our key partners and Denise has already mentioned the CYPM, the Crossover Youth Practice Model, they--were just an amazing partner in this work, Shay Bilchik and Megan Stewart and other folks at the Center for Juvenile Justice Reform, because not only did they have just such a wealth of expertise that they brought to the table, but they also allowed us access to the data that they've been collecting for a long time really, and the work that they have been doing over the years. And so we had access to the CYPM data and diving into kind of identifying what these best practices are. And we analyzed the practices that were implemented by the CYPM site, as well as some outcome data that they also collect as part of their technical assistance work. And then we also were able to visit some sites that we're working with CYPM or had all--previously worked with CYPM, so we're doing this integrated systems work, in part with the folks on the ground to identify some of the successes and challenges around implementing some of these best practices. And with all of that, we then identify then kind of synthesize and distill down what were these key domains or key practices that people were doing. So what were the consistencies across sites that folks were implementing for best practices that were actually related to improving youth outcome. And then once we have that, and of course, we have the input from our diverse group of experts in identifying these best practices, so we have, probation officers, and judges, and other service providers, and researchers, and so on, on this subcommittee. And then we propose the best practice rubric, which we suggest to be used to assess kind of where jurisdictions are in the developmental stages of this integrated systems or cross system collaboration work. So I'm going to walk through that today here.

So, you know, first, before I dive into the data points that I'm going to cover, I just want to give another nod to the Crossover Youth Practice Model in CJJR. Just in case, if anybody's listening isn't familiar with them, you should be, if you're interested in this work at all, because they've been doing this work for so long, so please go and check out their website, and see what they're up to. And, you know, currently, it says on their website, and my guess is there's even more jurisdictions now, but it says there, 119 counties across 23 states. So we're talking about a lot of jurisdictions that they have worked with, so again, that wealth of expertise. And what they do is they provide technical assistance to counties on the ground to help them implement practices that are meant to improve outcomes for dual system youth and their families. So they do this through a series of phases here that you can see. And I just pulled this off their website, just to give a quick example here. And when they work with jurisdiction, is it--jurisdictions, excuse me, it is really a reciprocal process. So they are about meeting, kind of, meeting the jurisdiction where they're at, and identifying where their strength and challenges are. But for the most part, they move through these phases and helping them identify dual system youth, work on joint assessment and case planning, coordinated case management, planning for furnishing transitioning case closure and ultimately to meet these four overarching goals of their practice model, which you'll see are really were born out in the research as well that Denise reviewed. With these goals of reducing, essentially, the number of youth crossing over, reducing the number of youth placed in out-of-home care, because as you saw it, that's a huge issue, reducing the use of congregate care, and reducing the disproportionate representation of youth of color in the population. So again, just another nod to the great work that they do.

So the information sources that we had in doing the jurisdictional case studies work were these, so we had these checklists that we received from CYPM where jurisdictions who were getting technical assistance, kind of, indicated which practices that they were going to be focusing on to implement, while the CYPM is working with them. They also had been--or collecting survey data, which was kind of newer at the time, to get a sense of the successes and challenges in implementing these different practices that jurisdictions indicated on the checklist. And then also, when jurisdictions engaged with the CYPM, they are asked to also collect outcome data. So they had site-specific data on the population that they were focused on for the dual system youth. And they had follow-up data as well. So it's nine months follow-up data. And then we also conducted a literature review, which, of course, we also integrated with the input from the experts on the committee. And for those who might be interested in reading that literature review, essentially, it is included in one of the publications that came out of this work that was published in the Journal of Youth and Adolescence for those that are interested with Denise as the first author, so I'm sure you can find that and it will be referenced in the overall report as well. And then as I mentioned, we also were able to visit five jurisdictions to chat with folks on the ground to really see the successes and challenges in this cross-systems work.

So these were the information sources that we had that went into the development of the best practices rubric. Let us quickly review kind of the main points from these different information sources. So you can see here, these were the top 10 most common practices implemented across jurisdictions. So this here is representing the jurisdiction to either enhance or implemented these different practices or outcomes, and we had data on 41 different jurisdictions at the time. What you can see here is that there's a lot of different things that the jurisdictions were focusing on. So it's not like they were just focusing on one or two things. There's a lot of practices they were focused on. But there were a few here that almost all of them had to focus on, and the first and foremost, and perhaps not surprisingly, was identifying, or early identification of dual involvement, which if you did already know, after listening to Denise, this is actually much more complex than you may have thought because of this historical case issue and the concurrent versus non-concurrent involvement. But really, the CYPM, at least at the time, and this may have changed over time, given their partnership with us, so most of the jurisdictions that they were working with at the time, were focused on that concurrent, dually-involved population.

So as you can see, most of the jurisdictions were enhancing or implementing early identification, as well as improving the information sharing between the juvenile justice and child welfare systems. So those were the top two areas that were most common, followed by the coordinated case supervision between juvenile justice and child welfare. And of course, these other important practices as well, so increasing the use of diversion, increasing youth engagement, reducing the use of pre-adjudication detention, increasing family engagement, as well as improving coordination across other service systems such as mental health and education, and reducing the youth of out-of-home placement. So these were the most common practices that we saw implemented across the jurisdictions that we were able to look at.

And then also some of the reflections on kind of what was working and what was challenging, and implementing these practices, I'm just going to highlight a couple of the quotes that we saw on the surveys, that I think are just important when we think about implementation, and kind of how valuable the work actually is too, how valuable this integrated systems work is. So this quote here, they said, "Recognizing the tools that were already at each agency, it was a matter of communicating and realizing what the other had to offer and then build our work off of that." And really, what this is about is just that valuable piece of sharing information and coordinating case supervision in order to maximize your resources, right? So having the line of communication open is so valuable so that you actually know what each agency has to offer, so you can build on each other's resources.

And then another jurisdiction talked about one of their biggest struggles being the communication between juvenile justice and child welfare, and this was something that they were so struggling with. So again, we saw that, kind of, those top practices that improve information sharing or communication between systems, which is not always easy. And Magistrate White is going to give some examples of that as well. But they talked about how they were working to overcome that challenge, right? So these are not insurmountable challenges, these are issues that come up as part of implementing these types of best practices in integrated systems work. So they talked about planning a multi-disciplinary training between the systems so that they can be clear on each other's roles and so on. They believe they're breaking down these barriers will improve the effectiveness of the initiative. And then, of course, you know, when we talk about cross training and several jurisdictions talked about this need for cross training, stat turnover is of course an issue in that as well, but I think this quote really kind of highlights some of the challenges we saw across jurisdictions about just having to have--needing to have everybody in the training because there are these different languages that are being used between agencies. And so you need to open the lines of communication to understand kind of where everybody is at. And also understanding kind of the different roles and relationships the agencies have and also--these trainings are important to keep up to date on the protocols. And then of course one of the big challenges that, you know, I think every jurisdiction struggles with is not having a data system that is integrated across agencies. So we saw examples where implementing kind of how this figure out--so, you know, having that integrated data system is important for so many reasons but especially for at least identification of the dually involved youth for instance, right? And so for instance we saw one jurisdiction in particular where they were actually doing this by hand, now that might have worked for them because they were a really small jurisdiction but in a place like where I'm at, in Los Angeles that wouldn't work, right? So really having these data systems that talk to each other is something that was a challenge but also is seen as extremely important in this type of work.

So with kind of that background on the different types of practices that jurisdictions were implementing we then kind of ask the question of, well, is it improving outcomes, right? And so the CYPM had a bit of data on this, we had 19 jurisdictions that had complete data. So again this is a nine-month follow up data, there were 12 different outcomes that they collect data on as part of their technical assistance work, just showing you the top five areas of improvement here. And so you can see--and when we say improvement in this case it's a 10% increase in the outcome. So that's what we mean by improvement in this context. And you can see that just over half of the jurisdictions has fewer sustained delinquency position, nearly half had increased involvement in pro-social activities. Around 40% had increased in placement permanency and just over a third reduced their use of pre-adjudication detention and sought improvements in behavior. So really what this outcome data did, is it gave us confidence that these are the--these are important practices that are being implemented, that we need to consider these as best practices as they are--they are impacting youth and family outcomes. But it is important also to remember that these outcome data were not collected as part of, you know, of a specific evaluation process, it's part of the technical assistance that CYPM provides. So we use it to inform our work. But While we were doing this work there was this quasi-experimental evaluation published on CYPM. For those that might be interested I suggest that you also check out that paper as well.

So with all of that information in mind including the literature review and again our discussions and our experts and whatnot, so now with all these information sources that we had we then move to developing the best practices rubric. And really what we saw in the information is that there's a lot of consistency in the types of practices the jurisdictions were actually implementing to improve their integrated systems. So it's not like all of the jurisdictions, some were doing this, and some were doing that; everybody was doing a lot of similar stuff, but there was some variation in which ones, kind of, we're either maybe enhancing certain practices or, you know, further developing certain practices, right? So the variation, the extent to which they had developed the actual practices. So consistency and practices across sites, the variations in the extent of implementation and development. And so with that in mind, that is really how the rubric was set-up, so it's designed to capture these 11 best practices that I'm going to describe to you here but also the extent to which that practice is implemented in that jurisdiction.

So the 11 best practices, or domains, we'll call them practices, kind of fell into these two areas, so one was the infrastructure to support cross systems work. So the practices within this domain were interagency collaboration, judicial leadership, information sharing, data collection, and training, and then what we did is we defined what each of these practices look like on this continuum. So what interagency collaboration looks like when that practice is not in place versus--as a highly developed practice for instance. And I'll show you specific examples of that in a second. And then the next piece was the identification or the next domain, identification and management of dual system cases. So the six practices within here was identifying dual system use, assessment, placement planning, permanency and transition plans, case planning and management, and service provision and tracking of services and, again, based on this spectrum.

So let me transition here to the actual rubric so you can see an example here. And again this rubric for those that are interested in looking at it or using it, it is freely available as it was developed through this federal grant and then is included in the report that we've referenced numerous times. So, first of all, when we look at the interagency collaboration here you can see that we describe that if that practice is not in place what that means is that crosses to new teams or committees, they have not been established and key stakeholders have not been engaged. Now if that’s an emerging practice, that would look like cross-system teams and committees and key stakeholders have been engaged in the work, so maybe they're not meeting regularly. So you can see there's something going on, but it's not yet highly developed, which we defined as that the cross-system teams or committees are established, they're meeting regularly, and the key stakeholders are consistently engaged and participates in ongoing review of the work.

Now one of the things that's important to note as well is that in, kind of, the directions that we offer for jurisdictions who want to use this rubric is we really encourage systems to sit down in a multi-disciplinary manner to discuss these different practices, so they can discuss collaboratively where they fall on this continuum. So one of the things we noticed when we were looking at some of the data is that it sort of depended on who you ask where they may rate their jurisdiction. So as you can imagine, you know, there's discrepancies based on people's role and that's understandable, but that also ceased to the value of sitting down and even just having discussions about these practices in what they look like in your jurisdiction. Whereas maybe a judge in a jurisdiction is expecting or maybe assuming that these teams are meeting regularly and so on, but actually if you talk to some of the frontline folks they may have a different perspective and say no, you know, we haven't met in a long time or something. So again there's a lot of value having that multi-disciplinary discussion even just about these different practices in order to identify where a jurisdiction falls on this spectrum.

So this next practice is judicial leadership, now this can be--the practice not in place looking like there's just no judicial support or leadership, or there might even be active judicial opposition. Fortunately I have not seen that but I have heard about it, can happen. And so folks who are listening might have experienced something like that, I hope not, but on the flipside of that when you have that judicial leadership we know judges can play such a critical role as a champions, they have this critical influence and for, you know, really holding people accountable that can be really helpful in driving this work, this work in particular. So highly developed practice would look like active judicial support of leadership, their convening and leading cross-system meetings, they're driving the work, and they're providing accountability. Now I've seen in this jurisdiction, for example, and this just speaks to the issue that jurisdiction--this is dynamic, right? Jurisdictions may fall in one place at one time and another place at another time. So I've seen a jurisdiction where they definitely were on the highly developed practice side of this particular domain but then there was judicial turnover, and then they fell down the scale. They're probably more emerging at least the last time I was involved, right? So this is dynamic and so having this kind of the spectrum here and the rubric helps to kind of understand sometimes change that might be happening over time. And, again, Magistrate White can give some great examples of this as well. And I also want to note in the report we also dive in to specific examples in a variety of different jurisdictions, so Magistrate White will talk about Mahoning County, but we talk about a lot of different jurisdictions with specific examples of each of the 11 practices.

I'm just going to go over this last practice here in the interest of time. So information sharing, so for this one, let's look at the emerging practice for this. So this would be an MOU or MOA or protocols in place that allows the information sharing between juvenile justice and child welfare systems, but information is never exchanged or it's only shared under special circumstances. So there's this protocol in place but there's not this actual systematic sharing of information verse a highly developed practice where you have the protocol in place, allowing information sharing between systems and information sharing is regularly happening between the systems in a structured and collaborative manner. Sorry, I just realized that I didn't move that slide forward. So you can see that here and again all of this is available in the report as well.

So with that how then can the rubric be used? To really summarize this, so the rubric can be used in so many different ways. And I really hope that if you're listening to this, that you are already coming up with ideas, you're already trying to Google the rubric, and think of ideas about how to use this. We have a lot of ideas, and I'll share some with you that are focused really on research, and policy, and practice. And so, first of all, it can be used to essentially just identify--helping jurisdictions identify where they fall on these different practices. So this can drive the need for training. This can drive the need for a better focus on certain practices. This may be used for evaluation at the jurisdiction level because also, again, using this kind of spectrum or continuum framework in the rubric allows for that potential assessment of change over time. So maybe you want to be measuring implementation fidelity over time, using it for quality improvement and sustainability of the practices over time. And as I mentioned before too, just sitting down and having the discussion about these different practices and where you fall on the spectrum really facilitates the planning around integrated systems work at the jurisdiction level. In kind of a broader--oh actually really quick, let me give one point here, is that those discussions are really helpful, not just because of the potential discrepancies that might be occurring between what people are expecting versus actually seeing it in practice, but also what can happen is you may ask someone, "Oh, are you identifying your--the dual system use in your jurisdiction," and they go, "Oh, yeah, we do that," but once you actually dive a little deeper and ask and go about, "Well how do you do it," and actually think about what that practice looks like from an implementation standpoint, they might start to realize that it's actually not done in a systematic way. So, again, having these discussions can really drive the need for ongoing improvements and practice. And then in a broader research sense, the rubric can be used to inventory the cross systems practice, really, across the nation. So we could essentially get an inventory of where all the jurisdictions are at, or maybe even in a state, or a random selection of counties across the nation, which, you know, we'd love to see, an inventory where folks fall on this continuum. So we have, really, a better sense of where we're at in the development of these said practices.

And then, finally, this best practice, identifying these and starting to define these really sets the stage for building the research evidence behind these practices. Now, we have some preliminary evidence for these practices work, as I've shown, you but there is a need for much more research using the rubric in order to build up evidence base for these practices. And one of the things that we can do--there's so many research questions that can be posed with this rubric, and sort of the ones that I think are really important is, you know, are there certain practices that is more developed than other practices and do we see better outcomes? So what if you're emerging in five of the 11 practices but you have six of them that are highly developed, is that enough, right? So you can start to think about the cumulative impact of the practices, as well as the differential impact of the different independent practices on youth and family outcomes. So with that, I'm going to now hand it over to Magistrate White so he can then describe his work in Mahoning County and his real process actually in implementing some of these practices that I just discussed as well.

RICHARD N. WHITE: Thank you, Dr. Dierkhising, very much. Several years ago, we began hearing cases involving a 15-year-old girl. She had been severely neglected as a young child. Developmental health issues. There were drug issues. She had pending delinquency charges involving domestic violence before she was put in a kinship placement. And while she was in that kinship placement, she was charged with setting her family's home on fire. Now, when the child welfare worker, mental health provider, court staff, and others became aware of this fact, not one of them said, "Gee, I wonder what caused this. Do you think it was that neglect when she was young? Or what about these mental health and drug issues that she's been suffering from?" You know, like don't forget about the delinquency that was pending, no. Every one of you listening knows that this is one child, one family with many complex issues, and it requires a multidisciplinary team to formulate a comprehensive plan to properly serve this dual system youth.

Before 2012, my jurisdiction did not, and many jurisdictions still do not, engage in this comprehensive planning. On Monday, let's say the abuse case is being heard while the child is in the role of the victim. Present is a child welfare worker, maybe a mental health provider, other service providers. Then comes Thursday, and the delinquency case is going to be heard. Now, the child is in the role of the perpetrator. Who's at this hearing? Probation officer maybe. A prosecutor. Police officer. There may be even a different judge or magistrate hearing each of these cases. Now, I'm not saying that there is no awareness of the other case, but there is no collaborative team concept. Instead there's an attitude of "we are here for this, and there are people over there to handle that."

Then 2012, my postindustrial Midwestern town located halfway between Cleveland and Pittsburgh, began to implement the Crossover Youth Practice Model. The Crossover Youth Practice Model was developed by Georgetown University and Casey Family Programs to comprehensively address all issues relating to youth who've crossed over from being an abused and neglected child to being a delinquent child, or vice versa. They are there for dual system youth. The Crossover Youth Practice Model teaches a team approach. Cooperation, collaboration, information sharing. I'm going to review with you very briefly a few of our successes and, yes, a few of our failures in trying to implement this model to better serve this dual system youth. And I'm going to focus on, as Carly said, three of the best practices from the rubric that has just been discussed.

Well, to say that things got off to a rough start in our jurisdiction would be an absolute understatement. Judge Theresa Dellick, a visionary judge who was a juvenile judge then and remains a juvenile judge to this day, called a meeting of supervisors of which I was one of the court and said, "We have the opportunity to train for 18 months with Shay Bilchik and Megan Stewart of Georgetown University. And they're going to help us implement this Crossover Youth Practice Model, so that we can better serve this population of dual status youth." Well, the anonymous response, including mine, was, "We don't need it. We don't want it. We're already doing it." Well, Judge Dellick said, "Let's talk about this just a little more, and let's vote on it. Let's just see where this falls after we discuss it." Well, the final tally of the vote was the judge's one vote in favor of the training and the supervisors' eight votes against the training. Fortunately, as I said, we had the benefit of visionary judicial leadership by Judge Dellick, so you all know what happened because I'm here talking to you today. Now, that's what I call judicial leadership. I'm my opinion, judicial leadership is the single most important component in trying to implement the best practices for cross systems work.

Now, in 2012, when we began our training, we didn't have the benefit of the rubric that has been discussed today, and it had not yet been created. As you know from the discussion, that it's relatively new. The value of this rubric is to objectively show a jurisdiction exactly where they are and exactly where they need to be. So at our initial meeting, when we were discussing interagency collaboration, although we didn't call it that because again the rubric wasn't in place. We were, I guess, discussing how are we getting along with child welfare workers and are we communicating information properly. The discussion before the vote went something like this, "Well, you know, probation officer does pick up the phone and called the child welfare worker, and lets him or her know kind of what's going on, and there are some emails going back and forth on the cases, so, you know, I guess we--you know, we're already doing it." However, if we had had the rubric at that time as Dr. Dierkhising just said, we would have been able to view it this way. We'd look at the rubric. We would look at the interagency collaboration and the highly-developed practice, and the discussion would have been more like this. "Do you have cross-system teams established?" "Hmm, no, we don't." "Are they regularly meeting?" "Well, they're not meeting because the teams don't exist." "And are your key stakeholders consistently engaged in ongoing review of the work?" "No."

Well, we obviously were absolutely not already doing it. Our interagency collaboration at that time between juvenile court and child welfare was horrible. This rubric really only has as the lowest category ‘the practice not in place for interagency collaboration.’ It says cross-system teams committees have not been established. Key stakeholders have not been engaged. That really doesn't begin to discuss where we were located. If I were to put us on the rubric, I would put us somewhere like this: "Key stakeholders and workers are engaged in open hostility and finger-pointing." So you can see we obviously have work to do.

Now, the good news in all of that was, I found through working with this through the years, that that really wasn't unusual. As a matter of fact, the Supreme Court of Ohio, 50 years ago, stated, "The juvenile court is an uneasy partnership of law and social work." I mean, think about that. Uneasy partnership of law and social work. And that's why it's so important to have this rubric and be able to help jurisdictions to cooperate, collaborate, so that the partnership isn't so uneasy anymore.

Well, after that vote and the decision that we are going forward with the training, Judge Dellick next met with the Director of Child Welfare and persuaded him that we would then begin the crossover youth training. Well, the agreement was made, with the two leaders, the key leaders, of course. You needed the juvenile court judge or judges. You need the director of the child welfare agency of your county. With these two leaders now committed, we next invited all community partners to participate: educators, councilors, guardian ad litems, and family members. Our training taught us that the family member was--were absolutely a key component of this team. You wanted them to understand that we were not there to talk at them and tell them what they were going to do. We were there to work with them in formulating the best plan for their family.

After that, then we formed committees, and we had various committees designed for whatever purpose were needed. There were ad hoc committees that came into existence. I'll discuss some of them as we go along. Got a certain issue done, and that was it. We regularly held these meetings. We created a manual, which is an extremely important part of this. The manual laid out all procedures of how we would handle cases of dual system youth, starting with how do you identify them? Once they're identified, who takes the lead role? What are all the procedures to be followed in handling the case from beginning, identification until closing? This manual, to this day, is used as a reference to go back and see are we following the practices and procedures that we set up. It's used in training, and, as you heard both doctors talk about, ongoing training are extremely important so that you keep everybody not only reminded about how this process works but also obviously the turnover of new workers being constantly trained in it. And this manual is also used from the bench to discuss in court if there is an issue, "Are complying with what we have said we are going to do procedurally?"

All of this was then formalized by the signing of a memorandum of understanding signed by the judge and the director of the child welfare. And this was really, I think, a very important piece because this memorandum of understanding says, "We're going to cooperate. We're going to collaborate. And, you know what, we really mean it." We have this document here that we put--this memorandum of understanding went into our manual at the front of the manual, and it basically says, "This is a serious protocol that we are going follow." And we have been following it since 2012.

Well, after all of that work, we were then, truly, about at the six, eight-month mark of our training, really struggling, and I just felt that it really wasn't beginning to feel like the way we were doing business. It just really wasn't integrated. So when Shay Bilchik came for one of our trainings, I asked him--and it was one of the most important meetings, in my opinion, that I had with him. And then I said, "We're just not gaining traction." And he said to me, "You lead this practice from the bench. It is judicial-driven." And that really changed everything as to how we proceeded. When you think about it, it's one thing to hold seminars and it's another to hold meetings, but when you're in court, it really is a much more formal proceeding where you can really address: "Are we doing what we said we're going to do?" If you look at the judicial leadership point, it's exactly what Shay Bilchik said all those years ago, and it says the active judicial support, if you're in a highly-developed practice, it convenes and leads cross-system meetings, yes, we do, and it drives the work and provides accountability. It really is a very necessary part to not only get this model up and running but to have it continue to run.

Well, now that we had agreed to coordinate our handling of these dual system youth, we have to be able to identify dual system youth. How do you identify? And there was a lot of talk about this earlier. Somewhat of the theoretical were looking at the data level. Let me tell you how it works in the trenches. How do you identify a child who has crossed over from being an abused child to also being a delinquent child? If you can't identify the dual system youth, you can't treat the dual system youth.

For us, as it was already discussed, technology was a huge roadblock in being able to share the information to determine what child was a cross-system child. The juvenile justice computer was from Mars. The child welfare computer was from Venus. They couldn't talk to each other in any way. There was no way that you push the button or ran a program, and all of a sudden you had every crossover youth, in the county, on a sheet. But since we were now cooperating, since we were now collaborating and holding meetings, we were able to develop a very simple low-tech solution. Not saying it's the best but it's the best we had, and we do it to this day. And that meeting went something like this as we sat down: "Do both intake departments exist in each of these two entities?" "Yes. Yeah. Yes." "Juvenile court has an intake department?" "Yes. Juvenile services, child welfare has an intake department." "Well, do both of these intake departments log their new cases into their computers?" "Yes, we do. We both log them in." "Well, do you both have email?" "Yes, we do." And so, to this day, once a week, each intake department emails the other intake department a list of all new cases established that week. The receiving department then determines if it already has an open case on any of these youth. If it does, then the child has crossed over and now has a case opened in both systems and is a dual system youth.

Once identified, the next step is to get everyone on board to sit around and have a meeting and discuss all aspects of this case around the table. I'm talking child welfare, drug and alcohol counselors, probation officers, guardian ad litems, attorneys. What's the barrier there? Well, as Dr. Dierkhising just told you, the huge barrier there is confidentiality laws. It is very, very difficult to share this information, and the solution isn't easy. I think it's a multifaceted solution that you have to work on in this situation. You first have to form a committee, as we did, on this subject alone. You have to research the laws. The HIPAA laws, as all of you know, are extremely complicated and detailed. Mental health records have their own set of very specific rules and laws about sharing information. You then have to look at the federal and the state laws of your state, and you have to come into compliance with them. So we did that. Now, I have attended seminars that really are two, three hours on this subject alone of information sharing so we're just scratching the surface, but suffice to say, this is the process that you have to go through so that you are complying with the laws but still sharing as much information as you properly can to aid in making a decision of what is in the best interest of this family.

Another thing that we did, we prepared an informed consent to release and exchange information. Remember, I said earlier, the family involvement is a key to this. The family member or members are on this group and meet as part of this group in planning these things. So with the parent or guardian on board, and, hopefully, you've established a rapport with them that they understand that you truly are trying to help, it has been our experience that they are willing to sign this consent to release form. And this is not some blanket exchange of information release. It is a very specific document which has lots of specificity that you can put into what documents for what period of time for what purpose. Again, this is a very complex issue, this information sharing, but I can tell you that taking the steps that I very, very quickly went over, we have never, in all the years we've been doing it since 2012, ever had an issue of any significance relating to the sharing of this information.

Finally, I want to show you one of our less than stellar moments, although it turned out well, in the category of information sharing. I'll call it the "be careful what you wish for" category. Probably the best model for handling in court these dual system youth would be the one family, one judge model, and that would mean that the same judge is hearing the abuse, neglect case and is also hearing the delinquency case. Now, for the rest of you, this model allowed for variations on that because every jurisdiction, including ours, simply was not set up for that type of approach. We were set up more that one magistrate, myself, hears the delinquency cases, and I, therefore, hear all of the delinquency cases that we have involving the dual system youth. And we have another magistrate that hears the abuse, neglect, and dependency case. Well, about a year or so into our training and implementation, I received an email from the Director of Child Welfare, and it went something like this: "We have been working together very well for over a year and we have developed trust and collaboration and a team spirit, and therefore I feel comfortable in sending you this series of emails from my staff that contain the unvarnished truth." Well, I think each one of you knows that whatever follows after that statement, it's not going to be good. And it wasn't.

It was series of emails that started at child welfare, the caseworker level, went to the supervisory level, ended up on the desk of the director who then sent them to me. And these emails basically said: "The magistrates are making us do all these training, and we're supposed to be sharing information and cooperating with one another. And when I go to hear--to delinquency hearing with that magistrate," that would be me, "He is clueless on what is going on in the abuse, neglect, and dependency case. How can this be?" And there were a lot of emails, but believe me, that's what they said.

Well, I'll tell you. That was an absolute turning point in our development, and I thought it was one of the best days that we've ever had since trying to implement because it showed me and our judge and everyone that we had developed a team spirit and we were working together and everybody was not only comfortable to write these and complain internally but the director was comfortable enough to send it over and say, "Hey, take a look at what they think." And they were right. From that day forward, we now meet. I meet with the magistrate who does the abuse, neglect, dependency case once a week. All files are pulled for every case that is a dual system case, crossover youth case that's going to be heard the following week, and we each review the other file so that we are totally up to date and up to the minute on motions filed, procedures, what the hearing is going to be for. And that was a very, very valuable thing. It only came about after the trust developed from interagency cooperation and collaboration and this team building spirit that understanding it's not us versus them.

I'm going to just wrap up here with leaving a very almost hypothetical example of how would we have handled this arson case, a hypothetical arson case, before the crossover youth training and how do we handle it after. Before, if we had an abused, neglected child who had delinquency charges pending and had an arson involving a fire in the home, I would assume that from the child welfare, it would have gone something like this, "That child clearly is a risk to the safety of the community, and that child should be in detention right now. It's a delinquency case. Let the police and court handle it." And the child welfare worker can put that file on their bottom drawer so that it doesn't gather dust sitting on top of the desk. What would detention court staff be saying? "This is a neglected child with severe mental health and drug issues who requires residential placement for treatment. Detention is not the answer. Get this child out of here and into treatment." So somewhat of an exaggeration? In some jurisdictions, I don't think so.

How would that be handled now hopefully, that same case? After the arson, fire, the child would probably be placed in detention because we have the safety of the child and the safety of the community to deal with. However, if the child would have been placed in detention, while that was happening, if it happened, team meetings would be taking place right away. Child welfare, probation officer, mental health, drug, family, GALs. And we'd be putting together hopefully a plan for placement, treatment, resolution of the charges, obviously, the delinquency piece, and, ultimately, plans, when and if appropriate, for a return to the community. That's the difference that this training makes. Listen, this isn't a fairy tale. This is the real world. We know the outcome of these type of cases, these are the very difficult and complex cases as you've heard. These results in these complex cases are not always what we want them to be. But I do know that by using this rubric to implement these best practices, we are trying our very best to share and utilize all available information and resources to aid in making a team decision to help these dual system youth. Thank you. Dr. Herz.

DENISE C. HERZ: Okay. Okay. [MISSING AUDIO] Well, still had operated historically and how that transformation can happen. And some of the major challenges and bumps in the road that can come but also the sheer success and how that success impacts the short-term and long-term lives of youth and families. So thank you again for giving us illustrations of how system change works. It is not an easy or a quick process but it is certainly a rewarding one. And that brings us to our last slide to really bring all of this together into why does it matter.

We hope today that we have shown you both quantitative data, qualitative data, as well examples from the field that really convince everyone that with deeper and more precise knowledge of dual system use, the pathways, best practices, we can begin, collectively, to reframe the narrative around those youth and their families, and fundamentally change the way child welfare and juvenile justice systems operate. You know, we did the study before the significant attention to social justice has impacted our society in the last few months and brought attention to issues that have been, in on our field, recognized but I think has brought poignant attention to the way systems operate and the way that they contribute to inequity and social injustice. And I think that the study that our collaborative team produced really does have implications for how we can begin to truly have discussions around social justice and how it intersects with child welfare, juvenile justice system and the youth and families that experience those systems. We have to reframe the way systems understand children, youth, and family. We have to integrate prevention into communities and help communities build the capacity. And if they have the capacity, continue to support their capacity, to do prevention of maltreatment and delinquency in the community and keep children, youth, and families out of systems. But when systems are necessary, how do we partner with communities and build trauma-informed practices, multidisciplinary and collaborative assessment and case planning, holistic and appropriate programming, and all of those things very much integrated and partnered with the community, with the family voice, with the youth and child voice. Because, ultimately, what we do know from this study, as well as others, is that safe, stable relationships and environments are what help our young people succeed, to become healthy, productive citizens that know how to produce and contribute and sustain their own wellness. Ultimately, we believe that conversation and reframing of it, not only improves system involvement for those who touch both systems, but those practice changes improve practice for all children, youth, and families who experience system. With that, I'm going to hand it back over to Mary Jo who is going to do some concluding and questions and answers.

MARY JO GIOVACCHINI: Thank you very much. At this time, I am going to turn it back over to Barbara Kelley who will start the Q&A.

BARBARA TATEM KELLEY: Hi. Well, we have a lot of questions. Well, starting with the first question. "What precipitated your choice of Cuyahoga County in your study?" I'll refer that to Denise.

DENISE C. HERZ: Okay. And that is a simple answer. When we wrote the proposal for the OJJDP RFP we had the benefit of many sites and research teams having already done the hard work of developing agreements with the government agencies to access and use administrative data to look at various social problems and social issues. And Cuyahoga County is a partner with Case Western and our collaborators there. So Case Western was identified as a research group to set up for poverty, urban poverty was identified as a viable partner to do this work, and they had the agreement with Cuyahoga County. So it was pretty much a result of their ability to do this work.

BARBARA TATEM KELLEY: Thank you very much. There was a question about, "Could you also calculate the proportion of youth who have touched the child welfare system, who have had a JJ system contact, rather than looking at the percentage of youth to already had a child welfare involvement?" So can you flip from where you start looking.

DENISE C. HERZ: Yeah. Yes. Yes. That would be a more of a perspective cohort study. And there has been studies like that done. The percentage overall is anywhere from 10% to about 29% of young people between the age of 10 and 18 in the child welfare system who subsequently become involved in the juvenile justice system. So the percentage is much smaller. We did not went out of the scope of the OJJDP study but we will under CVN's child data network at USC, we will be pursuing a perspective study to look at that issue and also begin to look at the pathway, because none of the previous research looked at the way in which those youth might touch both systems.

BARBARA TATEM KELLEY: Thank you. "Do you know how many states have child protection and juvenile justice under the same umbrella of child welfare?"

DENISE C. HERZ: According to--when we did the study, we looked at a piece, and you could find that piece in the study so you can look at the link, but it was by JJGPS. And they did, basically, a landscape of analysis of that very question, and according to that document, there are seven single agency integrations of child welfare and juvenile justice out of the fifty states.

BARBARA TATEM KELLEY: "Do any of these jurisdictions do a dual status screening tool upon entry into the system that gets submitted to the court?"

DENISE C. HERZ: I don't know. If you're speaking to the three sites in OJJDP, I just know the answer to that question. In terms of Los Angeles County, it's a little bit more complicated. The basic answer is no. However, if a youth has an open child welfare case and they are referred or arrested to the juvenile justice system, the state law has a process whereby they are identified as potentially a dual status or dual system youth, and there is a process that they follow. I give that as an example. I think the answer to your question is it varies by jurisdiction, and I just don't know in the three sites of the OJJDP whether they do or not. I know Cook County had begun very close to when the study time period was done. They were involved in some work around cross-system development, so they may very well have developed a tool.

BARBARA TATEM KELLEY: Thank you. I know that we don't have any more time left for questions. And so I want to thank each of my presenters, as well as thank Mary Jo Giovacchini, Michelle Gorham, and Daryl Fox for providing support for this. And I will read one last comment that showed up in the Q&A which I think sort of summarizes how I feel about this whole presentation. "I am in the trenches every day and your research is spot on." I thank you for that comment. Thank you for your attendance. Thank you for making me feel wonderful about so many years of research in this area. And please have a lovely evening. Thank you so much for joining us.

DENISE C. HERZ: Thank you everyone. Thank you, Barbara.