Development of a Model for Predicting Running Away ...

Residential Treatment for Children & Youth, 27:264-276, 2010 Copyright © Taylor & Francis Group, LLC

Routledge

ISSN: 0886-571X print/1541-0358 online

Taylor & Francis Group

Elopement, or occurrence amc

DOI: 10.1080/0886571X.2010.520634

people are rnor

Development of a Model for Predicting Running Away from Residential Treatment Among Children and Adolescents ANNA McINTOSH, BA and JOHN S. LYONS, PhD School of Psychology, University of Ottawa, Ottawa, Ontario, Canada

DANA A. WEINER, PhD and NEIL JORDAN, PhD Department of Psychiatry, Northwestern University, Evanston, Illinois, USA

The effectiveness of mental health care services is severely limited when young people run away from residential treatment. This study describes the development of a decision support model for predicting discharge due to running away on the basis of individual characteristics. Subjects include 667 wards of a large Midwestern state between the ages of 7 and 20 who were placed in residential treatment and discharged during 2007. A model combining eight predictors—namely School Attendance, History of Running' Runaway Ideation, and linear as well as quadratic terms for Age, Substance Abuse and Delinquency—demonstrated good internal validity and rnoderate predictive powerfor discharge due to elopement. Results of this study can support mental health care workers in identng potential recipients of interventions that reduce the likelihood of premature exits from residential care among young people. KEYWORDS runaway, residential treatment, substance use, predictive modeling, CANS

The authors thank Frank E. Harrell, Jr. for his correspondence regarding nonsignificant predictors and the post-hoc reduction of prespecified models. The authors also thank Gary McClelland, PhD, for his help with data preparation and cleaning. Address correspondence to John S. Lyons, PhD, School of Psychology, University of Ottawa, 145 Jean-Jacques-Lussier (352), Ottawa, ON KIN 6N5, Canada. E-mail• jlyons@ cheo.on.ca 264

institutions (Ber has negative rei remaining resid Greenberg, Mai Wade & Bienal selves to a host and even death develop a pre residential treats staff and promos Efforts to 4 elopement from ronmental facto soaT (Elistngat

Read, 1994). WI ous environment (Benalcazar, 198., of residential mil risk factors assoc dent of environs from residential dential care at 6 shown that indii ables and clinic by categorical v: independent role (2000) reported some residential account for the r all placements. Research on largely focused o tially distinguish however, to deve of such factors. F elopers have bee as history of sex Siegel & Callesei (Kashubeck et al (Eisengart et al., even have been

Runaway from Residential Treatment

265

Elopement, or running away, is a dangerous, yet alarmingly frequent occurrence among children and adolescents in residential treatment. Young people are more likely than older individuals to run away from psychiatric institutions (Berman & Goodrich, 1990), and it is clear that runaway behavior has negative repercussions for the youth involved as well as for staff and remaining residents in residential care (Abbey, Nicholas, & Bieber, 1997; Greenberg, Blank, & Argrett, 1968; Sledge, Benarroche, & Phillips, 1988; Wade & Biehal, 1998). Furthermore, youth who run away expose themselves to a host of additional adverse consequences including further abuse and even death (Tyler, Hoyt, Whitbeck, & Cause, 2001). This study aims to develop a predictive model of running away as reason for discharge from residential treatment that can facilitate decision-making by residential care staff and promote early interventions for young people at risk. Efforts to develop a deeper understanding of the determinants of elopement from residential treatment have generally focused on either environmental factors or individual characteristics of young people who run away (Eisengart, Martinovich, & Lyons, 2007; Kashubeck, Pottebaum, & Read, 1994). While research has demonstrated the importance of numerous environmental factors, including relationships between staff and wards (Benalcazar, 1982; Siegel & Callesen, 1993; Sledge et al., 1988) and stability of residential milieus (Sledge et al., 1988), other studies have established that risk factors associated with individuals also play an important role, independent of environmental conditions, in predicting the likelihood of elopement from residential treatment. Using a sample of 1,927 young people in residential care at 64 different agencies, Eisengart and colleagues (2007) have shown that individual characteristics—as measured by demographic variables and clinical assessments—and environmental factors—as measured by categorical variables for • program and RTC—all play a significant and independent role in predicting runaway behavior. Further, Biehal and Wade (2000) reported that while running away is a more common outcome in some residential placements than others, a small number of youth tend to account for the majority of runaway episodes (short- and long-term) across all placements. Research on individual characteristics predictive of running away has largely focused on analyzing separately various risk factors that might potentially distinguish elopers from non-elopers. Limited effort has been made, however, to develop predictive models of elopement based on combinations of such factors. Research findings regarding the characteristics of individual elopers have been complex and sometimes contradictory: some factors such as history of sexual abuse and of physical abuse (Kashubeck et al., 1994; Siegel & Callesen, 1993; Sunseri, 2003), presence of an affective disorder (Kashubeck et al., 1994; Sledge et al., 1988) and female (vs. male) gender (Eisengart et al., 2007; Siegel & Callesen, 1993; Wade & Biehal, 1998) have even have been cited as both positive and negative predictors of running

266

A. McIntosh et al.

away in different research contexts. Such discrepancies may be due to sample characteristics, variations in the definitions of running away employed in different studies, increased risk of Type 1 errors due to testing large numbers of predictors, the time and place in which research was conducted, and complex relationships between predictor variables and elopement. Review of Literature on Predictors of Running Away Despite the contradictions noted in the literature, several factors have surfaced with great consistency as predictive of running away. Age is regularly cited as a predictive demographic variable. Elopement is less frequent among younger children than among adolescents, but also decreases in frequency as adolescents get older (Sunseri, 2003; Wade & Biehal, 1998). In a study of 8,933 young people between the ages of 8 and 18 conducted by Sunseri (2003), frequency of elopement peaked at 38% at age 16 and then decreased to 28% among 18-year-olds. The factor perhaps cited most frequently and consistently in conjunction with predicting elopement from residential care is previous history of running away (see, e.g., Abbey et al., 1997; Berman & Goodrich, 1990; Biehal & Wade, 2000; Kashubeck et al., 1994; Sledge et al., 1988; Sunseri, 2003). History of running away from home has been considered an important predictor, in addition to history of running away from residential treatment (Wade & Biehal, 1998; Siegel & Callesen, 1993). Substance abuse has also commonly been associated with elopement (see, e.g., Kashubeck et al., 1994; Siegel & Callesen, 1993; Sunseri, 2003; Wade & Biehal, 1998). This factor is associated not only with short runaway episodes, but also with discharge due to running away. Eisengart and colleagues (2007) found that substance abuse was the only predictor among a range of clinical assessment items that significantly distinguished youth discharged due to running away from youth categorized into four other discharge types. School attendance issues and history of offenses have been cited together as important predictors (Siegel & Callesen, 1993; Wade & Biehal, 1998). Of a sample of 193 young elopers from substitute care studied by Wade and Biehal (1998), 40.4% either never attended or were excluded from school, as compared to 30% of all young people in residential care who were found to never have attended or to be excluded from school in a comparable study cited by the researchers, conducted by Sinclair and Gibbs (1998). While Wade and Biehal (1998) suggested that episodes of running away might cause increasing detachment from school and teachers, one might also expect that detachment from school would increase probabilities of lengthy or permanent stays away from residential care. Juvenile offending, including vandalism, theft, and property destruction, has been associated with running away (Sunseri, 2003; Siegel & Callesen,

1993 ; W elope ofi also hay Wade & promine indicate who did Fam ences or a positiA behavior for pred referral 5. ment epi In contra are most ple, whe frequent] them to related r with othi Witl (1984) p relations borderlir with pro ships wii away, w positive treatmen ing place Wade & Objectix This stuc western of the sti dential tr Child an 2004). of predic tant influ also aim(

Runaway from Residential Treatment

267

1993; Wade & Biehal, 1998; Abbey et al., 1997). While young people who elope often offend while out of residential care, those with prior convictions also have a far greater likelihood of running away (Sinclair & Gibbs, 1998; Wade & Biehal, 1998). Although history of offenses was mentioned rather prominently in the studies reviewed, Kashubek and colleagues (1994) did indicate that adolescents ages 14-17 who eloped were less likely than those who did not elope to have a history of legal offenses. Family has been shown to exercise important but rather complex influences on elopement from residential treatment. Sunseri (2003) has noted a positive relationship between stress in family functioning and runaway behavior, and included family functioning level in a four-variable model for predicting elopement, along with level of care, runaway history, and referral source. Siegel and Callesen (1993) calculated that 82% of 34 elopement episodes for 26 youth were preceded by escalations in family tensions. In contrast, Wade and Biehal (1998) have suggested that family influences are most important when they involve pull factors. This occurs, for example, when young people run away because they miss their family or, less frequently, because parents and other family members actively encourage them to run away. The researchers also suggested, however, that familyrelated reasons tended to influence runaway behavior through interactions with other factors and were rarely the primary instigators of elopement. With regards to interpersonal relationships, Goodrich and Fullerton (1984) proposed that youth who run away tend to have weak attachment relationships with staff and to be highly peer-dependent. Of a sample of 60 borderline patients in residential treatment, they calculated that adolescents with profiles of high peer-dependence and avoidance of positive relationships with adults had a 73% likelihood of being discharged due to running away, while those with profiles of high adult-dependence and avoidance of positive relationships with peers had only a 16% likelihood of terminating treatment by elopement. It has also been suggested that exposure to bullying places young people at risk for running away (Greenberg et al., 1968; Wade & Biehal, 1998). Objectives This study is based on a statewide system of residential care in a large didwestern state located within the United States of America. The main objective of the study was to develop a predictive model of running away from residential treatment based on data collected through the administration of the Child and Adolescent Needs and Strengths assessment tool (CANS; Lyons, 2004). We focused on developing a model on the basis of a small number of predictors consistently cited in the literature. In recognition of the important influences environmental factors may exert on youth who run away, we also aimed to analyze how well a model based on individual characteristics

268

A. McIntosh et al.

of elopers predicted running away when taking into account environmental effects related to the specific residential care agencies where youth were placed.

Procedi CANS a. resident assessed already thus firs assessmt admissic first asse Parc adolesce Consent of this st

METHOD Participants The sample consisted of 667 wards of the Illinois Department of Child and Family Services (IDCFS) who took part in residential treatment programs run by 44 different agencies. At the time of their first CANS assessment, children and adolescents ranged from 7 to 20 years old, with a median age of 16. Males constituted approximately 60% of the sample, while the remaining 40% of the wards assessed were female. Ethnic and racial composition was as follows: 58% African American. 27% Caucasian, 6% Hispanic, and 9% multiracial, other, or unknown.

Analytic

Measures The dependent variable in this study is discharge from residential treatment due to elopement, which was measured dichotomously. Predictor variables were drawn from the Child and Adolescent Needs and Strengths (CANS; Lyons, 2004). The CANS is an assessment tool used by mental health service providers to guide decisions related to level of care and treatment plans. The CANS comprises various scales including Trauma Experiences, Trauma Stress Symptoms, Strengths, Life Domain Functioning, Behavioral/Emotional Needs and Risk Behaviors. Items on the CANS are action-oriented: the score of each item is determined according to four clearly defined levels, and each level is associated with a different action indication. The levels and corresponding action indications for scales associated with needs are as follows: 0 = No evidence—no need for action; 1 = Watchful waiting/prevention; 2 = Action required—need is interfering with child's individual, family or community functioning in a notable way; and 3 = Immediate/intensive action—need is dangerous or disabling. Two audit studies have demonstrated the high field reliability of the CANS (Anderson, Lyons, Giles, Price, & Estes, 2002; Lyons, Rawal, Yeh, Leon, & Tracy, 2002). Service providers must be trained and certified with a minimum reliability of 0.70 in order to administer the CANS, and in most jurisdictions in the United States, average administrator reliability on test vignettes is well above 0.80.

=

Logistic I tial treatr the logist strated cc could be recomme Steyerber the numb terms for more that Boot estimates performai ple used in other s capitalize ables. In c were ca lc and colle: model de, models clt through rt and (3) fo the origin statistics o were cala tracting th

Runawayfrom Residential Treatment

269

Procedure CANS assessments were integrated into the routine procedures of IDCFS residential treatment facilities in 2005. While some of the youth were first assessed soon after admission to residential treatment, many others were already residents at the time of the introduction of the CANS and were thus first assessed at this point. All of the youth completed their first CANS assessments between August 2005 and January 2007. Median time from admission to first CANS assessment was 214 days, while median time from first assessment to discharge was 187 days. Parental consent for assessment was not required because children and adolescents in these residential treatment facilities were wards of the state. Consent from the IDCFS was acquired for use of their data for the purposes of this study.

Analytical and Statistical Methods Logistic regression was used to identify significant predictors of residential treatment discharge due to elopement. Factors included as predictors in the logistic regression model were selected on the grounds that they demonstrated consistent relationships with elopement in the literature, and that they could be measured by individual items on the CANS. In accordance with the recommendations of Harrell, Lee, Califf, Pryor, and Rosati (1984) (cited in Steyerberg et al., 2001, and Harrell, Lee, & Mark, 1996), we ensured that the number of predictors included in the model—with linear and quadratic terms for the same variable counting separately toward the total—was no more than the number of cases of runaways divided by 10. Bootstrapping was used to calculate bias-corrected model performance estimates and to evaluate the internal validity of the predictive model. Model performance estimates that describe how well a model performs in the sample used for its development over-represent the model's predictive ability in other samples, since models not only reflect stable relationships but also capitalize on shared error variance between predictor and outcome variables. In order to account for this overestimation, biased-corrected estimates were calculated according to the following procedure, outlined by Harrell and colleagues (1996). First, model statistics were generated: (1) for the model developed in and tested on the original sample ("apparent"); (2) for models developed in and tested on each of n bootstrap samples, created through resampling with replacement from the original sample ("training"); and (3) for models developed in each of the n bootstrap samples, tested on the original sample ("test"). Next, measures of the extent to which apparent statistics overestimate the predictive ability of a model (termed "optimism") were calculated by taking the mean of the test statistics from (3) and subtracting this value from the mean of the training statistics from (2). Then,

270

A. McIntosh et al.

bias-corrected values for statistics were calculated by subtracting measures of optimism from apparent statistics. Bootstrap replications were run using commands for Harrell's Design Package (2007) in R statistical software. All bootstraps were run with 500 replications. Bias-corrected statistics were generated for indicators of explained variation (Nagelkerke's R2) discrimination (Somer's D) and calibration (shrinkage) of the model. Finally, logistic regression analyses were used to test the comparative effects of individual characteristics, as measured by the predictive model, and the residential milieu, as measured by a categorical variable representing the residential treatment center (RTC) at which youth received treatment. Apparent Nagelkerke's R2 values were generated to compare explained variance, and statistics were not bias-corrected using bootstrapping in the context of these analyses.

RESULTS

TAB

and

Age 7 8 10 11 12 13 14 15 16 17 18 19 20 Totes n=

Choice of Predictors The sample included 84 youth who were discharged due to runaway. As such, we considered it appropriate to test a model with a maximum of 8 predictors (i.e., minimum 10 variables for each S to protect against taking advantage of chance in fitting the prediction parameters). Variables chosen based on the review of the literature were as follows: Substance Abuse, Delinquency, School Attendance, History of Running/Runaway Ideation and Age. Preliminary screening of the relationship between chosen predictors and the outcome variable indicated that quadratic terms for three of the variables—Age, Substance, 4nd Delinquency—would need to be included in the model. The quadratic trend in the relationship between Age and Runaway is evident from Table 1. Children and adolescents between the ages of 15 and 16 had the greatest odds of running away, while those younger than 13 and older than 19 had the lowest. For both Delinquency and Substance Abuse, quadratic trends were apparent such that the odds of running away peaked among those who scored a 2, on a scale that ranged from 0 to 3.

Model Properties The 8-variable model significantly predicted discharge due to runaway, x 2(8, N = 667) = 67.23, p < .001. Betas, odds and confidence intervals for the predictive model are reported in Table 2. Delinquency and Delinquency- 2 were nonsignificant predictors in this model, although a model including only these two variables as predictors was significant, x 2(2, N= 667) = 14.19, p = .001.

TABLE 2 Be

Predictor Substance Al Substance Al Delinquency Delinquency School Atten History of 111. Runaway I Age Age- 2 Constant *p < .05; **.p