The Journal of Forensic Psychiatry Vol 12 No 3 December 2001
592–609
Development and implementation of a functional skills measure for forensic psychiatric inpatients J AM E S V E S S
ABSTRACT Responsible clinical management of forensic psychiatric inpatients involves a variety of challenges. One such challenge is the lack of a generally accepted theoretical framework to guide the content of treatment interventions. Another is the limited utility of available assessment instruments for the purposes of treatment-planning and outcome evaluation with this population. This article describes the development, implementation and psychometric properties of an instrument for measuring the functional skills of patients committed to a maximum-security forensic state hospital. Also presented are the role of this instrument in de ning and standardizing the clinical approach of the facility’s interdisciplinary treatment teams, its use in a computer-assisted treatment-planning process, and use of the resulting data in outcome evaluation. Keywords: forensic, inpatient, skill training, measure, outcome, treatment
Placing patients in an enhanced security treatment facility is expensive. It currently costs approximately $105,000 per year to treat a patient in California’s maximum-security state hospital, compared with an average of $26,000 per year to con ne an inmate in one of the state’s prisons. Recidivism rates for The Journal of Forensic Psychiatry ISSN 0958-5184 print/ISSN 1469-9478 online © 2001 Taylor & Francis Ltd http://www.tandf.co.uk/journals DOI: 10.1080/0958518011009200 1
ATAS C A DE R O SK IL LS PR O F IL E
mentally ill offenders have been reported from 27% to over 70%, depending on the de nition of recidivism, the length of follow-up, and the nature of community care and supervision provided (Weideranders, 1992; Harris and Koepsell, 1996; Weideranders, Bromley and Choate, 1997; Harris and Koepsell, 1998; Ventura, Cassel, Jacoby and Huang, 1998). These gures provide compelling reasons to provide clinical services explicitly designed to minimize length of stay while maximizing the likelihood of successful community adaptation. A growing body of literature supports an approach to treatment of the chronically and severely mentally ill which emphasizes skills training (Dilk and Bond, 1996) as part of what has variously been called biopsychosocial rehabilitation (Katz, 1998), psychiatric rehabilitation (Anthony, 1980) or biobehavioral treatment and rehabilitation (Kopelowicz and Liberman, 1995). The other common elements of such an approach include an emphasis on using medication to reduce symptoms and providing environmental support. In adopting this approach to treatment, effective measures were needed in each area of emphasis, including a measure of functional skills. Others have noted the limited utility of available assessment instruments for the purposes of treatment planning and outcome evaluation with a severely mentally ill population (Wallace, Lecomte, Wilde and Liberman, in press). A review of over 70 instruments cited in the literature made clear the limitations of these tools for treatment-planning and outcome evaluation with a forensic inpatient population (Af eck and McGuire, 1984; Anthony and Farkas, 1982; Baker and Hall, 1988; Green and Gracely, 1987; Ellsworth, 1971; Evenson and Boyd, 1993; Hall, 1980; Honigfeld and Klett, 1965; Wallace, 1986; Weiner, 1993). DEFINING THE TARGET POPULATION Atascadero State Hospital is one of four California state hospitals. It is the maximum-security forensic facility which houses approximately a thousand judicially committed male patients from the state’s criminal justice system. The average age is 41.5, with a range of 19 to 85 years. In ethnic terms, 48% are White, 29% African-American, and 18% Hispanic. Patient records indicate that 68% have never married, 21% are divorced or separated and 9% married. The average reported education is eleventh grade. The distribution of primary diagnoses consists of 61% psychotic disorders (mostly schizophrenia and schizoaffective) and 26% sexual disorders (mostly paraphilias). The remaining 13% are mainly a mixture of mood and substance abuse disorders, often with psychotic features. There are three primary commitment types. Of the patients 41% are mentally disordered offenders who have served their sentence in prison and
593
594
J O U RN A L O F F O RE N SI C P SY C H IAT RY
Vol. 12 No. 3
subsequently been paroled to Atascadero State Hospital for mental health treatment prior to release to the community. Another 17% are inmates from the Department of Corrections who require psychiatric treatment unavailable in the state’s prisons. Following treatment, these patients are typically returned to the Department of Corrections to serve the remainder of their sentences. Finally, a statute enacted in 1996 has resulted in the civil commitment of a sex offender population mandated to receive treatment following their prison sentence and prior to release to the community. This group currently represents 30% of patients in residence. The remaining 12% consists of other commitment types, including individuals found incompetent to stand trial and those found not guilty by reason of insanity. Patients are treated on one of the 28 residential treatment units, each accommodating between 30 and 50 patients. Units are organized into six different treatment programs, which are primarily specific to one commitment type. This diversity of commitment types introduces differences in treatment goals. Patients being prepared for return to the Department of Corrections present different discharge criteria from those being prepared for release to the community under the supervision of the state’s conditional release program or parole. These disparate discharge environments impose different demands in terms of functional skill level and psychiatric stability. The Atascadero Skills Pro le (ASP) was developed to accommodate this range of treatment goals, allowing it to be used in a standardized fashion across the facility’s various patient commitment types (see the Appendix for the complete ASP). The purposes for which the ASP was designed include: organizing the initial comprehensive assessment of the patient’s level of functioning in areas relevant to post-discharge success; treatment-planning, whereby speci c skill de cits are prioritized for the assignment of treatment activities on the basis of the expected discharge setting; and outcome evaluation, both in terms of response to the course of care during hospitalization and of the association of skill acquisition with post-discharge outcomes. DEVELOPMENT OF THE INSTRUMENT Process of item generation: de ning content The rst step in item generation was to solicit input from a variety of clinical staff regarding those areas of functioning most relevant to the dispositional needs of our patient population. The resulting material was organized by a small workgroup into the initial outline of content areas, referred to as skill domains. This yielded 10 domains on which there was general consensus
ATAS C A DE R O SK IL LS PR O F IL E
regarding the relevance to the discharge goals of our patient population. The 10 domains were: 1 2 3 4 5 6 7 8 9 10
self-management of psychiatric symptoms through behavior; self-management of psychiatric symptoms through medication; substance abuse prevention skills; self-management of assaultive behavior; control of self-injurious or suicidal behavior; self-care; independent living skills; control of deviant sexual impulses and behaviors; interpersonal skills; leisure and recreation skills.
Individual items that re ect speci c skills within each content domain were then generated. This process involved breaking domains down into discrete sub-components based on a cognitive-behavioral, relapse prevention model. The next step was to convene an interdisciplinary group of staff with representatives from all clinical disciplines, including psychiatry, psychology, social work, rehabilitation therapy, nursing, psychiatric technician, dietary, and special education. This group met repeatedly over several weeks to perform two tasks. First, consensus was reached on the de nitions for each item in each domain of the instrument. Second, an effort was made to standardize the scaling of items within each domain, as well as to ensure that the behavioral referents for each score on each item were clinically meaningful in the context of treatment for the target patient population. The result was a scoring manual that de ned the content and language of the scoring criteria for each item. Following this review, a draft of the instrument and scoring manual was used to train the staff on two pilot units, each on different treatment programs. This provided direct feedback from users regarding the clarity and utility of the instrument. Based on this feedback, minor modi cations were made to the language of several items and to the descriptions used in some of the scoring criteria. The pilot units then used the instrument to score patients at the time of their routine treatment-planning reviews at 7 days following admission to the unit and at 90-day intervals. This pilot phase lasted for 3 months, with ongoing consultation with the participating interdisciplinary teams for problem-solving and to observe the team process involved in administering the instrument. At the end of this phase, the pilot units were formally debriefed about their experience with the instrument, in terms both of the content and scaling of the instrument and of the impact of its use on the team process. Based on this debrie ng, a nal draft of the instrument was completed, and a basic approach to using the instrument in the interdisciplinary team was
595
596
J O U RN A L O F F O RE N SI C P SY C H IAT RY
Vol. 12 No. 3
de ned. This became the basis for training the remaining ve units on the two pilot treatment programs. These units used the instrument to rate patients at each regularly scheduled treatment-planning review, upon admission to the unit and at 90-day intervals. These seven interdisciplinary teams administered the instrument for approximately 1 year while software was developed to incorporate the ASP into the routine treatment-planning process. During this time, the data were reviewed for each of the treatment units in order to identify anomalous patterns of scoring. As such patterns were discovered, the treatment teams were consulted in order to determine the decision-making process involved in rendering these scores. From this interaction, a number of common mistakes or misapplications of the scoring criteria/instructions were identi ed. This resulted in a compilation of common dif culties encountered in the standardized use of the instrument. This information was applied to the training of the remaining 21 units in the use of the instrument, once a supporting computer program for treatment-planning was developed. Training interdisciplinary treatment teams to use the ASP Teams were provided with an initial 2-hour training session in the use of the ASP. The rst hour was spent in a presentation of the terminology, theoretical framework, development process, domain content and scoring criteria of the instrument. During this time the scoring manual was reviewed to familiarize staff with the behavioral anchors used for the scoring of each item. During the second hour, the chart of a current patient on the unit being trained was reviewed in detail and, along with the staff’s clinical knowledge of the patient, used to score the ASP for that patient. Individual staff were requested to record a score for each item, and then a comparison of scores was used to identify areas of disagreement. These items were discussed to determine what factors had led to the differences in scoring, and an agreement was reached through the trainer regarding the most appropriate scoring. Following this training session, treatment teams were observed by the trainers in the scoring of patients during a treatment-planning meeting. Feedback was given when scoring appeared to deviate from the criteria provided by the scoring manual. This monitoring and feedback process continued until teams consistently demonstrated adherence to recommended procedures. Rating of patients using the ASP: standardizing clinical procedures The ASP is designed to be scored for a patient based on a consensus of the interdisciplinary team members. Sources of data include behavioral observation, patient interview, evaluation of the response to speci c treatment
ATAS C A DE R O SK IL LS PR O F IL E
activities, and in some cases results from speci c standardized assessment instruments. Teams have developed two basic approaches to the scoring process. One approach is for a single individual, usually the clinician designated as the case manager for a given patient, to gather the information necessary to render a preliminary scoring for all items. The second approach that has evolved is a speci c division of responsibility for the various skill domains according to clinical discipline. In this model, the psychiatrist, psychologist, social worker, rehabilitation therapist and nursing staff are responsible for speci c domains for each patient reviewed by the team. In both approaches, a key to the nal scoring is team review and consensus. This process ensures that the observational and assessment information available from a variety of staff, observing the patient in a variety of situations, is brought to bear on the scoring of each item. The experience of all treatment teams to date is that scoring consensus is usually obtained ef ciently. All ASP data analyzed for this study were based on consensus ratings. One common concern about the implementation of the ASP as a part of the treatment-planning process was the time it would take to score and review the instrument during the interdisciplinary team meeting. This concern was addressed by emphasizing during the training phase that all necessary assessment procedures related to ASP scoring must be completed prior to the team meeting. During the meeting, time was spent only on a review of the scores already suggested and a resolution of scoring con icts arising out of differing observational data. It was demonstrated during the pilot phase that teams who completed the necessary assessments before the team meeting could consistently review and agree to ASP scoring in under 10 minutes. Besides emphasizing the value of thorough patient evaluation prior to the treatment-planning review, the use of the ASP throughout the facility has allowed for a degree of standardization that had not previously been possible. In fact, one of the major challenges to the implementation of the ASP was the unprecedented degree of standardization necessary in team process, assessment procedures, and the focus of treatment activities. Team process and assignment of treatment were previously determined to a signi cant degree by the unique mix of clinical staff comprising the team, including the orientation, experience and personalities of the individual team members. Some teams focused primarily on symptoms and their remission, while others were more concerned with compliance with unit routine. Some teams spent the time of the team meetings in an ef cient process of reviewing treatment progress and assigning priorities according to the demands of the discharge environment, while others used a signi cant amount of time interviewing the patient and conducting essentially therapeutic exchanges. In order to ensure the necessary degree of standardization, signi cant attention was paid to the training of the interdisciplinary treatment teams and
597
598
J O U RN A L O F F O RE N SI C P SY C H IAT RY
Vol. 12 No. 3
their subsequent compliance with designated procedures. Administration and program management made clear their expectations about the adoption of this new instrument. Program managers often attended the training of their interdisciplinary teams, and continued to monitor their performance following the initial training. As with any large and diverse facility, some teams adapted smoothly to the new system, while others struggled. Criteria for outcome measures Criteria for the development and application of measurement instruments used in treatment-planning and outcome evaluation have been clearly articulated (Newman and Ciarlo, 1994). The criteria include: 1 2 3 4 5 6 7 8 9 10 11
relevance to target group; simple teachable methods; use of measures with objective referents; use of multiple respondents; more process-identifying outcome measures; psychometric strengths; low measure costs relative to its uses; understanding by non-professional audiences; easy feedback and uncomplicated interpretation; usefulness in clinical services; compatibility with clinical theories and practices.
The ASP was developed so as to meet each of these criteria. Specically, it was designed based on the needs of a speci c patient population. Manuals with behavioral referents and standardized training procedures were utilized. Assessment information is obtained from a variety of interdisciplinary team members. Measurement is repeated throughout the course of inpatient treatment, providing behavioral indicators of patient progress in response to treatment. While requiring signi cant staff resources, the ASP supports treatment-planning, quality assurance, program evaluation and utilization review, thus providing many uses relative to its cost. Its content areas are straightforward and easily understood, allowing easy interpretation for patients and for professional and non-professional audiences. It is the basis of treatment-planning and outcome evaluation. It directly re ects the theoretical approach to treatment adopted by the facility, that of biopsychosocial rehabilitation. PSYCHOMETRIC EVALUATION Minimal requirements for the adequate psychometric testing of an instrument’s reliability and validity have been suggested in the professional
ATAS C A DE R O SK IL LS PR O F IL E
literature (Norbeck, 1985). The recommended guidelines include content validity, test-retest reliability, internal consistency reliability, and criterionrelated or construct validity. For measures of outcome, there should also be sensitivity to treatment-related change (Newman and Ciarlo, 1994). Reliability Internal consistency and test-retest reliability statistics can be calculated from available ASP data. Cronbach’s alpha, a measure of internal consistency, is presented for various domains in Table 1. These values indicate adequate internal consistency for each of the domains evaluated. The lowest value was for the ‘independent living’ domain. This result can be understood by recognizing that this domain consists of the most heterogeneous set of items on the instrument. Sample sizes vary by domain due to differences in the number of patients rated in each of the domains (i.e. some domains are not applicable for some patients, and therefore not scored for those cases). Test-retest reliability was evaluated by using two sets of scores assigned to the same patient within 45 days by the same ID teams. The resulting correlations are presented in Table 2. These correlations represent adequate stability for each domain evaluated. One limitation of the preceding correlations is the length of time between ratings. With increasing time between ratings, the effect of the stability of the measure becomes dif cult to distinguish from the stability of the construct being measured (Knapp, 1985). In the present case, the correlations may re ect instability (i.e. unreliability) in the instrument as well as true patient change, which would be change in the construct. A better measure of the stability of the measure would involve ratings taken closer together in time. Of the available data, however, almost all such cases are obtained when two different teams have rated the same patient, as when Table 1 Cronbach’s alpha values for each ASP domain Domain
Cronbach’s alpha
N
Behavior Medication Substance abuse Assault Suicide Self-care Independent living Sexual deviance Interpersonal Leisure/recreation
0.94 0.89 0.90 0.95 0.94 0.92 0.74 0.97 0.86 0.92
1181 1172 1305 1215 641 1442 596 210 1455 1455
599
600
J O U RN A L O F F O RE N SI C P SY C H IAT RY
Vol. 12 No. 3
Table 2 Test-retest reliability values for each ASP domain Domain
Correlation
N
Behavior Medication Substance abuse Assault Suicide Self-care Independent living Sexual deviance Interpersonal Leisure/recreation
0.68 0.80 0.82 0.65 0.59 0.76 0.55 0.89 0.69 0.79
86 82 103 91 44 115 120 47 120 119
a patient is transferred from one treatment unit to another. This would introduce an interrater reliability factor that may confound an examination of testretest reliability. Therefore these data were excluded from this analysis. Adequate interrater reliability data were not available. Because the instrument is scored based on a consensus of interdisciplinary team members, the gathering of such data would optimally involve two different teams of clinicians evaluating a patient at approximately the same time. Such a datagathering process was not feasible within the constraints of hospital resources. Validity Two types of content validity can be evaluated for the ASP. One is face validity, which refers to the extent to which an instrument looks as if it measures what it is intended to measure. Any instrument that is intended to have content validity must meet this standard, although face validity alone is not suf cient. The ASP has been found by users to address issues relevant to our patient population, and therefore to have face validity. A second type of content validity is logical or sampling validity. While face validity is evaluated after the instrument is constructed, logical or sampling reliability refers to the method by which an instrument is constructed. It involves the use of expertise to de ne an area of interest, drawing a representative sample of ideas or issues from this area, and preparing instrument items that match these ideas or issues (Roscoe, 1975). In the construction of the ASP, the existing professional literature and the expertise of hospital staff from all clinical disciplines were used to de ne the content areas relevant to the patient population. A group of experienced clinicians then developed
ATAS C A DE R O SK IL LS PR O F IL E
items that assess patient functioning in these areas. Finally, a multidisciplinary group conducted a detailed review of the items and scoring criteria in order to enhance clinical relevance and consistency in the behavioral anchoring of each item. The third type of validity evaluated for the ASP is a form of construct validity referred to as factorial validity. Based on factor analysis, clusters of highly correlated items are considered to represent the distinct constructs measured by an instrument (Nunnally, 1970). To evaluate the factor structure of the ASP, a principal components analysis was conducted using varimax rotation on the ASP ratings of a sample of 1,458 patients. The results of this analysis are presented in Table 3. The factor loadings contained in Table 3 demonstrate that the factor structure corresponds closely to the content domains of the ASP. The rst factor consists of items from the rst two skill domains, both of which focus on managing psychiatric symptoms. Each subsequent factor consists primarily of items from a distinct skill domain. The exception is the interpersonal domain. Items from this domain are moderately associated with two factors, one primarily loaded with the items of the ‘leisure and recreation’ domain and the second primarily loaded with the items of the ‘self-care’ domain. For all other domains, it can be concluded that the items measure a distinct construct, representing a speci c area of skill functioning. One form of criterion-related validity is referred to as predictive validity, or the degree to which a measure is able to account for differences in outcomes associated with the construct being measured. One preliminary source of data for evaluating the predictive validity of the ASP comes from posthospital evaluations conducted 3 months following discharge. These posthospital data indicate that the most common reasons for short-term failure of patients in their post-discharge environment include psychiatric decompensation, often with medication non-compliance, and substance abuse. These areas of skill-functioning have also been found to be among the most common and most severe areas of de cit in our patient population on the ASP, affecting from 80% to 95% of patients. The convergence of these two different data sources suggests validity for the areas measured by the ASP. Additional predictive validity evaluations are required. Speci cally, suf cient data must be collected to allow for the statistical determination of the level of functioning in critical skill domains which are associated with placement success or failure in distinct discharge environments (e.g. conditional release program, prison, other state hospital). Preliminary analysis of the limited available data indicates differences in the anticipated direction, whereby patients who have failed at the time of the 3-month follow-up had lower skill ratings at the time of discharge. These differences, however, do not currently reach statistical signi cance. Another type of validity that has not been evaluated is concurrent
601
602
J O U RN A L O F F O RE N SI C P SY C H IAT RY
Vol. 12 No. 3
Table 3 Results of factor analysis of ASP scores ASP item
factor 1
factor 2
factor 3
factor 4
factor 5
factor 6
factor factor 7 8
Behavior 1 Behavior 2 Behavior 3 Medication 1 Medication 2 Medication 3 Medication 4 Substance 1 Substance 2 Substance 3 Assault 1 Assault 2 Assault 3 Assault 4 Suicide 1 Suicide 2 Suicide 3 Self-care 1 Self-care 2 Self-care 3 Independent 1 Independent 2 Independent 3 Sex 1 Sex 2 Sex 3 Sex 4 Sex 5 Interpersonal 1 Interpersonal 2 Interpersonal 3 Recreation 1 Recreation 2 Recreation 3
0.732 0.705 0.703 0.843 0.784 0.796 0.672 0.287 0.173 0.255 0.297 0.256 0.231 0.253 0.165 0.170 0.137 0.137 0.133 0.163 0.193 0.116 0.185 0.111 0.006 0.008 0.008 0.007 0.284 0.263 0.295 0.181 0.107 0.146
0.006 0.006 0.007 0.107 0.120 0.107 0.008 0.163 0.129 0.167 0.008 0.009 0.008 0.008 0.007 0.006 0.008 0.003 0.004 0.006 0.003 0.010 0.005 0.884 0.914 0.916 0.724 0.919 0.007 0.009 0.006 0.009 0.009 0.010
0.312 0.348 0.319 0.146 0.174 0.173 0.002 0.216 0.009 0.228 0.822 0.858 0.853 0.793 0.157 0.142 0.135 0.122 0.114 0.140 0.150 0.004 0.147 0.008 0.007 0.004 0.004 0.005 0.105 0.105 0.219 0.192 0.133 0.183
0.009 0.116 0.120 0.010 0.139 0.170 0.250 0.140 0.231 0.169 0.153 0.176 0.150 0.197 0.010 0.009 0.009 0.184 0.175 0.235 0.276 0.205 0.170 0.003 0.007 0.007 0.007 0.008 0.460 0.547 0.557 0.780 0.840 0.794
0.007 0.005 0.005 0.130 0.178 0.159 0.213 0.141 0.126 0.115 0.123 0.122 0.150 0.122 0.009 0.009 0.107 0.850 0.857 0.790 0.178 0.306 0.135 0.002 0.001 0.005 0.005 0.005 0.500 0.432 0.415 0.171 0.139 0.194
0.107 0.151 0.114 0.101 0.105 0.116 0.008 0.110 0.007 0.102 0.111 0.142 0.158 0.131 0.909 0.917 0.892 0.008 0.009 0.008 0.005 0.008 0.004 0.005 0.006 0.004 0.005 0.003 0.112 0.007 0.117 0.109 0.006 0.008
0.162 0.234 0.159 0.274 0.183 0.309 0.133 0.000 0.105 0.141 0.129 0.009 0.010 –0.149 0.794 0.107 0.830 0.005 0.803 0.128 0.143 0.007 0.126 0.010 0.112 0.009 0.159 0.009 0.006 0.003 0.006 0.005 0.106 0.008 0.009 0.172 0.009 0.187 0.108 0.159 0.250 0.425 0.000 0.668 0.111 0.694 0.105 0.002 0.122 0.003 0.102 0.001 –0.004 0.106 0.121 0.000 0.129 0.008 0.141 0.001 0.120 0.001 0.146 0.241 0.155 0.151 0.122 0.222
validity. As alluded to in the introduction, over 70 instruments presented in the professional literature were reviewed for use at this facility prior to the development of the ASP. Each was found to have an inadequate relationship to the critical areas of skill-functioning which serve as the
ATAS C A DE R O SK IL LS PR O F IL E
focuses of treatment-planning and delivery with a forensic inpatient population. In the absence of an instrument considered by consensus to be the standard measure of functional skills in this field, there was no means of gathering concurrent data with which to compare the ASP. Descriptive statistics As normative data on the present patient population, Table 4 presents means and standard deviations of total scores in each skill domain for patients’ initial rating by the treatment team following admission and the nal rating prior to discharge. Total scores are the sum of all item ratings within a domain. These gures provide a description of the overall patient population at the beginning and end of hospitalization. However, these scores may represent an underestimation of the patients’ response to treatment in that these samples did not exclude patients for whom some domains were not speci ed as a focus of treatment. Sensitivity to change In order to evaluate sensitivity to change, initial and nal domain scores were compared using those patients for whom the domain was de ned as a focus of treatment throughout the current course of hospitalization. Table 5 presents the results of these paired t-tests. As can be seen, for each skill domain there was a signi cant increase in the average domain total score from patients’ initial assessment following admission to the nal assessment prior to discharge.
Table 4 Means and standard deviations for initial and nal ASP ratings Domain
Initial mean
Final mean
Initial SD
Final SD
N
Behavior Medication Substance abuse Assault Suicide Self-care Independent living Sexual deviance Interpersonal Leisure/recreation
3.16 7.47 2.57 3.75 4.73 7.53 3.86 2.63 6.02 4.99
4.08 8.10 3.59 4.94 4.84 7.90 4.35 4.00 6.82 5.79
3.10 3.77 3.04 4.23 4.09 2.50 2.46 3.82 2.44 2.88
3.15 3.91 3.60 4.36 3.48 2.62 2.84 4.84 2.41 2.80
1498 1494 1609 1481 676 1822 1880 457 1853 1846
603
604
J O U RN A L O F F O RE N SI C P SY C H IAT RY
Vol. 12 No. 3
Table 5 Means for initial and nal ASP ratings using domains that were a focus of treatment Domain
Initial mean Final mean
Signicance
N
Behavior Medication Substance abuse Assault Suicide Self-care Independent living Sexual deviance Interpersonal Leisure/recreation
3.19 6.75 3.32 3.42 3.22 6.81 4.45 3.57 6.52 5.34
p < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001
920 664 1039 757 345 846 842 374 1452 1456
4.44 8.61 4.71 5.37 4.99 7.66 5.19 4.72 7.15 6.24
Data applications Data from the ASP have been utilized in several ways. They have been incorporated into a computer-assisted treatment-planning process. Data from each ASP administration are entered into a computer on each unit connected to a local area network (LAN). Domains that are designated as a focus of treatment form the basis for assigning all treatment activities, and are listed on the written treatment plan. ASP data are electronically accessible, and can be aggregated for any level of the patient population. One of the routine applications is to produce a quarterly pro le of ASP scores for each treatment unit, and comparing this measure of patient functioning to the amount of treatment services provided. In this way, discrepancies in available services can be modi ed to better match current patient need. Because the ASP is administered following admission and at each 90-day treatment-planning conference until discharge, it provides a repeated measure of patient response to treatment for both individual treatmentplanning and program evaluation. It is also used in conjunction with postdischarge data on patient-functioning to account for differences in clinical outcomes, which provides additional information for program development. CONCLUSION The ASP serves the purposes for which it was designed. It organizes the initial comprehensive functional assessment of forensic psychiatric inpatients in
ATAS C A DE R O SK IL LS PR O F IL E
areas relevant to post-discharge success. It is incorporated into treatment planning, whereby skill de cits are prioritized for the assignment of treatment activities based on the demands of the expected discharge setting. It is used in outcome evaluation, both in terms of response to the course of care during hospitalization and the association of skill acquisition with postdischarge outcomes. The direct relationship between the ASP, treatmentplanning and outcome evaluation, facilitated through a computer-assisted process, represents an advance from previous assessment instruments. The ASP meets the criteria set forth for instruments used in treatment outcome assessment (Newman and Ciarlo, 1994). Preliminary analyses have demonstrated adequate psychometric properties. It is sensitive to change and has been shown to be related to post-hospital outcomes. Implementation of the ASP has facilitated the standardization of clinical procedures based on a biopsychosocial rehabilitation approach to treatment. As part of a larger outcome evaluation and program development system, it is intended to maximize the utilization of scarce and expensive resources in the treatment of forensic psychiatric inpatients. James Vess, PhD, senior supervising psychologist, Evaluation and Outcome Services, Atascadero State Hospital, PO Box 7001, Atascadero, CA 93423, USA. [e-mail:
[email protected]]
REFERENCES Afeck, J. W. and McGuire, R. J. (1984) ‘The Measurement of Psychiatric Rehabilitation Status: a Review of the Needs and a New Scale’. British Journal of Psychiatry 145: 517–25. Anthony, W. A. (1980) The Principles of Psychiatric Rehabilitation. Baltimore, MD: University Park Press. Anthony, W. A. and Farkas, M. (1982) ‘A Client Outcome Planning Model for Assessing Psychiatric Rehabilitation Interventions’. Schizophrenia Bulletin 8: 13–38. Baker, R. and Hall, J. N. (1988) ‘Rehab: a New Assessment Instrument for Chronic Psychiatric Patients’. Schizophrenia Bulletin 14: 97–110. Dilk, M. N. and Bond, G. R. (1996) ‘Meta-Analytic Evaluation of Skills Training Research for Individuals with Severe Mental Illness’. Journal of Consulting and Clinical Psychology 64: 1337–46. Ellsworth, R. B. (1971) The MACC Behavioral Adjustment Scale: Revised. Los Angeles: Western Psychological Services. Evenson, R. C. and Boyd, M. A. (1993) ‘The St. Louis Inventory of Community Living Skills’. Psychosocial Rehabilitation Journal 17(2): 93–9. Green, R. S. and Gracely, E. J. (1987) ‘Selecting a Rating Scale for Evaluating Services to the Chronically Mentally Ill’. Community Mental Health Journal 23(2): 91–102. Hall, J. N. (1980) ‘Ward Rating Scales for Long-Stay Patients: a Review’. Psychological Medicine 10: 277–88.
605
606
J O U RN A L O F F O RE N SI C P SY C H IAT RY
Vol. 12 No. 3
Harris, V. and Koepsell, T. D. (1996) ‘Criminal Recidivism in Mentally Ill Offenders: a Pilot Study’. Bulletin of the American Academy of Psychiatry and Law 24(2): 177–86. Harris, V. and Koepsell, T. D. (1998) ‘Rearrest among Mentally Ill Offenders’. Journal of the American Academy of Psychiatry and Law 26(3): 393–402. Honigfeld, G. and Klett, C. J. (1965) ‘The Nurses’ Observation Scale for Inpatient Evaluation’. Journal of Clinical Psychology 21: 65–71. Katz, L. J. (1998) ‘Assessment and Planning for Psychosocial and Vocational Rehabilitation’. In G. Goldstein and S. R. Beers (eds) Rehabilitation. New York: Plenum Press, pp. 247–66. Knapp, T. R. (1985) ‘Validity, Reliability, and Neither’. Nursing Research 34: 189–92. Kopelowicz, A. and Liberman, R. P. (1995) ‘Biobehavioral Treatment and Rehabilitation of Schizophrenia’. Harvard Review of Psychiatry 3(2): 55–64. Newman, F. L. and Ciarlo, J. A. (1994) ‘Criteria for Selecting Psychological Instruments for Treatment Outcome Assessment’. In M. E. Maruish (ed.) Use of Psychological Testing for Treatment Planning and Outcome Assessment. Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 98–110. Norbeck, J. S. (1985) ‘What Constitutes a Publishable Report of Instrument Development?’ Nursing Research 34: 380–2. Nunnally, J. C. (1970) Introduction to Psychological Measurement. New York: McGraw-Hill. Roscoe, J. T. (1975) Fundamental Research Statistics for the Behavioral Sciences. New York: Holt, Rinehart & Winston. Ventura, L. A., Cassel, C. A., Jacoby, J. E. and Huang, B. (1998) ‘Case Management and Recidivism of Mentally Ill Persons Released from Jail’. Psychiatric Services 49: 1330–7. Wallace, C. J. (1986) ‘Functional Assessment in Rehabilitation’. Schizophrenic Bulletin 12: 604–30. Wallace, C. J., Lecomte, T., Wilde, J. and Liberman, R. P. (in press) Schizophrenia Research. Weideranders, M. R. (1992) ‘Recidivism of Disordered Offenders Who Were Conditionally vs. Unconditionally Released’. Behavioral Sciences and the Law 10: 141–8. Weideranders, M. R., Bromley, D. L. and Choate, P. A. (1997) ‘Forensic Conditional Release Programs and Outcomes in Three States’. International Journal of Law and Psychiatry 20: 249–57. Weiner, H. R. (1993) ‘Multi-Function Needs Assessment: the Development of a Functional Assessment Instrument’. Psychosocial Rehabilitation Journal 16(4): 51–6.
APPENDIX: ATASCADERO SKILLS PROFILE Scoring: Unk = insufcient information to rate skill N/A = skill area does not apply to this patient 0 = skills consistently absent or completely inadequate
ATAS C A DE R O SK IL LS PR O F IL E 1 = skills almost always absent or mostly inadequate 2 = skills occasionally present or partially adequate 3 = skills almost always present and adequate 4 = skills consistently present and adequate Circle the number of each current focus of treatment. I SELF -MANAGEMENT OF PSYCHIATRIC SYMPTOMS
– BEHAVIOR
_____ 1 Patient acknowledges psychiatric symptoms and how they affect his behavior. _____ 2 Patient can articulate warning signs which precede decompensation. _____ 3 Patient demonstrates realistic relapse prevention strategies appropriate to his discharge setting. II SELF -MANAGEMENT OF PSYCHIATRIC SYMPTOMS
– MEDICATION
_____ 1 Patient accepts need for prescribed medication. _____ 2 Patient demonstrates knowledge of prescribed medications including bene ts and side-effects. _____ 3 Patient demonstrates the ability to negotiate medication issues with physician. _____ 4 Patient complies with medications. III SUBSTANCE ABUSE PREVENTION SKILLS
_____ 1 Patient accepts substance abuse as a problem; recognizes how it has affected his functioning. _____ 2 Patient actively participates in some form of on-going substance abuse prevention treatment. _____ 3 Patient has realistic plan to prevent or avoid substance abuse in the community. IV SELF - MANAGEMENT OF ASSAULTIVE BEHAVIOR
_____ 1 Patient recognizes that he has a problem with assaultive behavior. _____ 2 Patient identi es high-risk situations and speci c external stimuli related to violent behavior. _____ 3 Patient recognizes internal precursors which escalate into violent behavior. _____ 4 Patient demonstrates realistic coping strategies to prevent assault. V CONTROL OF SELF-INJURIOUS OR SUICIDAL BEHAVIOR
_____ 1 Patient identi es high-risk situations and speci c external stimuli related to self-injurious or suicidal behavior.
607
608
J O U RN A L O F F O RE N SI C P SY C H IAT RY
Vol. 12 No. 3
_____ 2 Patient recognizes internal precursors which escalate to self-injurious or suicidal behavior. _____ 3 Patient demonstrates realistic coping strategies to prevent self-injurious or suicidal behavior. VI SELF CARE
_____ 1 Patient demonstrates ability to independently bathe, dress, feed and groom himself. _____ 2 Patient adequately maintains his personal property and living space. _____ 3 Patient manages his physical health functions by appropriately utilizing medical treatments, dietary/nutritional information, and other health maintenance activities. VII INDEPENDENT LIVING SKILLS
_____ 1 Patient demonstrates adequate work skills, and meets expectations of a work assignment. _____ 2 Patient demonstrates the ability to read and write adequately to manage his responsibilities in the dispositional environment. _____ 3 Patient demonstrates adequate skills for utilizing community resources to function in an independent living environment, including money management, transportation, etc. VIII CONTROL OF DEVIANT SEXUAL IMPULSES AND BEHAVIORS
_____ _____ _____ _____ _____
1 2 3 4 5
Patient accepts responsibility for his past deviant sexual behavior. Patient understands the trauma that resulted from his sexual crimes. Patient can correct deviant thoughts that promote sexual offending. Patient demonstrates ability to manage deviant sexual urges and impulses. Patient demonstrates ability to cope with high-risk factors for sexual reoffending. IX INTERPERSONAL SKILLS
_____ 1 Patient demonstrates adequate basic verbal and non-verbal communication for meeting daily needs. _____ 2 Patient demonstrates social interactions which respect the privacy, property and feelings of others. _____ 3 Patient demonstrates effective interpersonal problem-solving skills with peers and staff.
ATAS C A DE R O SK IL LS PR O F IL E X LEISURE AND RECREATION SKILLS
_____ 1 Patient recognizes the value of his involvement in constructive leisure activities. _____ 2 Patient engages in scheduled, prescribed leisure activities. _____ 3 Patient independently initiates, schedules and engages in appropriate leisure activities.
609
Copyright of Journal of Forensic Psychiatry is the property of Routledge and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
