Psychotherapy Research Does psychotherapy for

This article was downloaded by: [Linkopings University] On: 27 May 2010 Access details: Access Details: [subscription number 917403353] Publisher Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 3741 Mortimer Street, London W1T 3JH, UK

Psychotherapy Research

Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t713663589

Does psychotherapy for young adults in routine practice show similar results as therapy in randomized clinical trials? Fredrik Falkenströma a Samtalscentrum Unga Vuxna, Nyköping, Sweden First published on: 09 October 2009

To cite this Article Falkenström, Fredrik(2010) 'Does psychotherapy for young adults in routine practice show similar

results as therapy in randomized clinical trials?', Psychotherapy Research, 20: 2, 181 — 192, First published on: 09 October 2009 (iFirst) To link to this Article: DOI: 10.1080/10503300903170954 URL: http://dx.doi.org/10.1080/10503300903170954

PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

Psychotherapy Research, March 2010; 20(2): 181192

Does psychotherapy for young adults in routine practice show similar results as therapy in randomized clinical trials?

¨M FREDRIK FALKENSTRO Samtalscentrum Unga Vuxna, Nyko¨ping, Sweden

Downloaded By: [Linkopings University] At: 08:08 27 May 2010

(Received 11 December 2007; revised 3 June 2009; accepted 2 July 2009)

Abstract Previous research indicates that patients treated with psychotherapy in the community do not stay in treatment long enough to achieve clinically significant change. Because the average patient seeking treatment at a community center may not be as informed and motivated for change as the average patient participating in a research trial, the authors compared outcome among all patients presenting to a mental health clinic (n 416) and a subgroup of patients who started psychotherapy at the same clinic (n 101). Outcome was assessing using the Symptom Checklist-90, Inventory of Interpersonal Problems, and Global Assessment of Functioning. Results confirmed that outcome among all patients seeking the centre was worse than in an average clinical trial. However, patients who started psychotherapy after assessment achieved results comparable to those in RCT studies.

Keywords: mental health services research; outcome research; psychoanalytic/psychodynamic therapy

The general efficacy of psychotherapy is well established in controlled outcome research (Lambert & Ogles, 2004). For the most part, research to date suggests that none of the established forms of psychotherapy is superior to another (Wampold, 2001). Although psychodynamic psychotherapy had a late start in performing outcome research compared with cognitive and behavioral therapies, accumulating research documents its effectiveness for a range of psychiatric disorders (Abbas, Hancock, Henderson, & Kisely, 2006; Leichsenring, Rabung, & Leibing, 2004; Roth & Fonagy, 2005). However, the extent to which results from clinical trials can be generalized to clinical practice is still unclear. Can the Results of Research Therapy be Generalized to Clinical Practice? Research therapy usually has a number of characteristics that distinguish it from therapy in ‘‘the real world.’’ Weisz, Donenberg, Han, and Weiss (1995) mention the following differences between research therapy and clinic therapy: In research therapy, (a) patients are recruited by researchers rather than referred or self-referred; (b) samples are selected for

homogeneity, and therapy addresses one or two focal problems; (c) therapists receive special training in the techniques they are going to use; (d) therapy involves primary or exclusive adherence to these specific techniques; and (e) therapy is often highly structured, guided by a manual or monitored for adherence to a treatment plan. Clinic therapy, on the other hand, usually has none of these characteristics. Clinic therapists have to be flexible to adapt to patients with many different diagnoses or problems, and often they present with more than one problem at a time. Although therapists usually are trained in one orientation, they are not required to follow a manual and there are no adherence or competence checks. Weisz et al. (1995) also mention that research therapists often have smaller caseloads than clinic therapists. Thus, because of more favorable conditions, it seems reasonable to assume that research therapy would show superior results to clinic therapy. Indeed, the early meta-analysis of Weisz et al. (1995) yielded a mean effect size of clinic therapy for children and adolescents of .01. However, the authors cautioned that the results were preliminary because they were based on only nine studies. Later

Financial support for this study was obtained from the So¨rmland County Council Research and Development Center (Landstinget So¨rmlands FoU). Correspondence concerning this article should be addressed to Fredrik Falkenstro¨m, Samtalscentrum Unga Vuxna, Repslagaregatan 5a, Nyko¨ping SE-611 30, Sweden. E-mail: [email protected] ISSN 1050-3307 print/ISSN 1468-4381 online # 2010 Society for Psychotherapy Research DOI: 10.1080/10503300903170954


182

F. Falkenstro¨ m

meta-analyses by Shadish et al. (1997) and Shadish, Navarro, Matt, and Phillips (2000) showed effect sizes for clinic therapy comparable to those yielded by meta-analyses of research therapy. Shadish et al. (2000) also concluded that clinically representative studies more often use nonrandomized control groups, and that self-selection bias artificially makes these treatments look less effective. A recent study in Sweden is also worth mentioning in this context. The study compared manualized psychodynamic psychotherapy with community-delivered psychodynamic psychotherapy as usual for patients with personality disorders in a randomized design. The results showed that the community-delivered treatment (clinic therapy) was as effective as the manualized (research) therapy (Vinnars, Barber, Noren, Gallop, & Weinryb, 2005). In their meta-analysis of clinically representative psychotherapy, Shadish et al. (2000) also found that studies are becoming less representative of clinical conditions, presumably because researchers increasingly try to adopt the gold standard of the randomized clinical trial (RCT). To some extent there seems to be a trade-off between experimental rigor and clinical representativeness. The more internal validity is emphasized, the less clinically representative the study becomes (even if there are exceptions, as noted by Chambless & Hollon, 1998). Thus, there is a need for research on naturalistic samples in order to study clinically representative conditions. Results from naturalistic studies of psychotherapy have been mixed: Two U.S. studies (Hansen, Lambert, & Forman, 2002; Hansen & Lambert, 2003) showed that patients in clinical practice received too few sessions to generate clinically significant improvement to the same degree as in clinical trials. Using the doseeffect model (i.e., more therapy leads to more improvement), Hansen et al. (2002) showed that patients in various mental health centers in the United States received on average between three and nine sessions, even after all patients who received only one session (one third of the original sample) had been dropped from the study. Results on the Outcome Questionnaire-45 (OQ-45) showed that at termination only between 20 and 31% were reliably improved and between 9 and 20% were recovered. This contrasted sharply with patients in clinical trials who received a mean of 13 sessions, with a mean reliable improvement rate of 67% and a clinically significant improvement rate of 58% (Hansen et al., 2002). Although no information on missing data was reported in these studies (except for the exclusion of patients with only one session), the methodology of having patients complete the OQ-45 before each session is likely to result in low rates of missing data.

Naturalistic studies conducted in the United Kingdom (Barkham et al., 2006; Stiles, Barkham, Connell, & Mellor-Clark, 2008; Stiles, Barkham, Mellor-Clark, & Connell, 2008; Stiles, Barkham, Twigg, Mellor-Clark, & Cooper, 2006) have come to different conclusions. In these studies, patients achieved nearly identical improvement rates as in clinical trials, and the authors found no evidence of a doseresponse effect. The mean number of sessions attended was about the same as in the U.S. samples, about seven sessions (Stiles, Barkham, Connell et al., 2008). Even though based on large samples, these results must be treated with caution because of high rates of attrition. For example, in the largest study (Stiles, Barkham, Mellor-Clark et al., 2008), approximately 64% of patients were excluded because of missing data. Thus, the issue of effectiveness of psychotherapy in regular practice has not been settled definitely yet. An issue that, to our knowledge, has not been raised before is that results such as those found by Hansen et al. (2002) are caused by preexisting differences between patients in clinical trials and patients in regular practice rather than by the treatments they receive. Specifically, we believe that in clinical trials there may be a selection process before patients start therapy. Patients who come to a trial have to endure several time-consuming tests as well as diagnostic interviews before starting therapy. Also, there is a selection process even earlier, involving the procedure of being referred or self-referred to a psychotherapy research project. Patients who present for treatment because of pressure from others or who have not yet made a commitment to change are unlikely to be referred to a psychotherapy research study but may be common in clinical practice. Patients who are never referred to or who drop out from clinical trials before they have been assessed do not affect the dropout statistics of the study, but the result will nonetheless be that the patients who actually start therapy are more motivated for change than the average patient seeking a community mental health center. To assess the effectiveness of clinic therapy, a more fair comparison would be to contrast those patients who actually start therapy in the clinic with those who start therapy in a research trial. Thus, in the present study, wetested twohypotheses: 1. The average patient coming to a community mental health center receives fewer sessions and does not improve as much as the average patient in a typical research trial. 2. Patients who, after an assessment phase, start psychotherapy proper stay as long in treatment

Psychotherapy for young adults in routine practice and improve in similar numbers as in a typical research trial.


To test these hypotheses, we used data from a community mental health center for young adults ages 16 to 23 years. The center has a psychodynamic orientation and offers counseling and psychotherapy to all patients who want someone to talk to. The main purpose of the study is to assess the effect of psychotherapy in clinical practice using two different samples, one that includes all patients seeking an appointment at the center and another that includes only those patients starting psychotherapy. Because the present study is testing our alternative interpretation of the results of the Hansen et al. (2002) and Hansen and Lambert (2003) studies, the main comparisons of our data are with results from these studies. Method Setting In summer 2003, a project was initiated to evaluate the outcome of psychotherapy delivered at Samtalscentrum Unga Vuxna (Young Adults Counseling Center [YACC]) in Nyko¨ ping, Sweden. YACC is a small community-based clinic within the Swedish mental health care system. Its task is to offer counseling and psychotherapy to young people ages 16 to 23. YACC is not a traditional psychiatric outpatient clinic in that it only accepts self-referred patients. This means that YACC does not accept referrals from parents or mental health professionals. However, in practice, it turns out that most patients have problems that are well on a level with the typical patient in a standard outpatient psychiatric clinic, and many who come to the center have previously had contact with the local psychiatric clinic. Patients are offered an initial appointment within 10 days of first contact and are guaranteed at least four sessions of free counseling. Young adults come to YACC with problems varying widely in type and severity. Some are helped by a few sessions of counseling, although many have problems that require further treatment. For these patients, psychotherapy is offered, although the need to adhere to the 10-day time limit for a first appointment means that patients sometimes have to wait a few months for psychotherapy after the initial sessions. No patients are excluded from a first appointment at YACC, under the condition that they themselves request it by telephone or e-mail. Therapists refer psychotic or substance-dependent patients to specialist units, in the rare case that such

183

patients seek the center. Apart from these, no definite exclusion criteria are used. The center has a psychodynamic rather than psychiatric orientation, which means that the first-line treatment is always psychological rather than pharmacological. However, in some cases, the therapists refer patients to the center’s consultant psychiatrist, a psychotherapeutically trained psychiatrist, for adjunct medication or sick leave issues. The number of patients who are prescribed psychopharmacological medications is quite low.

Patients Sample 1. This sample is based on the center’s archival data on the 416 patients who had at least two appointments during the study time (August 2003December 2006).1 Of these, 104 (25%) were men and 312 (75%) women. The mean age was 19.1 years (SD 2.9). Most were native Swedes, only 16 (4%) were first-generation immigrants and another 15 (4%) second-generation immigrants (at least one parent was an immigrant). Nine (2%) were adopted. In terms of living arrangements 213 (52%) of the patients lived with their parents, 106 (23%) lived alone, 51 (14%) lived with a partner, 11 (2%) lived with a friend, and 23 (6%) in a foster home or an institution. Table I shows the primary Diagnostic and Statistical Manual of Mental Disorders (fourth edition [DSMIV]; American Psychiatric Association, 1994) Axis I and Axis II diagnoses in Sample 1. Because diagnoses were assigned clinically, a method known to underestimate comorbidity in comparison to structured interviews (Westen, 1997), we have chosen to present only the primary diagnosis for each patient on each axis. As Table I shows, the most common Axis I diagnoses in Sample 1 were mood (30%) and anxiety (24%) disorders. Because many patients in Sample 1 were seen for only a few sessions, 16% could not be diagnosed because of lack of information. In addition, 15% were diagnosed with a personality disorder. When it comes to Axis II diagnoses, the therapists are even more restrictive with diagnoses when information is lacking or when the patient is very young (especially those younger than 18). Therefore, it is likely that this figure is a very conservative estimate of the prevalence of personality disorders in this sample. Of the 416 patients, 17% had been at YACC at least once before. This confirms the therapists’ impression that although many patients are not motivated enough to continue with psychotherapy when they first seek the center, they often return at a later date.

184

F. Falkenstro¨ m

Table I. Primary DSM-IV Axis I and II Diagnoses


Diagnosis

Sample 1 (%) (n 416)

Axis I Mood disorders (total) Major depressive disorder Depressive disorder NOS Dysthymic disorder Anxiety disorders (total) Anxiety disorder NOS Panic disorder with or without agoraphobia Social phobia Obsessivecompulsive disorder Posttraumatic stress disorder Generalized anxiety disorder Other anxiety disorders Other diagnoses (e.g., drug abuse, impulse control disorders, eating disorders, adjustment disorders) Other conditions that may be a focus of clinical attention (e.g., relationship problems, phase of life problems, cultural adaptation problems, abuse) Not enough information to establish diagnosis Axis II Personality disorder (total) Cluster A Cluster B (mostly borderline) Cluster C (avoidant and dependent) Personality disorder NOS Mental retardation

15.4 1 3 4 7 0.4

Sample 2 (%) (n 101)

30 14 14 2 24 8 6 3 3 2 1 1 16

42 26 13 3 34 13 9 4 4 2 1 1 10

13

14

16

0

28 2 6 12 8 0

Note. DSM-IV Diagnostic and Statistical Manual of Mental Disorders (4th edition); NOS not otherwise specified.

Sample 2. This sample2 consists of all patients who agreed to start psychotherapy proper during the study time and who had finished their treatments by the end of 2006. Informed consent was obtained when there was an agreement to start psychotherapy, usually after about three to five sessions. Although participation was voluntary, only one patient refused participation. Of the 101 young adults who started and finished psychotherapy during the study time, 80 completed their therapy, 16 dropped out,3 and five were referred elsewhere for other treatments after some time in therapy (Figure I). Dropouts were compared with completers on all subscales of the Symptom Checklist-90 (SCL-90) and Inventory of Interpersonal Problems (IIP) as well as the total scores on both measures. Only the Interpersonal Sensitivity subscale of the SCL-90 differed significantly between dropouts and completers: Dropouts were more interpersonally sensitive, t(67)2.1, p.05. The Paranoia subscale of the SCL-90 and the Overly Nurturant subscale of the IIP were close to significance, t(67)1.7, p.10, and t(69)2.0, p .06, respectively, with dropouts having higher scores. Although it makes sense to think that dropouts are more interpersonally sensitive and paranoid, having

done 20 statistical tests this is approximately what is expected by chance. All patients attending at least one session during study time N = 416 (Sample 1)

Patients wanting / judged suitable for psychotherapy, n = 101 (Sample 2)

Patients completing therapy, n = 80

Full data Time 1 SCL-90: n = 56 IIP: n = 58 GAF: n = 80

Full data Time 2 SCL-90: n = 48 IIP: n = 49 GAF: n = 78

Dropouts, n = 16, (full data on 10 at T1)

Referred for other treatment, n = 5, (full data on 4 at T1)

Full data Time 3 SCL-90: n = 41 IIP: n = 42

Figure I. Flow chart of patients seeking an appointment, starting and completing psychotherapy, or dropping out. (SCL-90 Symptom Checklist-90; IIP Inventory of Interpersonal Problems; GAFGlobal Assessment of Functioning.)


Psychotherapy for young adults in routine practice The sample comprised 83 (82%) women and 18 (18%) men, with a mean age of 19.0 years (SD 1.8) at therapy start. Most patients were native Swedes; only one was a first-generation immigrant, although another five had at least one parent from a foreign country, and one was adopted. Fifty-four percent (53) of patients lived with their parents, 24% (25) lived alone, and 11% (11) lived with a partner, with a friend (4, 4%), or in foster homes (4, 4%). Of the total, 45% were attending high school, 24% were employed, 21% were unemployed, and 10% were in college or pursuing other levels of education. None of the patients were married. Twenty-one percent of the patients had sought treatment at YACC at least once before. DSM-IV diagnoses for all patients (n 101) in Sample 2 are presented in Table I. As can be seen, the most common Axis I diagnoses were mood disorders (42%), followed by anxiety disorders (35%). Twenty-eight percent of patients fulfilled diagnostic criteria for any personality disorder, most commonly avoidant or dependent personality disorder (12%) or personality disorder NOS (8%).

185

love and sexuality, demands in school or at work, the need to know what career to pursue, and the separation from the family of origin put the young adult ego under intense strain. This often leads to regression and symptom formation. Also, many patients need to work with traumas such as sexual or physical abuse. Techniques are generally explorative and include reflective listening, clarification, confrontation, and interpretation. The degrees of supportiveness and expressiveness vary between patients and phases of therapy. Interventions focus on emotional and relational issues, and the relationship between therapist and patient is seen as key to change. Transference and countertransference are constantly monitored and talked about with the patient when deemed appropriate. Therapists receive supervision regularly (2 hr every 2 weeks) by an experienced dynamic therapist or psychoanalyst (there have been three different supervisors since the center’s start in 2000) in a small-group format. Most of the treatments in the study were time unlimited, and the decision to end therapy was a matter of discussion between patient and therapist.

Therapists During the study time, seven therapists (two men and five women) had worked at YACC: five psychologists, one social worker, and one psychiatric nurse with basic psychotherapeutic training. The therapists varied widely in age and clinical experience. Treatments The first few treatment sessions at YACC comprise combined assessment, counseling, and psychotherapy. The aims are to find out what presenting patients want to get out of their contact, to make a rough assessment of patients’ level of psychic functioning, and to build a therapeutic alliance. For those patients who, after these initial sessions, feel they have a problem for which they need further help, a therapy contract is negotiated. Others may be satisfied with the help they have received already or may find that this particular clinic does not offer what they need, and the therapist may help them with advice on where to find other kinds of treatment. The psychotherapy delivered at YACC is psychodynamic in orientation. When working with young adults, the therapist takes a somewhat more active stance than with older adult patients, especially early in treatment. Many of the symptoms experienced by the patients are thought to be triggered by the phase-specific stresses of the transition from childhood to adulthood. Central issues such as

Measures Global Assessment of Functioning (GAF; American Psychiatric Association, 1994). The GAF scale measures a patient’s general level of psychiatric symptoms and functioning, ranging from 1 (extremely low functioning and severe symptoms) to 100 (extremely well functioning and no symptoms). Ratings were done by the treating therapist at Session 1 and at termination in discussion with the other therapists at the center. This was done for all patients seeking the center, not just those who continued with psychotherapy. SCL-90 (Derogatis et al., 1974). With three global measures and eight subscales, the SCL-90 measures different types of psychiatric and somatic symptoms on a scale ranging from 0 (not at all) to 4 (very much). In the present study, the General Severity Index (GSI), which is the mean of all 90 items, was used as a primary outcome measure, whereas the eight subscales were used for post hoc exploratory analyses. IIP (Horowitz, Rosenberg, Baer, Uren˜o, & Villasen˜or, 1988). The IIP measures interpersonal problems on a scale from 0 (not at all) to 4 (very much). The original IIP had 127 items, but for the present study the Swedish version of the 64-item circumplex version was used (Horowitz, Alden, Wiggins, & Pincus, 2002). The mean of all 64 items was used as a primary outcome measure of general

186

F. Falkenstro¨ m

interpersonal distress, whereas the eight circumplex subscales were used for post hoc exploratory analyses. The self-report measures were administered at three occasions: therapy start, termination, and at 1-year follow-up.


Data Analyses Data were analyzed both for the group as a whole and for each individual. Group data were analyzed using the linear mixed-models function of SPSS 17.0. Mixed models were chosen instead of traditional repeated measures analysis of variance because mixed-models analysis has the advantage that all available data are included in the analysis regardless of missing measurements. Data can thus be analyzed with an intent-to-treat approach without resorting to imputation of missing data. Also, less restrictive assumptions need to be made about the data, and covariance structures can be modeled flexibly. Mixed-model analyses were conducted using restricted maximum likelihood estimation. An unstructured mean model was chosen because preliminary data exploration showed a nonlinear time trend. Also, we wanted to compare change at termination as well as follow-up regardless of treatment length (which was highly variable). An unstructured mean model is suitable in such cases because time is treated as a categorical rather than a continuous variable. However, because we also wanted to control for the possibility that the variation in treatment length may have influenced outcome, treatment length (natural log transformation of months of treatment) was entered as a covariate in all mixed-model analyses. Covariance structures for the three dependent measures were chosen on the basis of Schwarz’s Bayesian information criterion (BIC). The covariance structures were set to unstructured (GAF), first-order autoregressive (GSI), and compound symmetry (IIP). Residual and normality plots were used to examine the assumptions of the fitted models. Effect sizes for within-group change were calculated based on estimated marginal means and standard deviations from the mixedmodel analyses. Individual change was analyzed using Jacobson and Truax’s (1991) methods for analyzing clinical significance. Because we did not have a reliability estimate for the GAF ratings in this study, we defined reliable improvement as an increase of at least 10 points on the GAF scale. The choice of 10 points was done on both empirical and conceptual grounds. So¨ derberg and Tungstro¨m (2006), using a database of almost 12,000 patients from outpatient psychiatric services, calculated both a Reliable

Change Index (RCI) for the GAF scale and Jacobson and Truax’s criterion c for distinguishing between normal and pathological scores. An RCI of 10 GAF points was found based on the intraclass correlation (ICC1,1) of 0.81 from the same authors’ reliability study (So¨ derberg, Tungstro¨m, & Armelius, 2005), which, according to these authors, can be generalized to other outpatient services. This also makes sense conceptually, because the GAF scale is constructed using intervals of 10 points. The authors also found that a GAF score of 69 or higher was more likely to belong to a normal than to a clinical population (criterion c). For the self-report measures, RCI was calculated for each patient using reliability coefficients reported for the Swedish norm groups for the SCL-90 (Fridell, Cesarec, Johansson, & Malling Thorsen, 2002) and IIP (Horowitz et al., 2002) to estimate the amount and statistical significance of each individual patient’s change. A cutoff point at which a patient is more likely to belong to a functional rather than dysfunctional population was also calculated for these measures (criterion c; Jacobson & Truax, 1991). Because the Swedish SCL-90 norms show significant age and sex differences (Fridell et al., 2002), four different cutoff points for the GSI were calculated depending on whether the patient was male or female and between 16 to 19 or 20 to 25 years of age. Each patient’s score was evaluated according to the age group he or she belonged to at the time of completing the questionnaire for the first time. Using the norms for both clinical and nonclinical samples reported by Fridell et al. (2002), we came up with the following cutoff points for the GSI: c .95 for women between 16 and 19 years of age, .84 for women 20 to 25 years, .66 for men 16 to 19 years, and .71 for men 20 to 25 years. Because we did not have access to clinical norm data for the IIP, we used the cutoff point of c1.19 reported by Woodward, Murrell, and Bettler (2005). Results Reliability and Validity of GAF Ratings The therapists at the center meet every week to discuss cases and assign GAF ratings, but no interrater reliability checks are made. Although GAF ratings assigned by clinicians in regular outpatient practice have been found reliable in previous research (So¨derberg et al., 2005), we still wanted to get some indication of validity of the ratings in the present study. We, therefore, correlated a subset of GAF ratings (for patients who agreed on a psychotherapy contract) with independent measures of

Psychotherapy for young adults in routine practice


self-reported symptom severity (the GSI of the SCL90) and general interpersonal distress (the mean of all IIP items). Therapist GAF ratings correlated significantly with self-reported symptom severity (r .52, n 73, p B.0001, at intake and r .55, n 53, p B.0001, at termination) and interpersonal distress (r .43, n 76, p B.0001, at intake and r .51, n 56, p B.0001, at termination). Thus, the higher GAF rating by the therapist, the fewer and less severe symptoms and relationship problems reported by the patient. The correlations were moderately high, which is what might be expected because the scales are not meant to be identical. This supports the validity of the therapists’ GAF ratings, because symptom severity and relationship problems are among the most important issues that the GAF scale is supposed to measure. Hypothesis 1: The Average Patient Coming to a Community Mental Health Center Receives Fewer Sessions and Does Not Improve as Much as the Average Patient in a Typical Research Trial To test this hypothesis we used Sample 1, which consisted of all patients coming for at least two sessions to YACC during the study time. For Sample 1 the only outcome measure available was the therapist GAF rating at Session 1 and at termination. Number of sessions attended. The mean number of sessions attended for all 416 patients was 7.5 (SD 10.6, 95% confidence interval [CI]: 6.5 8.5, mdn 4). This is slightly more than in the mental health centers in the Hansen et al. (2002) study, but the difference is quite small. It is also much less than the 12.7 (SD 4.6) sessions that Hansen et al. found for the average clinical trial. Outcome. Three extreme outliers with GAF values at or below 20 were removed from the mixed-model analyses. The mixed model (n 399) showed a statistically significant main effect of time, F(1, 365) 228, pB.001, and treatment length, F(1, 397) 4.6, p .03. The estimated mean GAF value at Session 1 was 56.3 (SD 8.4, 95%CI: 55.5 57.1), representing ‘‘moderate symptoms or moderate difficulty in social, occupational or school functioning,’’ and at termination 60.9 (SD 10.0; 95%CI: 59.961.9), which is at the threshold between ‘‘moderate symptoms etc’’ and ‘‘mild symptoms or some difficulties in functioning in social, occupational or school functioning.’’ This mean change of 4.6 GAF points represents an effect size of .55.

187

Seventy-one patients (19%) achieved reliable improvement (at least 10 GAF points). Of these, 34 patients (9%) also crossed the boundary from a clinical to a normal population (clinically significant improvement according to Jacobson & Truax, 1991). These results are at the lower end of the results reported by Hansen et al. (2002). Hypothesis 2: Patients Who, after an Assessment Phase, Start Psychotherapy Proper Stay as Long in Treatment and Improve in Similar Numbers as in a Typical Research Trial To test Hypothesis 2, we examined change in psychiatric symptoms, interpersonal problems, and general functioning during and after therapy for the group of patients who started psychotherapy (Sample 2). Attrition. As in most research on therapy conducted in clinical practice, there was considerable attrition of self-report measures. Twenty-eight percent of patients did not return or were never given questionnaires at therapy start. Of those who did return questionnaires at therapy start, another 9 to 24% failed to return questionnaires either at termination or at follow-up. However, there was almost no attrition of therapist GAF ratings (see Figure I). The attrition of questionnaires was, for the most part, due to patients forgetting to complete the questionnaires and the therapists, in turn, forgetting to remind their patients. Only one patient explicitly refused to complete the questionnaires. Because the relatively high rate of attrition on selfreport measures raises the risk of introducing bias in the sample, we contrasted the group of patients (n 51) who completed at least one self-report measure both at intake and at termination with the group of patients (n 29) who did not on the variables age, sex, initial GAF rating, and change in GAF from pre- to posttherapy using relevant statistical tests. We found no statistically significant differences between groups on either variable. Of course, this does not rule out the possibility of bias, although the fact that therapist-rated outcome was not significantly lower for those without complete questionnaires makes it unlikely that the group who forgot to complete their questionnaires had significantly worse outcome than the rest. Among patients who completed at least one selfreport measure, there was a nonsignificant trend for patients who had missing data at Time 1 to score somewhat higher than patients who had no missing data. Because mixed-model analyses are based on the assumption of missing data being randomly

188

F. Falkenstro¨ m

distributed, a dummy variable coding missing data as 1 and no missing data as 0 was included in mixedmodel analyses for the GSI and IIP total mean.


Length of therapy. The mean length of therapy for those who completed treatment was 11 months (SD 8, Mdn 7) from intake to termination. The mean number of sessions attended was 23 (SD 19, 95%CI: 18.427.0, Mdn 17, range 6121). Thus, the length of therapy was very varied, but both the mean and median lengths were considerably longer than what Hansen et al. (2002) found for both regular mental health centers and clinical trials. Still, although most therapies were time unlimited, the median therapy consisted of 17 attended sessions, which can be considered within the range of brief therapy. Mean group change at termination and follow-up. Three separate mixed-model analyses were conducted, one for each dependent variable. For all three dependent variables, the mixed models included as fixed effects the categorical variables time (before treatment, at termination, and 1-year followup) and missing data (missing or no missing) as well as treatment length (continuous covariate). In addition to main effects of these three variables, the Time Missing Data interaction was included because exploratory analyses indicated such an interaction. When analyzing the GAF, two extreme outliers with GAF values at or below 20 were removed from the analysis. Patients with such low GAF values are highly uncommon in an outpatient setting with only self-referred patients, and should they come they are not likely to be judged suitable for outpatient psychotherapy. Most likely, these GAF ratings were not correct. The mixed model with GAF as a dependent variable showed statistically significant

main effects of time, F(1, 95) 136, p B.001, and treatment length, F(1, 97) 4.9, p .03. For both GSI and IIP, one extreme outlier was removed from the analyses. This was a patient who had five appointments altogether, was judged psychotic, and was referred for early intervention for first-time psychosis. The mixed model with GSI as a dependent variable showed a statistically significant main effect of time, F(2, 128)56, pB.001. The main effect of treatment length was close to significance, F(1, 72)3.5, p.07, while the main effect of missing data was not, F(1, 84)1.8, p.19. Finally, the TimeMissing Data interaction was statistically significant, F(2, 128) 4.2, p .02. The mixed model with IIP as a dependent variable showed a statistically significant main effect of time, F(2, 117) 34, p B.001, and treatment length, F(1, 72) 7.1, p .01, and also a statistically significant Time Missing Data interaction, F(2, 117) 3.3, p .04. However, the main effect of missing data was nonsignificant, F(1, 82) 1.7, p.19. Table II shows estimated marginal means and post hoc comparisons between different time points and effect sizes based on estimated marginal means from the mixed-model analyses. As can be seen from Table II, there were statistically significant and quite large differences between treatment start and termination for all primary outcome measures and no change between termination and follow-up. Table III shows change in SCL-90 subscales from therapy start to termination and follow-up. All subscales on the SCL-90 improved to a statistically significant degree, although effect sizes at termination varied between a small effect on hostility, medium effects on phobic anxiety, paranoid ideation, psychoticism, somatization and interpersonal sensitivity, and large effects on obsessivecompulsive symptoms, depression, and anxiety.

Table II. Estimated Marginal Meansa and Effect Sizes for Primary Outcome Measures (Sample 2, Intent-to-Treat Analysis) 95% Confidence interval Intake (Time 1)

Termination (Time 2)

Follow-up (Time 3)

Scale

n

M

Lower

Upper

M

Lower

Upper

M

Lower

Upper

GSI IIP GAF

74 76 99

1.40 1.29 56.6

1.28 1.19 55.1

1.53 1.39 58.1

0.70 0.89 64.8

0.56 0.78 62.8

0.85 1.01 66.8

0.72 0.93 *

0.54 0.80 *

0.90 1.06 *

Effect size

GSI IIP GAF

Time 12 1.29*** 0.90*** 1.11***

a Based on mixed model, including covariates. ***p B.001 (two-tailed).

Time 13 1.27*** 0.82*** *

Time 23 ns ns *

Psychotherapy for young adults in routine practice

189

Table III. Subcale means and standard deviations for Symptom Checklist-90 and Inventory of Interpersonal Problems Subscales Before Treatment and at Termination and 1-Year Follow-Up and Levels of Statistical Significance (Sample 2, Completers)


Scale

Pre-tx (T1)

Termination (T2)

t (T1T2)

1-year FU (T3)

t (T1T3)

Symptom Checklist-90 Somatization ObsessiveCompulsive Interpersonal Sensitivity Depression Anxiety Hostility Phobic Anxiety Paranoid Ideation Psychoticism Additional items

1.1 1.6 1.4 2.0 1.6 0.8 0.8 1.3 0.9 1.5

(0.7) (0.8) (0.9) (0.9) (0.8) (0.8) (0.8) (0.9) (0.7) (0.8)

0.6 0.8 0.8 1.0 0.8 0.5 0.4 0.6 0.4 0.8

(0.5) (0.6) (0.7) (0.7) (0.6) (0.5) (0.6) (0.6) (0.4) (0.6)

5.4*** 6.5*** 7.1*** 7.3*** 7.0*** 3.0** 4.6*** 5.6*** 7.7*** 6.9***

0.5 0.9 0.8 1.0 0.7 0.3 0.4 0.5 0.4 0.8

(0.5) (0.8) (0.7) (0.8) (0.6) (0.4) (0.5) (0.5) (0.4) (0.8)

5.0*** 5.0*** 4.8*** 5.9*** 5.5*** 4.5*** 4.1*** 5.2*** 4.0*** 5.6***

Inventory of Interpersonal Problems Domineering Vindictive Cold Socially Avoidant Nonassertive Exploitable Overly Nurturant Intrusive

0.6 1.0 1.0 1.5 1.7 1.7 1.6 1.0

(0.6) (0.7) (0.8) (0.9) (0.8) (0.7) (0.8) (0.7)

0.4 0.7 0.7 1.0 1.2 1.1 1.1 0.7

(0.4) (0.6) (0.6) (0.8) (0.7) (0.6) (0.6) (0.5)

3.2** 4.4*** 3.4** 5.0*** 4.7*** 5.3*** 4.6*** 3.7**

0.4 0.6 0.7 0.9 1.2 1.2 1.1 0.7

(0.4) (0.6) (0.7) (0.9) (0.8) (0.7) (0.7) (0.6)

2.6* 4.3*** 3.7** 4.8*** 5.5*** 4.1*** 4.5*** 1.8

*p B.05. **pB.01. ***p B.001 (two-tailed).

All IIP subscales improved significantly (see Table III). Effect sizes at termination ranged from small on the Domineering, Vindictive, Cold, and Intrusive subscales and medium for the other scales. The largest effect was on the Exploitable subscale. Individual change and clinically significant improvement at termination. Table IV shows the proportion of reliable and clinically significantly changed patients at termination. Among the 48 patients who completed the SCL-90, 32 (67%) were reliably improved (RCI 1.96) at termination while the remaining 16 (33%) were unchanged (RCI between 1.96 and 1.96). None deteriorated reliably. Using the cutoff points for clinical significance described in the Method section, 35 (73%) were classified as having symptoms within a clinical range of severity before treatment. After treatment only 13 (27%) had symptoms within the clinical range. Based on Jacobson and Truax’s (1991) criteria for clinically significant improvement, of the 35 participants initially within a clinical range of severity, 20 (57%) both changed reliably and crossed into a nonclinical range of severity after treatment. These figures are almost identical to those reported by Hansen et al. (2002) for the average clinical trial. On the IIP, the proportion reliably improved was slightly lower; of 51 participants, 28 (55%) were reliably improved at termination, 22 (43%) were unchanged, and one (2%) deteriorated reliably. Using the cutoff point for clinical significance of

1.19 proposed by Woodward et al. (2005), 27 (53%) had interpersonal problems within a clinically significant range before treatment compared with 11 (22%) after termination. Of 27 participants initially within a clinical range, 15 (56%) improved to a clinically significant degree using Jacobson and Truax’s (1991) criteria, whereas one (4%) of the 24 initially within a normal range deteriorated and crossed into a dysfunctional range. On the GAF scale, 41 of 79 (52%) participants were reliably improved at termination. Before treatment, 71 of 79 (90%) belonged to a clinical group compared with 43 (54%) at termination. None deteriorated from a functional to a dysfunctional group. Twenty-two (31%) patients initially within a clinical range achieved clinically significant improvement. Discussion As predicted, results confirmed the conclusion of Hansen et al. (2002) that many of the patients who seek a community mental health center attend fewer sessions and do not achieve reliable and clinically significant improvement to the same degree as patients in RCT studies. When all patients who came to the YACC were studied as if they were all psychotherapy patients, it turned out that a majority of patients stayed only for a few sessions and that they did not improve as much as patients in clinical trials usually do. The reliable and clinically

190

F. Falkenstro¨ m

Table IV. Psychotherapy Patients (Sample 2) Achieving Reliable and Clinically Significant Change at Termination Variable

SCL-90

IIP

GAF

Reliable improvement No change Reliable deterioration

67% (32/48) 33% (16/48) 0% (0/48)

55% (28/51) 43% (22/51) 2% (1/51)

52% (41/79) 48% (38/79) 0% (0/79)

Proportion with clinically significant symptoms Pretherapy Posttherapy Clinically significant improvement Clinically significant deterioration

73% 27% 57% 0%

53% 22% 56% 4%

90% 54% 31% 0%

(35/48) (13/48) (20/35) (0/13)

(27/51) (11/51) (15/27) (1/24)

(71/79) (43/79) (22/71) (0/5)


Note. SCL-90Symptom Checklist-90; IIP Inventory of Interpersonal Problems; GAFGlobal Assessment of Functioning.

significant improvement rates in Sample 1 were in the lower end of those reported by Hansen et al. (2002).4 However, when only patients who had agreed on a psychotherapy contract were studied, results were much better. The young adult patients studied were substantially improved after comparatively brief psychodynamic psychotherapy. The largest effects were shown on global functioning and psychiatric symptoms, but interpersonal problems, which have been shown in previous research to take a longer time to change (Huber, Henrich, & Klug, 2007), also improved considerably. This may indicate that the IIP is less stable in young adults, because personality is still in formation. Effect sizes on all measures were well on a level with mean effects of RCT studies (Hansen et al., 2002). They were also as good as or better than those reported in metaanalyses of psychotherapy (Lambert & Ogles, 2004), and on par with other naturalistic studies of psychodynamic psychotherapy for young adults using the same measures (e.g., Lindgren, Werbart, & Philips, in press). When studying only patients who started psychotherapy proper, the median length of therapy was 17 attended sessions, far more than the three to eight sessions reported by Hansen et al. (2002) for mental health centers in the United States. Also, the rate of clinically significant improvement on the SCL-90 was almost identical to the rates shown by Hansen et al. (2002) for the average clinical trial and considerably higher than rates for mental health centers in the same study. Although there may be many important differences between the conditions at community mental health centers in the United States and Sweden, this result, in combination with those of Stiles, Barkham, Mellor-Clarke, et al. (2008), indicates that it may be premature to conclude that therapy in clinical practice in general is too brief to achieve meaningful change.

The center’s statistics also show something important about the way patients approach psychotherapy in a community sample. As many as one of five (17% in Sample 1 and 21% in Sample 2) patients presenting at the center had attended at least one session at the center before. Patients who drop out before beginning therapy may seek the center again later, and sometimes they may be more motivated for change the second time. Among clinicians working in community mental health centers, it is well known that many patients who seek help are not actually motivated for change but may be at a later date. It is also well known clinically that ambivalence about help seeking is especially pronounced among adolescents. Therefore, it is possible that the figures in the present study would look different in a clinic working with only adult patients. The strength of this study is the use of clinically representative patients, therapists, and treatments as well as the use of the total sample of patients seeking a community mental health center. However, as with all naturalistic studies, the present study has several methodological limitations. The absence of a control group makes causal interpretations uncertain. Although the inclusion of treatment length as a covariate in outcome analyses controls for the variation in time, improvements could be, and to some degree most likely are, caused by common factors and extratherapeutic events. Another limitation is that diagnoses and GAF ratings are done by the treating therapists without formal reliability checks. However, routine clinician GAF ratings have in previous research been shown to be reliable and valid (So¨ derberg et al., 2005), and in the present study they show the same pattern of change as patient self-report measures. A third limitation is the attrition of self-report measures, which, although common in studies in clinical practice (e.g., Stiles, Barkham, Mellor-Clarke, et al., 2008), could introduce bias in the data even though

Psychotherapy for young adults in routine practice attempts to test for this possibility showed no such bias. In conclusion, although the results of the study indicate that when studying patients in clinical practice who actually start treatment psychotherapy may be as effective as psychotherapy in a typical research trial, the possibility of generalizing from the results is limited. The center from which the data were drawn as well as the patient category (selfreferred young adults) limit generalizability to centers with similar orientations and patient categories. Also, methodological shortcomings of the study (i.e., attrition rates, absence of control group, diagnostic and outcome evaluations) means that results must be interpreted with caution. Notes


1

2

3

4

The reason for excluding patients with only one session is that it is impossible to calculate outcome with only one measurement. Also, the same procedure was used in the comparison studies (Hansen et al., 2002; Hansen & Lambert, 2003). The two samples used in the study are not independent, but Sample 2 is included within Sample 1. Dropout is difficult to define with this population because the length of therapy is often difficult to predict in advance. Therefore, we have chosen to define as dropouts only those patients who suddenly stopped coming for therapy without discussing this with their therapist. All others are considered treatment completers. Hansen et al. (2002) used the OQ-45 as a dependent variable. It is clear from the results in Sample 2 that the GAF scale yields lower estimates of clinically significant change than the SCL-90 and IIP (see Table IV). Therefore, the comparison with the results of Hansen et al. (2002) is a bit unfair, but nevertheless it is obvious that Sample 1 as a whole did not do as well as Sample 2.

References Abbas, A. A. Hancock, J. T. Henderson, J. & Kisely, S. (2006). Short-term psychodynamic psychotherapies for common mental disorders. Cochrane Database of Systematic Reviews, Issue 4, Art. No.: CD004687. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Barkham, M., Connell, J., Stiles, W. B., Miles, J., Margison, F., Evans, C., et al. (2006). Doseeffect relations and responsive regulation of treatment duration: The good enough level. Journal of Consulting and Clinical Psychology, 74(1), 160167. Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 718. Derogatis, L. R., Lipman, R. S., Rickels, K., Uhlenhuth, E. H., & Covi, L. (1974). The Hopkins Symptom Checklist (HSCL): A self-report symptom inventory. Behavioral Science, 19, 115. Fridell, M. Cesarec, Z. Johansson, M. & Malling Thorsen, S. (2002). Svensk normering, standardisering och validering av symtomskalan SCL-90 [Swedish norms, standardization and validation of the symptom scale SCL-90]. Va¨ stervik, Sweden: National Board of Institutional Care.

191

Hansen, N. B., & Lambert, M. J. (2003). An evaluation of the dose-response relationship in naturalistic treatment settings using survival analysis. Mental Health Services Research, 5(1), 112. Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9, 329343. Horowitz, L. M., Alden, L. E., Wiggins, J. S., & Pincus, A. L. (2002). Inventory of Interpersonal Problems. Manual. Stockholm: Psykologifo¨ rlaget. Horowitz, L. M., Rosenberg, S. E., Baer, B. A., Uren˜ o, G., & Villasen˜ or, V. S. (1988). Inventory of Interpersonal Problems: Psychometric properties and clinical applications. Journal of Consulting and Clinical Psychology, 56(6), 885892. Huber, D., Henrich, G., & Klug, G. (2007). The Inventory of Interpersonal Problems (IIP): Sensitivity to change. Psychotherapy Research, 17(4), 474481. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 1219. Lambert, M. J., & Ogles, B. M. (2004). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed) (pp. 139193). New York: Wiley. Leichsenring, F., Rabung, S., & Leibing, E. (2004). The efficacy of short-term psychodynamic psychotherapy in specific psychiatric disorders: A meta-analysis. Archives of General Psychiatry, 61(12), 12081216. Lindgren, A., Werbart, A., & Philips, B. (in press). Long-term outcome and post-treatment effects of psychoanalytic psychotherapy with young adults. Psychology and Psychotherapy. Roth, A., & Fonagy, P. (2005). What works for whom? (2nd ed). New York: Guilford Press. Shadish, W. R., Navarro, A. M., Matt, G. E., & Phillips, G. (2000). The effects of psychological therapies under clinically representative conditions: A meta-analysis. Psychological Bulletin, 126(4), 512529. Shadish, W. R., Matt, G. E., Navarro, A. M., Siegle, G., CritsChristoph, P., Hazelrigg, M. D., et al. (1997). Evidence that therapy works in clinically representative conditions. Journal of Consulting and Clinical Psychology, 65(3), 355365. So¨ derberg, P. & Tungstro¨ m, S. (2006). The dose of sessions effect in psychiatric outpatient services. Unpublished manuscript, Department of Psychology, Umea˚ University, Umea˚ , Sweden. ˚ . (2005). ReliaSo¨ derberg, P., Tungstro¨ m, S., & Armelius, B.-A bility of Global Assessment of Functioning ratings made by clinical psychiatric staff. Psychiatry Services, 56(4), 434438. Stiles, W. B., Barkham, M., Connell, J., & Mellor-Clark, J. (2008). Responsive regulation of treatment duration in routine practice in United Kingdom primary care settings: Replication in a larger sample. Journal of Consulting and Clinical Psychology, 76(2), 298305. Stiles, W. B., Barkham, M., Mellor-Clark, J., & Connell, J. (2008). Effectiveness of cognitive-behavioural, person-centred and psychodynamic therapies in UK primary-care routine practice: Replication in a larger sample. Psychological Medicine, 38, 677688. Stiles, W. B., Barkham, M., Twigg, E., Mellor-Clark, J., & Cooper, M. (2006). Effectiveness of cognitive-behavioural, person-centred and psychodynamic therapies as practised in the UK National Health Service settings. Psychological Medicine, 36, 555566.

192

F. Falkenstro¨ m


Vinnars, B., Barber, J. P., Noren, K., Gallop, R., & Weinryb, R. (2005). Manualized supportive-expressive psychotherapy versus nonmanualized community-delivered psychodynamic therapy for patients with personality disorders: Bridging efficacy and effectiveness. American Journal of Psychiatry, 162(10), 19331940. Wampold, B. (2001). The great psychotherapy debate. London: Erlbaum. Weisz, J. R., Donenberg, G. R., Han, S. S., & Weiss, B. (1995). Bridging the gap between laboratory and clinic in child and

adolescent psychotherapy. Journal of Consulting and Clinical Psychology, 63(5), 688701. Westen, D. (1997). Divergences between clinical and research methods for assessing personality disorder: Implications for research and the evolution of axis II. American Journal of Psychiatry, 154(7), 895903. Woodward, L. E., Murrell, S. A., & Bettler, R. F. (2005). Stability, reliability, and norms for the Inventory of Interpersonal Problems. Psychotherapy Research, 15(3), 272286.